Advice for the PeopleSoft Oracle DBA

Table of Contents
Introduction...................................................................................................................................................................2
Hardware and Operating System Support.................................................................................................................2
Operating Systems vs. Hardware...........................................................................................................................2
Hardware Virtualization..........................................................................................................................................2
Oracle DBMS Versions and Patchsets.......................................................................................................................3
Certified Versions and Configurations.................................................................................................................3
Supported Version Guidelines...............................................................................................................................4
Oracle Limitations and Defects.............................................................................................................................4
Initialization Parameters...............................................................................................................................................4
"Required" Parameters............................................................................................................................................4
Parameters to be Avoided.......................................................................................................................................5
Parameters Often Confused...................................................................................................................................6
Parameters Used as Customizations.....................................................................................................................6
Support of Non-standard Configurations............................................................................................................7
Statistics Management..................................................................................................................................................7
Oracle Features Critical to Statistics Management..............................................................................................7
Recommendations Regarding Managing Statistics..............................................................................................9
Indexing Techniques...................................................................................................................................................11
General Concepts..................................................................................................................................................11
Access vs Filter Predicates....................................................................................................................................12
Key Ordering for Performance............................................................................................................................12
Table Row Ordering...................................................................................................................................................13
Optimizer Limitations................................................................................................................................................13
Useful Tools & References........................................................................................................................................14
SQLTXPLAIN (SQLT)........................................................................................................................................14
The pscbo_stats Package......................................................................................................................................14
AWR / StatsPack Report......................................................................................................................................14
PSPRCSRQST Batch Trigger..............................................................................................................................14
OS Watcher............................................................................................................................................................14
ORAchk Health Checks for the Oracle Stack (ORAchk)................................................................................15
How to Configure PeopleSoft for Maximum Availability (MAA)..................................................................15
Updates and Suggestions............................................................................................................................................15

20160112-AdviceForThePeopleSoftOracleDBA.odt 1 of 15
Introduction
This document is intended the Oracle DBA who seeks a "big picture understanding" regarding how to
support a PeopleSoft Application. It is a brief overview that contains advice from PeopleSoft Support
Install/Upgrade and CoE teams with links to relevant KM documents.
It is organized as a series of steps starting with comments related to hardware, and working "up" from
there, e.g. from hardware, operating systems, database versions and patchsets, initialization settings,
statistics, and so on.
This document was written with the anticipation that the reader is an Oracle DBA that is skilled in the
various aspects of implementation, administration, and tuning of the Oracle DBMS. Additionally, it will be
helpful for the reader to be fluent in PeopleSoft technologies and/or have access to PeopleSoft technical
staff that can modify settings to meet specific implementation needs.
The information contained in this document applies to Oracle databases used for PeopleSoft Applications
built on PeopleTools 8.50+, specifically on Oracle 10g, through Oracle 12c.

Hardware and Operating System Support

Generally speaking, PeopleSoft doesn't constrain hardware choices as long as the equipment is
appropriately configured and sized for the role it plays in a given architecture. The intent is to allow
significant flexibility when designing a PeopleSoft system.

Operating Systems vs. Hardware

Within the PeopleTools Install/Upgrade Support team a phrase is used, "...we support Operating Systems, not
hardware..." emphasizing that, on the whole, hardware choices are not as critical as those regarding
Operating Systems. For example, if a given variant of 64-bit Linux is supported, the choice regarding the
hardware vendor's 64-bit machine is not the primary concern.
Any limitations that do exist should be listed on the "Certifications" tab in My Oracle Support. It is is wise
to confirm any hardware or Operating System choice there, especially during an upgrade, as the
requirements may change over time.
Naturally, when a database server plays more than one role in the PeopleSoft Architecture, e.g. when the
process scheduler is installed on the database server, there may be settings and constraints in the Operating
System necessary to support that particular, blended role. When this occurs, the various requirements of all
of the components need to be reconciled. When a server supports only the database, no significant
hardware or Operating Systems restrictions are imposed by PeopleSoft other than those imposed by the
DBMS vendor.
Useful Resource

• Certifications tab on My Oracle Support

Hardware Virtualization
Hardware Virtualization plays an increasingly larger role in the modern implementation architecture.
Generally, virtual infrastructures are very capable, and overall performance is quite good. But in the cases
where there is impact, it can be significant. In those cases, Oracle Support may redirect you to the vendor

20160112-AdviceForThePeopleSoftOracleDBA.odt 2 of 15
that provides support for those products.
As an example, very small delays in a Production infrastructure can be problematic. For example, some
applications iterate through many small transactions, e.g. HCM's Payroll Calculation, so even very small
delays, e.g. sub-millisecond, caused by CPU virtualization, power management, or I/O contention, can
accumulate into a significant performance impact.
Another common issue is the performance of shared, or SAN-based storage. When compared to local,
dedicated storage, the actual performance can be quite different. When monitored with tools that provide
“average response time,” the performance of the I/O systems can be reported as adequate, but the actual
throughput and responsiveness be quite poor. In those cases, using the AWR Report's Wait Event
Histogram to evaluate the various read/write events can be very useful.

Oracle DBMS Versions and Patchsets

Decisions related to DBMS software version and patchset are the next considerations in implementing the
Oracle database architecture. These choices form the basis of many recommendations in the following
sections.

Certified Versions and Configurations

PeopleTools Development provides information related to certified versions and configuration
requirements. Version information is provided in at least three places:

• Certified Combinations: Certain combinations of the various component parts, including DBMS
are certified for support. The "Certifications" tab in My Oracle Support lists these combinations.

Note: ensure that the versions of the various components being implemented, e.g. SQLNet, OCI, etc. "match" and
are supported before contacting Oracle Support.

• Oracle DBMS Patches and Settings: For the various patchsets, some critical patches and
initialization parameters have been identified as "required" for proper operation of the Oracle
DBMS. These patches and settings are listed on the Note entitled "Required Interim Patches for
the Oracle Database with PeopleSoft" (Doc ID 1100831.1). Be sure to review the index to
understand which tabs apply, since there are typically two that apply: one for patches and once for
settings.

• Operating System Support: The patches required for support of the component parts of the
PeopleSoft Infrastructure, e.g. WebLogic, Tuxedo, COBOL, Operating Systems, are listed on the
Note entitled "Operating System, RDBMS & Additional Component Patches Required for
Installation PeopleTools - Master List" (Doc ID 756571.1).
Useful KM Documents:

• "Certifications" tab in My Oracle Support

• Required Interim Patches for the Oracle Database with PeopleSoft" (Doc ID 1100831.1)
• Operating System, RDBMS & Additional Component Patches Required for Installation
PeopleTools - Master List" (Doc ID 756571.1)

20160112-AdviceForThePeopleSoftOracleDBA.odt 3 of 15
Supported Version Guidelines
A given PeopleTools release is certified on a given Oracle DBMS release and patchset. As a rule, within that
Oracle release, the patchset and all later patchsets are supported as well, unless otherwise indicated.
For example, PeopleTools 8.51 is initially certified on Oracle releases 10gR2, 11gR1, and 11gR2 - patchsets
10.2.0.4, 11.1.0.7, and 11.2.0.1, respectively. All patchsets to each release on or after those certified patchsets
are supported as well.
Naturally, this broad support of the Oracle DBMS certainly does not imply that all PeopleSoft
Applications will perform equally well on all versions and patchsets of the Oracle DBMS. Some
characteristics of a given release or patchset can impact the PeopleSoft Application significantly.
PeopleTools Support encourages customers to use patchsets that are as current as practical.

Oracle Limitations and Defects

Not all limitations and defects in the Oracle DBMS impact PeopleSoft Applications. But there are a few
issues that over time have impacted PeopleSoft applications more than others. One deserves special
mention: Function-Based Indexes.
Function-Based Indexes became common in PeopleSoft Applications when, in PeopleTools 8.48 and later,
many were constructed to support range queries Those indexes included the DESC keyword.
Unfortunately, their use opened PeopleSoft implementations to some of the side effects associated with
Function-Based Indexes. Moreover, due to the complexities of many PeopleSoft Application queries, their
benefits were not widely appreciated. In most situations removing the DESC keyword has been quite
helpful by avoiding the side effects of Function-Based Indexes without causing significant performance
impacts.
The PeopleTools strategy regarding the use of descending indexes has changed to avoid the use of the
DESC clause in Oracle 12c. This approach is required for implementations on all Oracle 12c databases and
is optional, appropriate, and supported for existing Oracle 10g and 11g. In any implementation using
PeopleTools 8.54 or later, Application Designer will not emit the DESC clause.
Useful KM Document:
• Removing Descending Indices for PeopleSoft Databases (Doc ID 1909646.1)

Initialization Parameters
Initialization parameters are critical to decisions that the Oracle optimizer makes. This section describes
some of the more significant settings and their impact on a PeopleSoft Application.

"Required" Parameters
Related to currently supported Oracle versions, a few initialization parameters have been indicated as
"required" or “recommended” by PeopleTools Development. While the My Oracle Support Certification
documentation is the authoritative repository of these, this section highlights these parameters and briefly
explains how they came to be required.

• _gby_hash_aggregation_enabled=false - Oracle 10g implemented a high-performance, hash-based

sorting algorithm for use when processing DISTINCT clauses. This new algorithm does not
guarantee that the rows will be ordered where the previous algorithm did sort the rows.

20160112-AdviceForThePeopleSoftOracleDBA.odt 4 of 15
 Many PeopleSoft applications were written with the expectation of the sorted rows and will not
behave properly without this sorting. So, on the Oracle DBMS, the legacy algorithm should be
used. So, it is required that this value be disabled in all Oracle Databases running PeopleSoft
Applications.

• _unnest_subquery=false - During internal testing with the default value, performance degradations
occurred in on-line processing. There were also some occasional errors creating views. It is required
that the default value be disabled in all Oracle Databases running PeopleSoft Applications. (There is
flexibility in the use of this parameter explained later in this document.)

• optimizer_adaptive_features = FALSE - After upgrading to Oracle Database 12c, many PeopleSoft

customers have noticed overall performance degradation which is related to Oracle 12c Optimizer
Adaptive Feature (OAF). It is recommended that this value be disabled in all Oracle Databases
running PeopleSoft Applications.

• _optimizer_skip_scan_enabled=FALSE - PeopleSoft SQL Queries running on Oracle 12c Exadata

uses index skip scan instead of Full Scan and can run considerably slower than the Full Scan
despite lower cost evaluation in the execution Plan. It is recommended that this value be disabled in
all Oracle 12c Exadata Databases running PeopleSoft Applications.

• _fix_control='14033181:0' PeopleSoft SQL Queries were returning wrong results in Oracle 12c
when the Query has bitmap conversion enforced in the Execution Plan. It is recommended that this
value be disabled in all Oracle Databases running PeopleSoft Applications.

• nls_length_semantics=char. This setting is required for Unicode databases which are used in most
current implementation running PeopleSoft Applications 9.0 and later. See Installation Guide,
Section "Preparing for Installation" for additional details.

• o7_dictionary_accessibility - This parameter was initially added to allow functionality needed for the
triggers implemented by PeopleSoft in PeopleTools 8.48 to support PeopleSoft mobile applications
framework. In May 2010, the requirement to use this parameter was eliminated and superseded
with a requirement to implement two GRANTs. There are some prerequisites before this is a
required setting.
Useful KM Document:

• Required Interim Patches for the Oracle Database with PeopleSoft (Doc ID 1100831.1)

Parameters to be Avoided
It is not unusual to see certain settings “creep” into a PeopleSoft system over time, typically as
workarounds to address either performance issues or perceived optimizer limitations. The following
parameters heavily skew the CBO's calculations. With rare exceptions - typically only as a workaround to a
CBO defect or design limitation - we recommend that only the default values should be used for these
parameters.

• cursor_sharing - default: EXACT - The cursor_sharing parameter deserves special attention as its
global misuse can have a deleterious impact on stability and performance of a PeopleSoft
Application due in part to the size and skew of the typical data set. The default setting of

20160112-AdviceForThePeopleSoftOracleDBA.odt 5 of 15
 cursor_sharing=exact and is the only value should be used with PeopleSoft Applications. The setting
of cursor_sharing=similar has been deprecated by Oracle and should not be used in any context for
PeopleSoft Applications.

• optimizer_index_cost_adj - default: 100 - This setting has been used in earlier versions of Oracle DBMS
to globally coerce the CBO to prefer the use of indexes. The continued global use of this parameter
in current versions of Oracle is not recommended since it can have a profoundly negative impact
on the overall performance stability of the database.

• optimizer_index_caching - default: 0 - This setting has also been used in earlier versions of Oracle
DBMS to globally coerce the CBO to prefer the use of indexes. The continued global use of this
parameter in current versions of Oracle is not recommended since it can have a profoundly
negative impact on the overall performance stability of the database.

• db_file_multiblock_read_count - default: unset, dynamically calculated by the DBMS - The default value for
this parameter is recommended in Oracle 10g and later.

• various "underscore" parameters - The use of underscore parameters, other than those listed in the
previous section, should be avoided unless at the specific direction of Oracle Server Tech Support
for the purpose of addressing a specific bug. Care must be taken a.) to apply them as “locally” as
possible, e.g. at the statement or session level and b.) to remove them as soon as the database is
upgraded to a version/patchset that addresses the bug.

Parameters Often Confused

The following two parameters are used from time to time and deserve mention simply because they are so
often confused. Only in rare cases is their non-default use appropriate.

• optimizer_features_enable - default: {current DBMS version}. This setting allows customers to use a newer
DBMS version or patchset, but ask the CBO to "behave" more like a past version. It can be used as
a temporary workaround for bugs discovered after implementing a specific version of Oracle.

Unfortunately, use of the setting often lingers beyond the point where it is needed, unnecessarily
limiting features available to the optimizer. Unless Oracle Server Tech Support specifically indicates
that this setting be used as a temporary measure, leaving this value unset is highly recommended.

• compatible: This setting impacts the file storage features available to a given release. Typically, it is
used to provide for a "back out strategy" during an DBMS upgrade. While PeopleSoft uses some
feature impacted by this setting, e.g. locally managed tablespaces, generally it is not significant in
PeopleSoft implementations and does not need to be set.

Parameters Used as Customizations

• _unnest_subquery=true: If there is a specific use case where the use of the query unnesting feature is
necessary, enabling this parameter is allowed, i.e. as either a hint or as an altered session setting.
The use of this parameter will be supported as a customization, and may need to be be removed
during the diagnostic process of a Service Request.

20160112-AdviceForThePeopleSoftOracleDBA.odt 6 of 15
Support of Non-standard Configurations
As problems are reported to Oracle Support, one of the first lines of inquiry is to remove non-standard
configuration parameters. If is found that a "required" parameter is in fact not set, or that an extraneous
setting has been used other than those required by Oracle Support, be prepared that to provide a test case
with the parameters reset to the required values.
But, any change to initialization parameters can have far reaching consequences. For example, changes to
the settings may produce side effects, even if the change is to reset it a required or recommended state.
Administrators should be able to adequately test any global change for regressions before deploying it in a
production environment.
Useful KM Documents:

• From 10gR2, HASH UNIQUE Operation Returns Results in UNSORTED ORDER by Default
(Doc ID 341838.1)
• Required Interim Patches for the Oracle Database with PeopleSoft (Doc ID 1100831.1)
• Operating System, RDBMS & Additional Component Patches Required for Installation
PeopleTools - Master List (Doc ID 756571.1)
• ANNOUNCEMENT: Deprecating the cursor_sharing = 'SIMILAR' setting. (Doc ID 1169017.1)
• E-ORA: How to convert an Oracle Non-Unicode Database to a Unicode Database (Doc ID
1437384.1)

Statistics Management
Traditionally, PeopleTools Development has provided guidance to assist in the gathering of statistics that
have come in the form of defaults, samples, and documents. Final decisions regarding statistics gathering
have been left to the customer.
In spite of the maturity of the Oracle CBO, some characteristics have emerged over time that make the use
of delivered "defaults" of PeopleTools and the Oracle DBMS problematic for PeopleSoft Applications.
This section will explore the more significant anomalies and then provide recommendations to mitigate
their impact.

Oracle Features Critical to Statistics Management

The following Oracle features impact PeopleSoft Applications, and are described so that their impact can
be better understood.

AUTO_SAMPLE_SIZE
In Oracle10g, the default AUTO_SAMPLE_SIZE produced samples that were too small to be effective.
To that end, Oracle Support recommends that, in Oracle10g, the sample size be set to 100% whenever
practical, or based on the following schedule that is table size-dependent:
100% for tables < 1M rows
30% for tables up to 10M rows
10% for tables up to 100M rows
3% for tables up to 1B rows
1% for tables > 1B rows

In Oracle 11g and later, the automatic sampling algorithm was so completely changed that the use of

20160112-AdviceForThePeopleSoftOracleDBA.odt 7 of 15
AUTO_SAMPLE_SIZE is now appropriate and recommended. Studies have shown that the quality of
statistics gathered approaches that of a 99% sample size yet provides, the speed of a 10%, or less, sample
size. One exception to the use of AUTO_SAMPLE_SIZE exists when a column has histograms and the
data in the column is very skewed. In that case, it may be better to explicitly collect with a large sample size.
To summarize, the recommended sample size to be used is as follows:

• In Oracle 11g and 12c, use AUTO_SAMPLE_SIZE as the default.

• In Oracle 10g, avoid the use of the AUTO_SAMPLE_SIZE in all cases. Use a sample size of 100%
where practical. Where size prohibits a 100% sample, use the schedule above to improve
performance.

Histograms
With the evolution of the Oracle 11g and Oracle 12c optimizers, histograms have more broadly become
accepted as beneficial and their use is now recommended to enhance plan stability and performance. When
gathering statistics on Oracle11g or later, using a METHOD_OPT of AUTO is appropriate.
As a rule, avoid the default use of histograms on Oracle 10g. Use them only in specific situations where it
is known that they address a specific column and there is no side effect with their use. When gathering
statistics on Oracle 10g, use a METHOD_OPT that includes 'SIZE 1', which will prevent the generation
of histograms.
As mentioned previously, on any Oracle version, when gathering histograms for large objects with very
skewed data, consider increasing the sample size to improve accuracy of the histograms.
To summarize, the use of histograms is as follows:

• In Oracle 11g and Oracle 12c, use METHOD_OPT of AUTO to automatically gather histograms
as needed.

• In Oracle 10g, avoid the use of histograms unless a specific use case merits their use.

Delayed Invalidation of Statistics

There is a delay that occurs between the gathering of statistics on a table and the invalidation of plans
associated with that table. The delivered syntax from PeopleSoft does not override this delay and can be
the cause of unpredictable run times. It is recommended that NO_INVALIDATE be set to FALSE when
gathering statistics.
• E-ORA How to alter the syntax that PSDDLMODEL issues when %UpdateStats is used (Doc ID
1537099.1)

Automatic Statistics Collection Default Values

Oracle databases are delivered with either a job or an AUTOTASK that schedules the
dbms_stats.gather_schema_stats() procedure. Because of the peculiarities of PeopleSoft applications running on
Oracle, some problems arise with these default values, particularly in Oracle 10g.

• Histograms: The procedure will automatically generate histograms based on whether a column has
been used as a predicate in SQL statement history. As mentioned earlier, this is problematic in

20160112-AdviceForThePeopleSoftOracleDBA.odt 8 of 15
 Oracle 10g, but not an issue in Oracle 11g and later. But the delivered syntax delivered in
PeopleTools does not collect statistics, so depending on which method is used, statistics may or
may not exist.

• Sample Size: The default sample size used by the procedure is AUTO. As mentioned earlier, this is
problematic in Oracle 10g, but not an issue in Oracle 11g and later. But the delivered syntax
delivered in PeopleTools sets the sample size to 1 percent, so depending on which method is used,
statistics be quite different.

• Dynamic Sampling: There is insufficient control over what tables should not have statistics
gathered, i.e when statistics are deleted to invoke dynamic sampling. To prevent the procedure
from collecting those missing statistics, it is typical for a DBA to lock the statistics on that table.
But if the statistics are locked, it will likely cause a PeopleSoft batch processes to ABEND if an
update of the statistics is attempted from the program.
The pscbo_stats package was written to overcome these limitations, and is recommended as an
enhancement for the default automatic statistics collection.

Recommendations Regarding Managing Statistics

There are a few recommendations to address most common statistics-related issues with PeopleSoft
Applications.

Make changes to the default DDL.

The delivered DDL for the %UpdateStats() includes FOR ALL INDEXED COLUMNS, does not include
the clause NO_INVALIDATE=FALSE, and uses a sample size of 1. Changing all of these will
significantly improve the quality of statistics.
The recommended syntax would use FOR ALL COLUMNS and NO_INVALIDATE=FALSE.
Additionally, in Oracle 11g and later, including the AUTO_SAMPLE_SIZE and
METHOD_OPT=AUTO is appropriate as well. Review the KM listed below which explains how to edit
the default DDL, which include a suggested syntax as well.
Useful KM Document:

• E-ORA How to alter the database preference for NO_INVALIDATE for PeopleTools
applications (Doc ID 1537099.1)

Implement pscbo_stats package

The pscbo_stats package is a tool for implementing dynamic sampling as well as embracing the preferred
settings outlined in this document. It is a joint venture between PeopleSoft CoE and Oracle Server Tech
CoE.
The pscbo_stats package provides a consistent, reliable way to maintain database statistics for PeopleSoft
Applications. It includes methods to gathering stats at either the schema- or table-level, and automatically
implements dynamic sampling for known volatile tables, e.g. temporary storage tables in Application
Engine and COBOL.
This tool deserves special mention since it makes the use of %UpdateStats() largely irrelevant. Instead of
relying on the program to update the statistics on a temporary table, Oracle's Dynamic Sampling is used.

20160112-AdviceForThePeopleSoftOracleDBA.odt 9 of 15
When the %UpdateStats() is called, it is simply ignored.
Useful KM Document:

• KM document "pscbo_stats - Improving Statistics in Oracle RDBMS for PeopleSoft Enterprise

(Doc ID 1322888.1)"

Use %UpdateStats metaSQL

PeopleTools provides a PeopleCode metaSQL construct called %UpdateStats that generates platform-
specific SQL to gather statistics. %UpdateStats references are commonly used in Application Engine and
COBOL programs that significantly alter the “shape” of the data in a given table. When that happens,
having the application update the statistical metadata for the table can improve the decisions that the
optimizer makes.
While the delivered applications put %UpdateStats in strategic locations, in some implementations it may
not be sufficient to produce consistent, adequate performance. In those cases, manually adding
%UpdateStats to the application can be beneficial. That change would be considered a customization, but
can be extremely effective in enhancing performance and run-time stability when properly used.

Gather data dictionary statistics

The PeopleSoft Application database has many objects, numbering well in the tens-of-thousands. Each of
those objects is defined in the Oracle data dictionary, a set of tables that the optimizer is constantly
querying to validate the existence of tables, columns, indexes, constraints, etc.
Neglecting to update the statistics on this data dictionary following major DDL changes is a common
oversight. After building, or upgrading, a PeopleSoft database, it is wise to gather the Oracle dictionary
statistics.
The following example script shows the necessary syntax, e.g. (run as SYSDBA):
SET SERVEROUT ON TRIMS ON LINES 1000;
SPO gather_dict_stats.log;
exec dbms_stats.gather_dictionary_stats(estimate_percent=>100,
method_opt=>’FOR ALL COLUMNS SIZE 1’);
QUIT;

Gather Workload System Statistics

Current workload statistics can be very useful for the optimizer (especially as of Oracle 11g), but are quite
often overlooked since the defaults often work quite well. If you have reason to believe that the default
values are inadequate, or if for some reason the behavior of your system changes from time to time,
gathering workload statistics may be appropriate.
If you choose to gather the workload statistics, be sure to refresh them after any major DBMS upgrades or
changes, or when significant hardware or OS changes are performed. Typically, the data is collected during
a one-to-two hour window of "normal" system activity, but that sample varies depending on several
implementation factors.
Be prepared to re-gather workload statistics each time database usage patterns changes significantly, e.g.
after a database upgrade or when there is significant change on the System affecting CPU or IO metrics.
As with any global change, manage the risk associated with regressions by performing adequate testing.

20160112-AdviceForThePeopleSoftOracleDBA.odt 10 of 15
Since this data is being collected by taking two snap shots of the metadata during "normal" system activity,
the remote possibility for an observable performance impact exists. This impact is temporary and is usually
quite small. The benefits of having valid workload statistics normally outweigh the temporary costs of
gathering them.
The command to start the baseline sample is:
exec DBMS_STATS.GATHER_SYSTEM_STATS('START');

The command gather the ending sample is:

exec DBMS_STATS.GATHER_SYSTEM_STATS('STOP');

Is is extremely important to review the results after gathering workload system statistics. Confirm all values
are within reasonable ranges with the following sample script (run as SYSDBA):
SET SERVEROUT ON TRIMS ON LINES 1000;
SPO display_workload_stats.log;
SELECT pname, pval1
FROM sys.aux_stats$
WHERE sname = 'SYSSTATS_MAIN';
QUIT;

One defect deserves special mention. Typical values for SREADTIM and MREADTIM are between 1
and 100, representing a value in milliseconds. If they happen to be 10,000 times greater that value, you may
be experiencing the bug described in Note 9842771.8.
Useful KM Document:

• Bug 9842771 - Wrong SREADTIM and MREADTIM statistics in AUX_STATS$ (Doc ID

9842771.8)

Indexing Techniques
Indexing strategies abound and are, in many ways, an art form for the skilled application tuner. While
improving an individual access path is somewhat mechanical, building an index that will assist several
access paths through various data sets without interfering with application performance can require
significant skill. These trade offs should always be made in an informed manner.

General Concepts
While this section does not intend to exhaustively advise how to perform index tuning, it does include
some valuable "lessons learned" for the DBA doing index tuning for PeopleSoft Applications.

• Index Uniqueness - PeopleSoft Applications use unique indexes to enforce data integrity. Although
re-ordering the keys in a unique index will not impact the integrity of a PeopleSoft Application, it is
imperative that the presence of the keys in a unique index not be altered in any way. Generally,
modifying the unique indexes is not needed, even for performance reasons.

• Non-unique indexes - Non-unique indexes are algorithmically built by PeopleTools Application

Designer based on the application developers' choices when defining record structures. As such,
they represent an "educated guess" regarding how data will be accessed.

20160112-AdviceForThePeopleSoftOracleDBA.odt 11 of 15
 Other than for performance reasons, there is nothing inherent in the PeopleSoft Application
requiring these indexes to remain, or if they remain, to be unmodified. Be sure to use Application
Designer to update the PeopleTools Data Dictionary when modifying these indexes.

• PeopleSoft Data Dictionary - Remember that PeopleTools maintains its own data dictionary, distinct
from the Oracle data dictionary, that needs to be updated with any changes made to indexes so that
those changes can be preserved during an Application upgrade.

• Function-Based Indexes - The DESC index is an example of a Function-Based Index that may add
enough optimizer complexity and storage requirements that its benefits are offset. With Oracle 12c
databases, PeopleTools requires the disabling of DESC indexes, and encourages disabling them in
previous versions of the database. For details, see the KM documented listed at the end of this
section.

• Equality and equi-join predicates should precede scanned predicates in indexes - "Best practices" used to
suggest that highly selective values should always lead an index. But that is not always true, and in a
PeopleSoft Application where range scanning is common, this misunderstanding can cause
significant performance issues. The following sections provide an introduction to this concept.
Useful KM Documents

• Removing Descending Indices for PeopleSoft Databases (Doc ID 1909646.1)

Access vs Filter Predicates

Before adding or changing indexes, understand the impact of access vs filter predicates and projected keys.
Generally, when you are building indexes, you want keys that are access predicates to precede keys that are
filter predicates, followed lastly by relevant projected keys.
There are many articles available through a Web search and Published books, and understanding those
concepts well is critical to effective index tuning. For example:
• https://asktom.oracle.com/pls/apex/f?p=100:11:::NO:RP:P11_QUESTION_ID:7807480400346035212

Key Ordering for Performance

When an index is added to support a query with an equality (and/or equi-join) predicate and a range
predicate, it is typically better to locate the keys used in the equality (and/or equi-join) predicate before the
keys used in the range predicate. An example will help illustrate this suggestion.
To illustrate, consider the following simple query:
SELECT COL_A, COL_B, COL_C
FROM MYTABLE
WHERE COL_A=:1 AND COL_B =:2 AND COL_C > 100;

For the purposes of this example, assume that all three columns are reasonably selective and that the
optimizer "wants" to use an index to retrieve this data. Also, assume COL_C is the most selective and
COL_A is the least selective. Based purely on cardinality, it would be tempting to add the following index
to support this query:

20160112-AdviceForThePeopleSoftOracleDBA.odt 12 of 15
 MYTABLE(COL_C, COL_B, COL_A)

But that arrangement would be inefficient for this query because the range scan on COL_C would prevent
the COL_B or COL_A from being used as efficient access predicates, though they might be used as filter
predicates. For best performance, COL_A or COL_B should precede COL_C because COL_C is a range
predicate. A better arrangement of these keys would be one of the following indexes:
PS_MYTABLE(COL_A, COL_B, COL_C)
PS_MYTABLE(COL_B, COL_A, COL_C)

The decision regarding which key should come first should be informed by whether other queries use only
COL_A or COL_B.

Table Row Ordering

Over time, the order in which rows are inserted into a table can significantly differ from the order in which
rows are commonly selected from that table. While indexes can improve the ability to identify table rows,
there may come a point where the order of the rows in the table diverges so significantly from the key
order of ALL of the indexes that performance can degrade significantly because the indexes are no longer
an efficient method of retrieving the data.
When this condition occurs, the optimizer will tend to choose full table scans where an index would
previously have been more effective. Additionally, changes in join order often occurs, further degrading
performance.
To remedy this condition, reorder the rows in the table so that the row order more closely matches the
most common use cases. In PeopleSoft Applications, the default, unique index is a good indicator of the
row order that the applications will typically access the table data. (One exception is when a table is
extensively used for month-end reporting with tools like nVision, reordering the table based on time
period and/or departmental values may be appropriate.)
Useful KM Documents
• How to use CTAS to improve clustering factors on Oracle RDBMS to improve SQL performance
(Doc ID 2065790.1)
• https://docs.oracle.com/cd/E17952_01/refman-5.5-en/create-table-select.html

Optimizer Limitations
There are a few limitations that can cause issues in PeopleSoft Applications and should be avoided if at all
possible.

• Avoid the use of BOTH histograms AND bind peeking in Oracle 10g - The combination of these features
tend to produce plan instability in Oracle 10g. PeopleSoft relies heavily on the use of bind
variables, so avoiding histograms is a reasonable approach to avoid this problematic combination.

• Avoid range scans on "padded" fields of type VARCHAR - When the VARCHAR field is used in a
BETWEEN clause, the CBO can make cardinality estimations that are not accurate, most notably
seen in cases where the length of the data is large and starts with, or is "padded" with, the same
several characters, e.g. 00000123. If this type of data is used in an implementation, additional care
will be needed to ensure that the optimizer has enough information to generate efficient plans.

20160112-AdviceForThePeopleSoftOracleDBA.odt 13 of 15
 • Avoid Function-Based Indexes - As described previously, there are limitations introduced when using
Function-Based Indexes. As described previously, PeopleTools Development generally
discourages the liberal use of use of Function-Based Indexes unless there is a compelling use case
where they are known to help with no negative side effects.

Useful Tools & References

There are several tools that should be part of every PeopleSoft Oracle DBA's tool kit.

SQLTXPLAIN (SQLT)
SQLT is a powerful reporting tool that collects data related to the execution of a given piece of SQL. It
conveniently collects execution plans, CBO statistics, metadata, performance statistics, configuration
parameters, etc. into a single archive that can easily be attached to an SR.

• SQLT - Tool That Helps To Diagnose SQL Statements Performing Poorly (Doc ID 215187.1)

The pscbo_stats Package

pscbo_stats is a joint venture between PeopleSoft CoE and Oracle Server Tech CoE and is a tool for
implementing dynamic sampling as well as embracing the preferred settings outlined in this document. It
provides a consistent, reliable way to maintain database statistics for PeopleSoft Applications. It includes
methods to gathering stats at either the schema- or table-level, and automatically implements dynamic
sampling for known volatile tables, e.g. temporary storage tables in Application Engine and COBOL.

• pscbo_stats - Improving Statistics in Oracle RDBMS for PeopleSoft Enterprise (Doc ID

1322888.1)

AWR / StatsPack Report

The AWR is a great tool for capturing the behavior of the database as a whole. In a RAC environment,
remember to gather reports from all of the instances when attempting to understand a particular behavior.
Generally speaking, the larger the time span a report covers, the less it is useful as a diagnostic tool. Try to
produce reports for 30 minute intervals, or less, if possible. If you have questions regarding the licensing
of the AWR tool, see the following Note:
• AWR Reporting - Licensing Requirements Clarification (Doc ID 1490798.1)

PSPRCSRQST Batch Trigger

A KM document exists that explains how to install a trigger on the PSPRCSRQST table that alters the
session when a batch process starts to run. This trigger is useful for implementing session-level changes,
either for diagnostics or performance tuning.

• Create Oracle Database Trigger to Manipulate the Session Associated with a PeopleSoft Batch
Process (Doc ID 1415642.1)

OS Watcher
OS Watcher (OSW) is a collection of UNIX shell scripts intended to collect and archive operating system
and network metrics to aid support in diagnosing performance issues. OSW operates as a set of
background processes on the server and gathers OS data on a regular basis, invoking such Unix utilities as

20160112-AdviceForThePeopleSoftOracleDBA.odt 14 of 15
vmstat, netstat and iostat.

• OS Watcher User Guide (Doc ID 301137.1)

ORAchk Health Checks for the Oracle Stack (ORAchk)

ORAchk proactively scans for known problems within the Oracle database, simplifying how to investigate
and analyze which known issues present risks. High level reports show system health risks with the ability
to drill down into specific problems and understand their resolutions.
• ORAchk - Oracle Configuration Audit Tool (Doc ID 1268927.2)
How to Configure PeopleSoft for Maximum Availability (MAA)
This paper describes the PeopleSoft Maximum Availability Architecture and the required operations and
configuration practices to maximize PeopleSoft availability against unplanned outages and minimize
downtime for planned maintenance activities. This paper also describes how recent enhancements in
PeopleSoft enable faster application fail over and reporting offloading to Active Data Guard.

• E-ORA How to Configure PeopleSoft for Maximum Availability (Doc ID 1446793.1)

Updates and Suggestions

The nature of this document is somewhat fluid. Updates will be provided there major changes occur in the
offerings in the Oracle DBMS, PeopleSoft Applications, or PeopleTools products. To provide feedback on
the document and provide suggestions regarding content, post an entry - clearly identifying this document in its
title - to the PeopleTools Install/Upgrade Community, which can be found at the this URL:
https://community.oracle.com/community/support/peoplesoft/install_upgrade_-_psft

20160112-AdviceForThePeopleSoftOracleDBA.odt 15 of 15