2000-01-28 Strange - Perf. For Peoplesoft Apps

Performance Enhancement Techniques for
Peoplesoft Applications on Oracle Databases

Version 3.2 February 7 2000
Paper # 507 IOUG-A Live 2000
Authors: James E. Strange & Raman R. Batra
Abstract:
The paper covers a range of methods for improving the performance of Peoplesoft HR
applications on an Oracle database. It includes information on implementing the cost-based
optimizer along with a variety of indexing strategies and other information. The paper presents
innovative ways to enhance the performance of Peoplesoft HR applications, encompassing Batch,
OLTP and DSS environments. The authors also share documented performance enhancements
and benchmark information.
The paper is a team effort of two Oracle Database Administrators who met on-line soon after the
formation of the Peoplesoft-SIG of the IOUG-A. After the group started, they began to exchange
notes and information regarding Peoplesoft HR on Oracle. The database environments at both
sites are somewhat similar - in terms of the size of the database and organizational size, however
the workload and on each database environment is as different as the business needs which drive
the use of the application. The similarity of the techniques used in both environments, mainly
implementation of the Cost-Based Optimizer, was striking. The authors decided to share their
experiences with the Oracle community in the form of this publication.
Table of Contents
1
Introduction:............................................................................................................................4
General Tuning and Architectural Considerations:..................................................................4
Identifying SQL and Performance Bottlenecks:......................................................................4
The Oracle Optimizer..............................................................................................................5

4.1
Driving Tables, Join Orders and Optimizer Modes:..........................................................6
4.2
Choosing an Oracle Optimizer:......................................................................................10
4.2.1
Setting the Optimizer Mode:...................................................................................11
4.2.2
Collecting Statistics:................................................................................................12
4.2.3
Ongoing Maintenance with the Cost-Based Optimizer:..........................................13
4.2.4
Preparation Before Tuning:.....................................................................................13
OnLine Access.......................................................................................................................13
5.1
General Information.......................................................................................................13
5.2
Case Study......................................................................................................................14
Batch Jobs.............................................................................................................................14
6.1
6.2
Payroll Calc Case Study.................................................................................................15
6.2.1
6.3
7
Actions taken...........................................................................................................15
Final Results...................................................................................................................30
Decision Support System.......................................................................................................30

7.1
7.2
The Infamous Security Tree...........................................................................................31
7.3
Case Studies...................................................................................................................31
7.4
Final Results...................................................................................................................44
Conclusions and Summary....................................................................................................44
Appendix A: Scripts..............................................................................................................45
10
Appendix B: New Indexes.................................................................................................48
11
Appendix C: Test Environments........................................................................................50
12
Appendix D: Recommended References............................................................................51
Table of Figures
Figure 1: A simple query joining three-tables and the RBO execution plan....................................7
Figure 2: Query with table order switched in the FROM clause for RBO......................................8
Figure 3: Query with cost-based optimizer.....................................................................................9
Figure 4: Query with table order switched in the FROM clause for CBO......................................9
Figure 5: Panel Access Speeds......................................................................................................14
Figure 6: Calc Query....................................................................................................................15
Figure 7: Calc Tables....................................................................................................................16
Figure 8: Initial Explain................................................................................................................17
Figure 9: PS_JOB1_NDX.............................................................................................................19
Figure 10: Bit Map Explain..........................................................................................................19
Figure 11: TKPROF......................................................................................................................22
Figure 12: PS_PAY_LINE1_NDX................................................................................................27
Figure 13: Final Tkprof for CALC................................................................................................27
Figure 14: Run-time comparison of 12 queries between Rule-Based Optimizer and Cost-Based
Optimizer...............................................................................................................................32
Figure 15:PS_PERSONAL_SRCH View.....................................................................................32
Figure 16: Details of Query2 (without hinted PS_PERSONAL_SRCH_Q View)........................32
Figure 17: Details of Query2 (with hinted PS_PERSONAL_SRCH_Q view)..............................39
1 Introduction:
Access to Peoplesoft HR Applications (HR, Benefits, Payroll, Self-Service etc.) on an Oracle
Database can be classified into three major categories: Batch Jobs (Payrolls, SQR's/Reports),
Online Access (OLTP - Panel Users) and Decision Support (PSQUERY and custom reports.) The
following sections present tuning studies and suggestions for each of the categories including test
results, case studies and associated data. Information about Batch Jobs and Online Access was
provided by James Strange. Information about DSS systems was provided by Raman Batra.
2 General Tuning and Architectural Considerations:

It must be assumed that the database server and instance are tuned properly. Before exploring
relatively advanced options such as implementing the Cost Based Optimizer, the SGA must be
sized so that the critical hit ratios - database buffer hit ratios, library cache hit ratio, data
dictionary cache hit ratios, are optimized. Peoplesoft can be CPU intensive, so size your system
accordingly. In addition to the SGA size, you may also need to look at the total memory on the
database server and your concurrent user population. Other issues such as the number of
dedicated two-tier connections and logical/physical three-tiers also may affect the CPU and
memory available on the server. Each dedicated connection (two-tier) takes from 6 to 10 MB of
memory on the server, depending upon the platform of Oracle and the sort_area size/
hash_area_size, and memory used per connection. It would be a good idea not to give ALTER
SESSION privilege to the general user community, since there is the potential for misuse. A big
sort can take a large chunk of the PGA (private global area) and eventually consume all available
memory.
In addition to CPU and memory, you may also need to ensure your disk system uses hardware or
software mirroring (or both) and the IO is balanced. If possible, use the Oracle recommended
OFA architecture as much as possible to avoid contention between data, index, rollback, and
temporary segments and redo logs. Sizing of rollback segments can be crucial depending upon
the size of the batch jobs and their frequency. Also, last but not the least, do not forget sizing of
temporary tablespaces, since some temporary activity will always need disk and may not fit into
shared memory. Be sure to declare the temporary tablespaces as TEMPORARY in order to
improve space management for sorts.
3 Identifying SQL and Performance Bottlenecks:

You may need to set up procedures and/or scripts or use tools to identify and isolate bottlenecks.
These could be as simple as querying the server OS environment for the top 10 resource intensive
processes (CPU and memory) and/or querying V$SQLAREA with limits set for buffer gets etc.
You could also use third-party tools or the tuning tools available with the OEM Performance
Pack. At Ramans employer, the HR/IT team set up a CGI script to create a web-page showing
CPU and memory utilization and resource availability.
Jim uses several different ways to monitor the databases at his employer. These include OEM,
SQLab, and a number of home grown SQL scripts. In order to get a quick snapshot of the users
currently connected to the database, one can use the script allActive.sql which is included in the
appendix. The script sql.sql extracts the current SQL for a specified sid from the V$SQLTEXT
table. Finally, the scripts makeExplain.sql and exp.sql can be used to obtain the execution plans
for the SQL associated with a particular sid.
No matter what method used to monitor your system, you must be able to identify problem areas,
especially during payroll batch runs. Thomas (1988)1 has published an excellent roadmap for
identifying bottlenecks and tuning your Oracle server and database environments for Peoplesoft
Applications. In particular the techniques used to identify and isolate performance bottlenecks
are definitely worth reading. Peoplesoft Corporation seems to have increased the number of bind
variables with release 7.x of Peoplesoft as compared to release 6.x, so some time is saved parsing
SQL statements with different variable values, which helps performance somewhat. In addition,
the bind variables help reduce contention in the buffer cache.
4 The Oracle Optimizer

After a SQL statement is parsed for syntax accuracy and object dependencies, it is processed by
the Oracle Optimizer to generate an optimal path or execution sequence. It is at this point that the
Optimizer Mode can greatly affect the access paths, processing, and ultimately the performance
and resource utilization needed by a SQL statement.
The Oracle Database basically has two optimizers:
1. Cost Based Optimizer (CBO): The Cost-Based Optimizer was introduced by Oracle about the
time of release 7.2. This feature came in response to similar moves by other database vendors.
The CBO received a large amount of adverse reaction from the user community. Much of that
reaction was well deserved. The early versions of the cost based optimizer were notoriously
unpredictable. It frequently chose execution paths inferior to the rule-based optimizer. However,
each successive version of the database server has included enhancements to the cost based
optimizer. Gradually, the cost based optimizer has improved. It can now be a vital element to
improving the performance or your system.
The CBO calculates the relative cost of a query by trying several combinations of query paths,
selecting the one with the lowest cost. Still, it is not perfect and sometimes there can be no
substitute for human intelligence, especially if knowledge of business processes can help you take
decisions on schema design. However, the CBO is generally effective at choosing the best access
path when implemented and supported correctly.
Another important consideration is that the CBO is a requirement for using the new advanced
features of Oracle. All the new features of Oracle 8.i (materialized views, function indexes, hashpartitioning), and some features of Oracle 8, such as range-partitioning and parallel options, work
only with the CBO. In order to take advantage of the latest and greatest features in scalability and
performance, you may need to implement the CBO.
2. Rule Based Optimizer (RBO): The Rule-Based Optimizer is the "old faithful" way of Oracle
processing, and a very predictable way of choosing a query plan. It works exactly the same way
even if a table has 10 rows or 10 million rows, which can be a blessing or a curse, depending
upon several factors, most importantly your data distribution. The RBO is not aware of the
distribution of values in a data-segment and does not take advantage of histograms, which track
the dispersion of values in a data and/or index segment. The RBO uses a predefined set of rules
based on precedence logic to figure out what access path to use. It is driven by about 15
precedence rankings (with single row access by rowid at the very top and full table scan right at
the bottom of the rankings. Several good tuning publications 2,3,4 detail these precedence
rankings. The rule-based optimizer chooses an access path based on:
Available access path for the statement
Ranks of the access paths
The WHERE clause of the SQL statements and the order of the table-names in the FROM clause
help the RBO to determine possible access paths. The optimizer then chooses the highest ranked
access path. Access by index is always ranked higher than tablescans. As a result, the optimizer
will always prefer an access path which uses an index. In some cases that approach can be a
problem. Sometimes a full table scan may execute faster than access by index. The usual
solution for a bad plan with the RBO is to place hints in the query, which will direct the optimizer
to a better plan. Unfortunately, that solution is frequently not available, and most ERP systems,
including Peoplesoft, do not encourage you to change the code even where possible. In the case
of Peoplesofts PSQUERY tool, it is effectively impossible to add hints to queries.
If you do not have a separate environment for decision support, you could have a few "queries
from hell" consuming most of your system resources and dragging down the online and payroll
processes. It is mostly in decision support and OLTP where you will find the biggest limitations
of the RBO. As Jim found out with online access, the improvements were dramatic after the RBO
was replaced with the CBO.
4.1 Driving Tables, Join Orders and Optimizer Modes:

The driving table is the table processed first in a query. The rows retrieved from the driving table
are matched with the other tables in the FROM clause. One of the most annoying limitations of
the RBO is the heavy dependence on the order in which tables are joined in the SQL statement.
The execution plan can be changed dramatically by reordering the tables in the query. That can
have terrible results in an online environment using tools such as PSQUERY where the SQL
statement is created by people who have no idea how the database is designed.
From our experience, the Cost Based Optimizer does not seem to be dependent on the order of the
tables in the query, as is the RBO. The driving tables for the RBO or CBO can be determined as
follows:
In the RBO, the last table in the FROM clause acts as a driving table, all things being equal.
In the CBO, according to published literature, the first table in the FROM clause acts as a
driving table, unless the optimizer determines a different plan will perform better. This again
depends upon how the tables are joined and if the join results in a table scan, unique index
scan or index range scan. We found the Cost-Based Optimizer generated the same execution
plan, no matter what table order was used in the FROM clause. The following figures
illustrate the effect of changing the table order for the RBO.
Figure 1: A simple query joining three-tables and the RBO execution

plan
selecta.name,a.phone,b.empl_status,b.effdt,c.empl_rcd#,c.eff_status
from
sysadm.ps_personal_dataa,
sysadm.ps_direct_depositc,
sysadm.ps_jobb
wherea.emplid='99999'
andc.emplid=a.emplid
anda.emplid=b.emplid
Theexecutionplanforthiswiththerulebasedoptimizeris:
QueryPlan
SELECTSTATEMENTCost=
NESTEDLOOPS
NESTEDLOOPS
TABLEACCESSBYINDEXROWIDPS_PERSONAL_DATA
INDEXUNIQUESCANPS_PERSONAL_DATA
TABLEACCESSBYINDEXROWIDPS_JOB
INDEXRANGESCANPS#JOB
TABLEACCESSBYINDEXROWIDPS_DIRECT_DEPOSIT
INDEXRANGESCANPS_DIRECT_DEPOSIT
In the above plan the first table processed is PS_PERSONAL_DATA, followed by PS_JOB, the
results are put into a nested loop and the result set is matched with data from the third table -
PS_DIRECT_DEPOSIT. The driving table, or the table first processed will always be
PS_PERSONAL_DATA because of the clause a.emplid='99999'.
Figure 2: Query with table order switched in the FROM clause for
RBO
fromsysadm.ps_jobb,
sysadm.ps_personal_dataa
ExecutionPlanfortheabovequery:
QueryPlan
NESTEDLOOPS
NESTEDLOOPS
In the figure, the first table processed is PS_PERSONAL_DATA, followed by

PS_DIRECT_DEPOSIT, the results are put into a nested loop and the result set is matched with
data from the third table - PS_JOB. The driving table will always be PS_PERSONAL_DATA
because of the clause a.emplid='99999'.
The order of tables in the query will affect the execution plan. As a result, although the final
result is the same, the performance can vary tremendously. It can make a huge difference;
especially if a full table scan is performed on a table with 10 million rows as compared to a table
with say 10000 rows of data. The following figures demonstrate how changing the table order
does not effect the plan for the CBO.
Figure 3: Query with cost-based optimizer

from
sysadm.ps_personal_dataa,
sysadm.ps_jobb
QueryPlan
SELECTSTATEMENTCost=9
NESTEDLOOPS
NESTEDLOOPS
Figure 4: Query with table order switched in the FROM clause for
CBO
fromsysadm.ps_jobb,
sysadm.ps_personal_dataa
QueryPlan
NESTEDLOOPS
NESTEDLOOPS
This behavior can be both a blessing and a curse. When dealing with Decision Support
Environments, the person writing the queries is intent on getting the business logic and typically
does not want be concerned about the order of tables in a query. In fact, quite a few queries are
constantly changed as managers or end-users add requirements. This frequently results in
complicated FROM and AND clauses, and ever growing numbers of columns in the SELECT
statement. In such a case, the CBO frees the developer from worrying about the table order. This
also holds true for adhoc tools such as PSQUERY. Unfortunately, the CBO can hurt
performance in some circumstances. If it does not have the right statistics or chooses a bad
execution path, performance can be very bad.
In Ramans system, which is a mixed OLTP, Decision Support and Batch environment, frequently
they came across "queries from hell" which would take hours, even days to complete. Simply reordering the tables in the join made a tremendous difference under the RBO.
Comparing the two optimizer modes is somewhat like comparing an automatic transmission
(CBO) to a manual transmission (RBO). The RBO, maintained by Oracle Corporation for
backward compatibility, gives developers a lot of control over the way SQL statements execute.
However, in most ERP applications, and Peoplesoft is no exception, it sometimes is impossible to
modify SQL, so the optimizer to make intelligent selections to satisfy rather generic queries.
That is where the CBO can be of crucial benefit.
4.2 Choosing an Oracle Optimizer:

The Oracle default optimizer is Choose. This simply means the system will run CBO whenever
there are statistics available. Otherwise it will use the RBO. If one contacts Peoplesoft, their
technical support staff will automatically advise setting the database to RULES mode. Why do
10
they advise using RULES? Peoplesoft is not in the database business. They operate on a number
of platforms and have little desire to keep up with changes to Oracle. Plus, as mentioned before,
the early versions of the CBO were justifiably scorned.
Starting with Oracle 7.3.4 things have changed. Our tests have indicated that significant
performance gains may be achieved by going against the advice of Peoplesoft and using the CBO
set to first_rows mode. However, the unfortunate truth is that while the CBO has great potential
for improving performance, it can also take a good deal of effort to make it work well for your
environment. If your shop has a number of custom reports or jobs tuned to work with the RBO,
changing to CBO may in fact hurt the performance of those jobs.
As an example, Jim implemented cost based optimization at his employer. They experienced
increased performance for nearly all online access. Panels, which formerly required 20+ seconds
would appear in 2 or 3 seconds using the cost, based optimizer. This was greeted by a great deal
of enthusiasm by the customers. However, when the batch cycle ran for payroll the party
ended. Payroll was extremely slow. So, they restarted the database in order to switch back to
rules. The payroll then finished in a reasonable amount of time.
Eventually Jim was able to identify and cure the problems with the batch jobs. Their shop now
runs in CBO 24 hours a day. However, this serves to illustrate that implementing the CBO can be
a challenge, initially requiring significant work from the DBA.
4.2.1 Setting the Optimizer Mode:

The Optimizer Mode can be set at the following levels:
1. Database Instance Level - this sets the defaults for all sessions connected to the database. (set
in the init[SID].ora by the DBA).
RULE: Sets up default optimizer mode as Rule for every connection to the instance. Set
OPTIMIZER_MODE=rule in your init.ora file.
CHOOSE: This is the default. If you do assign any value to the initialization parameter
OPTIMIZER_MODE, CHOOSE is the default. If statistics are present, the optimizer will try
cost-based optimization. If no statistics are present, it will try rule-based optimization. The
situation can worsen, when we have statistics present on some tables and indexes and missing
on others. In this case, the cost-based optimizer tries to assume statistics, which can
sometimes lead to all kinds of unexplained behavior. It is generally a very bad idea to leave
the database in CHOOSE unless statistics are properly maintained.
FIRST_ROWS: These invoke the Cost-Based Optimizer. FIRST_ROWS is more useful to
online access situations, where a partial result of the query is presented to the user. This
mode is somewhat similar to the rule-based optimizer as far as row operations go, and has a
tendency to use more nested loops, which presents data as it is being processed by result sets.
Set OPTIMIZER_MODE=FIRST_ROWS in your init.ora
11
ALL_ROWS: ALL_ROWS is better for batch situations where you want the batch job to
complete, but you don't necessarily want to monitor data being processed. You may find that
the percentage of hash-joins increases as compared to nested loops.
In our experience, we found FIRST_ROWS to give superior results to ALL_ROWS for
Peoplesoft applications.
2. Session Level - Within each session, you can change the optimizer mode, as long as you have
the ALTER SESSION privilege. In Peoplesoft, it is very hard use this option with online
panels. It is relatively easy to insert an ALTER SESSION in the SQR's. Alternatively you can
change the optimizer mode in the initialization file of the SQR (Ask Dustan/Kendall).
However, either of these approaches can lead to a lot of customizations in SQRs, which
makes it a questionable course of action.
When starting to implement the CBO, one approach is to set the default mode for the instance
as FIRST_ROWS and use RULE for the batch jobs via the ALTER SESSION command until
you have tuned them.
An example would be:
SQL>ALTER SESSION SET OPTIMIZER_GOAL=RULE;
3. Code Level (using Hints) - In a SQL statement, you can give hints. Hints tell the optimizer
what access path you want it to use. You can force full-table scans, or force the statement to
use a specific index. Unless you use the RULE hint, almost every hint uses cost-based
optimization.
The optimum way would be to set the init.ora to a particular mode and then let every process online, batch and decision support to default to that optimizer mode. However, as you transition
to the CBO, you may use some or all of the above methods to change your optimizer settings.
4.2.2 Collecting Statistics:

The CBO needs statistics on every data segment, for Peoplesoft everything owned by the schema
owner must be analyzed (in most cases, this will be SYSADM). The objects owned by SYS and
SYSTEM should not be analyzed, as recommended by Oracle, especially if the optimizer mode is
set to choose.
There are several ways to collect statistics:
1. ANALYZE TABLE SYSADM.PSTREENODE ESTIMATE STATISTICS sample 30 percent;
(this will estimate statistics with a sample of 30% of table size, you can compute them too)
2. ANALYZE INDEX SYSADM.PSTREENODE COMPUTE STATISTICS;
This computes statistics for indexes, to be used when you rebuild indexes or alter/create new
indexes.
12
3. Execute DBMS_UTILITY.ANALYZE_SCHEMA('SYSADM','COMPUTE',NULL,30);
This Oracle supplied package will analyze the entire schema, tables and indexes both, and
you can specify the method (compute or estimate).
Depending upon the volatility of the data, you may increase the sample to 100 percent or to a
specified number of rows, in most cases 25-30% sample is enough for the optimizer to work
accurately. Computing statistics is CPU and memory intensive and will take a fair amount of
temp space, so it is recommended this be performed during off-hours. You will need statistics for
tables and indexes. Collecting statistics for tables will also collect index statistics, however if you
rebuild an index or create a new one, you will need to collect statistics for the rebuilt/new/altered
indexes.
4.2.3 Ongoing Maintenance with the Cost-Based Optimizer:

Once you start to use the cost-based optimizer, you will have to ensure statistics are present and
fairly current (once a week should be enough for most medium sized systems). With Peoplesoft
you may be performing Peopletools upgrades and/or tax upgrades several times a year. These are
the occasions you when you really need to be careful. All it takes is one table missing statistics,
to totally confound and upset your query run-times.
4.2.4 Preparation Before Tuning:

It is highly recommended you experiment and read up on tuning indexes and complex joins, since
Peoplesoft HR will give you ample opportunity to test your knowledge in those specific areas. In
most cases, you cannot change the SQL code, or at least should keep the code "vanilla"; your
tuning strategy must be designing and tuning appropriate indexes. Also, experiment with
different optimizer modes. Practice in tuning joins and designing indexes with Rule Based and
the Cost Based Optimizer would really help. We recommend taking a few simple queries (4-5
tables) and analyze and compare the execution plan for the RBO and the CBO.
The following sections demonstrate performance enhancement techniques for Peoplesoft
databases. This includes a look at enhancing online performance by using the cost based
optimizer, tuning batch jobs for optimum performance, and tuning DSS queries for Peoplesoft.
5 OnLine Access
5.1 General Information
When the cost based optimizer was implemented at Jims employer, there was a significant
improvement in online response time. Screens, which formerly required 20 seconds to appear,
would appear in less than 10 seconds using cost based. The customer service representatives
greeted this with a great deal of enthusiasm. However, when the batch cycle ran for payroll the
party ended. Payroll was extremely slow. So, the database was restarted under rules in order to
finish payroll in a reasonable amount of time.
Later Jim, was very surprised to find the users insisting he switch the system back to the way
you had it before payroll. The users perception of online performance under CBO was so much
better that they agreed to limit batch jobs to designated windows of time. The database was
13
manually rebooted at the start and end of each batch window in order to run RULES during batch
jobs and cost for online access.
Of course, that was not an ideal situation. So, significant effort was devoted to figuring out how
to improve batch performance so the system could remain in cost based mode 100% of the time.
Eventually, a solution was found. The key is to use FIRST_ROWS along with carefully tuning
the indexes in the database. The following case study shows how one job was improved by the
use of the CBO.
5.2 Case Study

Testing online performance was a simple matter of timing how long it takes to bring up panels of
information from Peoplesoft. In virtually every case tested, the performance of the cost based
database was significantly superior to the response time from RBO. It must be noted that in order
to be sure caching on the client did not influence the timings, the cache files on the PC were
deleted between tests. The following table lists some of the panels accessed and the times
observed for each database type.
Figure 5: Panel Access Speeds
Panel
Rules Based
Cost Based
employee contact info
20 seconds
9 seconds
business expenses
4 seconds
2 seconds
parking expenses
3 seconds
1 second
current job- benefits partic 1st
5 seconds
1 second
benefits
33 seconds
25 seconds
paycheck review
15 seconds
11 seconds
As the data demonstrates, online access is often faster under cost based than under rules. The
difference in times is not always dramatic; however, in terms of user perception, the difference is
significant.
6 Batch Jobs
As mentioned previously, the cost based optimizer can have a terrible affect on batch jobs for
Peoplesoft. However, that can be fixed. The key is to use FIRST_ROWS along with carefully
tuning the indexes in the database. The resulting system will produce performance equal or
superior to a rule based system. After tuning, the performance will almost always be better than a
vanilla Peoplesoft installation using rules.
There are a wide variety of jobs which take place during the payroll cycle. Two of the major jobs
will be covered in this section. Those are the payroll jobs: calc and confirm. Calc jobs can be
14
extremely time consuming. To help speed the process, Peoplesoft advises to limit the number of
employees in individual paygroups to a very small number. That solution, while workable, is not
always very useful. In addition, while it does speed up the process overall, the Calc process itself
remains quite inefficient. As it turns out, the SQL used for the Calc jobs is quite similar to the
SQL used for the confirm jobs. So, tuning for one is quite likely to improve the performance of
the other.
A third type of job is actually a general group of jobs. These are custom reports. Jims system
has a large number of reports run via SQR against the database. These must be individually tuned
to ensure proper performance.
So, Jim set out to capture the SQL running against the database for each of these scenarios.
During this effort he used two tools. One was Quest Softwares SQLab Tuner. This allowed you
to create a repository, which collects all of the SQL issued during a batch job. The second tool
was a series of scripts which capture the SQL for processes running against the database. The
scripts also can be used to produce explain plans and tkprof output (see appendix for scripts.)
Although not all of the many enhancements made to the database will be covered by this paper,
the following section provides a case study showing how to improve the performance of the calc
routines dramatically. The results of a properly tuned cost based database are almost always
faster than an RBO database.
6.2 Payroll Calc Case Study

Calc on large paygroups takes an exceptionally long time. As the company increased in size, a
step that formerly took 3 to 5 minutes for an average sized company and paygroup began to
require hour or more for a single large paygroup. Contacting Peoplesoft did not produce any
helpful information. They advised breaking the paygroup into multiple smaller groups. So, it
was left to Jim to improve the performance.
6.2.1 Actions taken

Capturing the SQL to identify problem queries was relatively easy using SQLab and a series of
homegrown scripts. SQLab was used to create a repository, which collected all SQL executed
during the calc job. By simply watching the log output and periodically using the scripts
allActive.sql and sql.sql, it became readily apparent that the longest part of the batch job was a
single step. Improving the speed of that step became the focus of the tuning effort. For purposes
of this paper, the offending SQL statement will be referred to as the CALC statement.
The following figures show the CALC statement, list information about the underlying tables,
and show an Explain plan for the CALC statement.
Figure 6: Calc Query
SELECTA.EMPLID
,A.EMPL_RCD#
FROMPS_JOBA
15
,PS_PERSONAL_DATAP
WHEREA.COMPANY='046'
ANDA.PAYGROUP='A01'
ANDA.EFFDT<='03DEC1999'
ANDA.EMPL_STATUSIN('A','P','Q','U')
ANDP.EMPLID=A.EMPLID
ANDP.PER_STATUS='E'
ANDNOTEXISTS(SELECT'X'FROMPS_JOBFWHEREF.EMPLID=A.EMPLID
ANDF.EMPL_RCD#=A.EMPL_RCD#
ANDF.EFFDT<='03DEC1999'
ANDF.EFFDT>A.EFFDT
AND(F.EFFDT<='03DEC1999'OR(F.COMPANY='046'
ANDF.PAYGROUP='A01'
ANDF.EMPL_STATUSIN('A','P','Q','U')
ANDF.EFFSEQ=(SELECTMAX(G.EFFSEQ)FROMPS_JOBGWHEREG.EMPLID=F.EMPLID
ANDG.EMPL_RCD#=F.EMPL_RCD#
ANDG.EFFDT=F.EFFDT))))
ANDA.EFFSEQ=(SELECTMAX(H.EFFSEQ)FROMPS_JOBHWHEREH.EMPLID=A.EMPLID
ANDH.EMPL_RCD#=A.EMPL_RCD#
ANDH.EFFDT=A.EFFDT)
ANDNOTEXISTS(SELECT'X'FROMPS_PAY_LINELWHEREL.EMPLID=A.EMPLID
ANDL.EMPL_RCD#=A.EMPL_RCD#
ANDL.COMPANY='046'
ANDL.PAYGROUP='A01'
ANDL.PAY_END_DT='03DEC1999'
ANDL.OFF_CYCLE='N')
ORDERBYA.EMPLIDASC;
Figure 7: Calc Tables

SQL>selectnum_rowsfromdba_tableswheretable_name='PS_JOB';
16
NUM_ROWS
233417
SQL>selectnum_rowsfromdba_tableswheretable_name='PS_PAY_LINE';
NUM_ROWS
828720
SQL>selectnum_rowsfromdba_tableswheretable_name='PS_PERSONAL_DATA';
NUM_ROWS
32057
Figure 8: Initial Explain

SQL>@explain
QueryPlan
SORTORDERBY
CONCATENATION
FILTER
NESTEDLOOPS
TABLEACCESSFULLPS_JOB
TABLEACCESSBYROWIDPS_PERSONAL_DATA
TABLEACCESSBYROWIDPS_PAY_LINE
INDEXRANGESCANPS#PAY_LINE
SORTAGGREGATE
INDEXRANGESCANPSAJOB
17
FILTER
TABLEACCESSBYROWIDPS_JOB
SORTAGGREGATE
FILTER
NESTEDLOOPS
SORTAGGREGATE
FILTER
SORTAGGREGATE
FILTER
NESTEDLOOPS
SORTAGGREGATE
FILTER
18
SORTAGGREGATE
FILTER
NESTEDLOOPS
SORTAGGREGATE
FILTER
SORTAGGREGATE
59rowsselected.
The query plan indicated a very costly query, which included a full table scan against PS_JOB.
That table scan was obviously a problem, so an index was added to PS_JOB to eliminate the table
scan. Both a normal index and a bitmapped index were compared. This situation was an ideal
candidate for a bitmapped index. Comparing performance between a normal index and the bitmapped index, the bit-mapped index produced marginally better performance.
Figure 9: PS_JOB1_NDX
createbitmapindexps_job1_ndxonps_job(company,paygroup);
analyzeindexps_job1_ndxestimatestatisticssample30percent;
The CALC query performed considerably better with an index. The runtime decreased to 16 to
17 minutes; however, that still was not fast enough.
Figure 10: Bit Map Explain
SQL>@explain
19
QueryPlan
SORTORDERBY
CONCATENATION
FILTER
NESTEDLOOPS
INDEXRANGESCANPS_JOB1_NDX
SORTAGGREGATE
FILTER
SORTAGGREGATE
FILTER
NESTEDLOOPS
SORTAGGREGATE
FILTER
20
SORTAGGREGATE
FILTER
NESTEDLOOPS
SORTAGGREGATE
FILTER
SORTAGGREGATE
FILTER
SORTAGGREGATE
FILTER
SORTAGGREGATE
NESTEDLOOPS
21
At this point a comparison to rules mode was made. It turned out that the bit mapped index
brought performance up to par with rules based optimizer. The Calc completed in 15:09.08
minutes under rules.
Since it was going to be necessary to better the performance even further, tkprof was used to get
more detailed information about the query.
Figure 11: TKPROF
TKPROF:Release7.3.4.0.0ProductiononThuDec210:22:231999
Copyright(c)OracleCorporation1979,1996.Allrightsreserved.
Tracefile:ora_26971.trc
Sortoptions:prselaexeelafchela
********************************************************************************
count=numberoftimesOCIprocedurewasexecuted
cpu=cputimeinsecondsexecuting
elapsed=elapsedtimeinsecondsexecuting
disk=numberofphysicalreadsofbuffersfromdisk
query=numberofbuffersgottenforconsistentread
current=numberofbuffersgottenincurrentmode(usuallyforupdate)
rows=numberofrowsprocessedbythefetchorexecutecall
********************************************************************************
SELECTA.EMPLID
,A.EMPL_RCD#
FROMPS_JOBA
,PS_PERSONAL_DATAP
ANDA.PAYGROUP='A01'
22
ANDP.PER_STATUS='E'
ANDF.EFFDT>A.EFFDT
ANDF.PAYGROUP='A01'
ANDH.EFFDT=A.EFFDT)
ANDL.COMPANY='046'
ANDL.PAYGROUP='A01'
ANDL.OFF_CYCLE='N')
ORDERBYA.EMPLIDASC
callcountcpuelapseddiskquerycurrentrows
Parse10.790.8930640
Execute10.000.000000
Fetch11056.251253.051024881661197812
total31057.041253.9410249116611977212
23
Missesinlibrarycacheduringparse:1
Optimizergoal:FIRST_ROWS
Parsinguserid:269(SYSADM)
RowsExecutionPlan
0SELECTSTATEMENTGOAL:FIRST_ROWS
12SORT(ORDERBY)
0CONCATENATION
753FILTER
753NESTEDLOOPS
233376TABLEACCESSGOAL:ANALYZED(FULL)OF'PS_JOB'
753TABLEACCESSGOAL:ANALYZED(BYROWID)OF
'PS_PERSONAL_DATA'
1506INDEXGOAL:ANALYZED(RANGESCAN)OF
'PS_PERSONAL_DATA'(NONUNIQUE)
32361TABLEACCESSGOAL:ANALYZED(BYROWID)OF'PS_PAY_LINE'
106661796INDEXGOAL:ANALYZED(RANGESCAN)OF'PS#PAY_LINE'
(NONUNIQUE)
22090SORT(AGGREGATE)
38167INDEXGOAL:ANALYZED(RANGESCAN)OF'PSAJOB'
(NONUNIQUE)
12632FILTER
12632TABLEACCESSGOAL:ANALYZED(BYROWID)OF'PS_JOB'
(NONUNIQUE)
0SORT(AGGREGATE)
(NONUNIQUE)
0FILTER
24
0NESTEDLOOPS
'PS_PERSONAL_DATA'
(NONUNIQUE)
(NONUNIQUE)
12632FILTER
(NONUNIQUE)
0SORT(AGGREGATE)
(NONUNIQUE)
1422FILTER
1422NESTEDLOOPS
'PS_PERSONAL_DATA'
(NONUNIQUE)
25
(NONUNIQUE)
12632FILTER
(NONUNIQUE)
0SORT(AGGREGATE)
(NONUNIQUE)
47360FILTER
47360NESTEDLOOPS
'PS_PERSONAL_DATA'
(NONUNIQUE)
(NONUNIQUE)
12632FILTER
(NONUNIQUE)
0SORT(AGGREGATE)
(NONUNIQUE)
************************************************************************
26
The Tkprof output made it readily apparent that the index scan of PS#PAY_LINE was
tremendously expensive. The major problem was the number of blocks being accessed to get the
PS_PAY_LINE data.
Given that information, efforts re-focussed on access to PS_PAY_LINE. The CALC query tries
to access the following fields:
emplid, empl_rcd#, company, paygroup, pay_end_dt, and off_cycle.
The PS#PAY_LINE index being used was significantly wider than needed for this query. So an
index targeted specifically for the CALC query was created:
Figure 12: PS_PAY_LINE1_NDX

createindexPS_PAY_LINE1_NDXonPAY_PAY_LINE(emplid,empl_rcd#,company,paygroup,
pay_end_dt,off_cycle);
analyzeindexPS_PAY_LINE1_NDXestimatestatistics30percent;
The CALC query was then tested against the system. The results were fabulous. The CALC
query finished in 1:00.51 minutes.
Figure 13: Final Tkprof for CALC

TKPROF:Release7.3.4.0.0ProductiononThuDec211:23:531999
Copyright(c)OracleCorporation1979,1996.Allrightsreserved.
Tracefile:ora_28091.trc
Sortoptions:prselaexeelafchela
********************************************************************************
count=numberoftimesOCIprocedurewasexecuted
cpu=cputimeinsecondsexecuting
elapsed=elapsedtimeinsecondsexecuting
disk=numberofphysicalreadsofbuffersfromdisk
query=numberofbuffersgottenforconsistentread
current=numberofbuffersgottenincurrentmode(usuallyforupdate)
rows=numberofrowsprocessedbythefetchorexecutecall
********************************************************************************
SELECTA.EMPLID
27
,A.EMPL_RCD#
FROMPS_JOBA
,PS_PERSONAL_DATAP
ANDA.PAYGROUP='A01'
ANDP.PER_STATUS='E'
ANDF.EFFDT>A.EFFDT
ANDF.PAYGROUP='A01'
ANDH.EFFDT=A.EFFDT)
ANDL.COMPANY='046'
ANDL.PAYGROUP='A01'
ANDL.OFF_CYCLE='N')
ORDERBYA.EMPLIDASC
callcountcpuelapseddiskquerycurrentrows
28

Parse10.070.0700200
Execute10.000.000000
Fetch156.6987.91106106493984012
total356.7687.981061064939842012
Missesinlibrarycacheduringparse:1
Optimizergoal:FIRST_ROWS
Parsinguserid:269(SYSADM)
RowsExecutionPlan
0SELECTSTATEMENTGOAL:FIRST_ROWS
12SORT(ORDERBY)
49535FILTER
49535NESTEDLOOPS
53695BITMAPCONVERSION(TOROWIDS)
12BITMAPINDEX(SINGLEVALUE)OF'PS_JOB1_NDX'
'PS_PERSONAL_DATA'
47944INDEXGOAL:ANALYZED(RANGESCAN)OF'PS_PAY_LINE1_NDX'
(NONUNIQUE)
(NONUNIQUE)
12632FILTER
29
(NONUNIQUE)
0SORT(AGGREGATE)
(NONUNIQUE)
************************************************************************
The number of blocks processed using the new index were significantly fewer than in the original
query.
6.3 Final Results

The batch testing was all performed against a copy of the production database. The copy had a
far smaller SGA than the production database. When the new indexes were implemented in
production the CALC query timings were even better than on the test database. The CALC query
completed in 12 seconds on the production database. As a result, the full calc process performed
considerably better. Formerly, the calc process required 3 hours 14 minutes to complete. The
new indexes decreased that time to 2 hours 9 minutes. The confirm process, which runs similar
queries, ran in 2 hours less time.
7 Decision Support System

The DSS study was conducted by Raman Batra. Ramans database environment is mixed
decision support, OLTP, and batch processing. Payroll is run several times a week to support
different sites and companies. Analysts in payroll and benefits generate reports depending upon
the requirements of company management and functional areas. The analysts use PSQUERY to
generate adhoc queries. The queries are completely unpredictable. Their occurrence is random
and driven by business needs.
Some of the queries run for hours, and in a few cases they run overnight. More importantly, the
resources consumed by these queries overload the database server during OLTP and Payroll runs.
It was very common to have a few big queries take 40-60% CPU of a 4-way 400 MHz Solaris
E5000 Server. Since the queries constantly change, it is very hard to tune for them. There is not
much value in creating indexes and attempting to tune SQL unless there is a specific query which
is run repeatedly.
30
Initially, Raman examined the top ten processes (CPU and memory) to identify problem queries.
Sometimes an analyst would call the DBA asking for help on a specific query. Addressing the
issue on a case-by-case basis provided little relief. The queries continued to change which
constantly caused new problems. A more general solution was needed.
It was obvious the situation was crying for intelligent optimizer choices. Most analysts do not
want to be bothered with join orders and careful planning of SQL statements. They are concerned
with coding business logic. However, it was impossible to implement the CBO for the system
without going through a significant stage of testing and planning. An interim solution was
required.
7.2 The Infamous Security Tree

Raman began to collect offensive SQL statements to maintain a history and find some common
trends. A detailed analysis of the SQL generated by the adhoc queries and reports found that the
Peoplesoft security views were a common factor which could be improved. Virtually all queries
generated by PSQUERY will automatically have the security views joined into the query by
PSQUERY. This join is not under the control of the users. In Peoplesoft parlance, the result is
called a search record. PS_PERSONAL_SRCH_Q is an example of such a view. It was obvious
that improving the performance of the security tree would help overall performance of queries via
PSQUERY.
7.3 Case Studies

In order to improve the performance of the views, Raman experimented with a number of
approaches. Custom indexes were of limited use due to the views being joined with a larger
query. A different approach was needed, so Raman began experimenting with views. The
FIRST_ROWS hint was added to 7 views. They also reviewed the ORDERED , USE_MERGE
and USE_HASH hints, however, the FIRST_ROWS hint seemed to need less work and gave
better results.
The hints yielded a dramatic improvement in performance. Customers started to ask, what did
you do ? A query nicknamed "The Bible", which formerly took almost an hour, now it ran in
less than a minute. Benchmarks tests with the twelve "baddest and meanest" queries were very
encouraging. The test result were so convincing that the hints were moved into Production.
Sure, putting hints in view is a customization. But the gains were dramatic. Also, putting hints
in views can be done through Peopletools, so they can be carried forward during upgrades.
As an added bonus, once the FIRST_ROWS hint is put in the view syntax, any query using the
VIEW in a join statement automatically invokes the Cost-Based Optimizer. We were therefore
able to get the advantages of the CBO, but limit it to the PSQUERY tool. Batch jobs never use
the security views. As a result, they never invoke the cost based optimizer.
There is one important caveat to this approach. You MUST run statistics on the database
regularly for this to work properly. The CBO needs statistics to work well. If the statistics are
not present, it will guess. That can be a very bad thing.
31
Time (seconds)
Results of 12 famously "bad" queries

400.00
300.00
RBO
200.00
CBO
100.00
0.00
Query Query Query Query Query Query Query Query Query

1
2
4
5
6
8
9
10
12
RBO
5.44 122.53 33.15 2.21
3.09 36.25 2.85 228.00 0.51
CBO
0.06
0.15 14.00 0.09 128.00 0.11
7.00 48.00 6.54
Figure 14: Run-time comparison of 12 queries between Rule-Based

Optimizer and Cost-Based Optimizer.
Figure 15:PS_PERSONAL_SRCH View

CREATEORREPLACEVIEWSYSADM.PS_PERSONAL_SRCH
(
EMPLID,OPRCLASS,EMPL_RCD#,NAME,LAST_NAME_SRCH,SSN,
SIN,NATIONAL_ID,ACCESS_CD
)
AS
SELECT/*+FIRST_ROWS*/B.EMPLID,C.OPRID,B.EMPL_RCD#,A.NAME
,A.LAST_NAME_SRCH,A.SSN,A.SIN,A.NATIONAL_ID,C.ACCESS_CD
FROMPS_JOBB,
PS_PERSONAL_DATAA,
PS_SCRTY_TBL_SUPRVC
WHERE
(SQLhasbeentruncatedhere)
C.K_HOURLY_SALARY=B.EMPL_TYPE
);
32
Figure 16: Details of Query2 (without hinted PS_PERSONAL_SRCH_Q

View)
SELECTA.EMPLID,PS_RAM_NAME_LOOKUP(A.EMPLID),E.SSN,A.COMPANY,
B.DESCR,C.STATE,A.WORKERS_COMP_CD,D.DESCR,A.PAYGROUP,F.DESCR,G.ERNCD,
H.DESCR,G.BALANCE_YEAR,G.BALANCE_PERIOD,G.GRS_MTD,B.COMPANY,
TO_CHAR(B.EFFDT,'YYYYMMDD'),F.COMPANY,F.PAYGROUP,TO_CHAR(F.EFFDT,'YYYYMMDD'),
H.ERNCD,TO_CHAR(H.EFFDT,'YYYYMMDD')
FROMPS_K_JOB_QUERY_VWA,
PS_PERSONAL_SRCH_QA1,
PS_COMPANY_TBLB,PS_LOCATION_TBLC,
PS_K_CO_WCOMP_TBLD,PS_PERSONAL_DATAE,
PS_PAYGROUP_TBLF,PS_EARNINGS_BALG,PS_EARNINGS_TBLH
WHEREA.EMPLIDIN
('00156','00871','00959','01203','01706','01749','01915','02316','02475','02628','02967','
03403','03426','03476','03583','04267','05130','05423','06107','06108','06303','06576','06
771','07252','07282','07422','07463','08380','09165','09340','09373','09442','09531','0992
6','10576','11496','11886','12028','12038','12336','12530','13050','14336','14471','14972'
,'16956','18069','18070','18362','18442','18816','19243','19762','19857','19858','20560','
21517','21687','21791','22589','22718','22910','23097','23936','23960','24033','24046','24
108','24115','24168','24308','24404','24444','24768','24921','25028','25137','25171','2546
9','25480','26239','26980','27160','27217','27218','27410','27569','27729','28065','29750'
,'30950','31508','31960','32249','32470','32825','32952','33803','33835','37430','37865','
38266','38543','45433','49184','54100','54840','63550','68060','68690','70137','71818','75
420','76390','78090','80500','81220','83696','83985','89930','93892','97288')
ANDB.COMPANY=A.COMPANY
ANDC.LOCATION=A.LOCATION
ANDD.WORKERS_COMP_CD=A.WORKERS_COMP_CD
ANDD.COMPANY=A.COMPANY
ANDE.EMPLID=A.EMPLID
ANDF.PAYGROUP=A.PAYGROUP
ANDF.COMPANY=A.COMPANY
ANDH.ERNCD=G.ERNCD
ANDA.EMPLID=A1.EMPLID
ANDA.EMPL_RCD#=A1.EMPL_RCD#
33
ANDA1.OPRID='BATRAR'
ANDE.EMPLID=A1.EMPLID
ANDG.EMPLID=A.EMPLID
ANDG.BALANCE_YEARIN(1998,1999)
ANDG.BALANCE_PERIODIN(1,2,3,8,9,10,11,12)
ANDG.BALANCE_PERIOD=(SELECTMAX(BALANCE_PERIOD)FROMPS_EARNINGS_BALGG
WHEREGG.COMPANY=G.COMPANY
ANDGG.ERNCD=G.ERNCD
ANDGG.EMPLID=G.EMPLID
ANDGG.BALANCE_ID=G.BALANCE_ID
ANDGG.BALANCE_YEAR=G.BALANCE_YEAR)
AND(A.EFFDT=
(SELECTMAX(EFFDT)FROMPS_K_JOB_QUERY_VW
WHEREA.EMPLID=EMPLID
ANDA.EMPL_RCD#=EMPL_RCD#
ANDEFFDT<=SYSDATE)
ANDA.EFFSEQ=
(SELECTMAX(EFFSEQ)FROMPS_K_JOB_QUERY_VW
ANDA.EFFDT=EFFDT)
ANDB.EFFDT=
(SELECTMAX(EFFDT)FROMPS_COMPANY_TBL
WHEREB.COMPANY=COMPANY
ANDEFFDT<=A.EFFDT)
ANDC.EFFDT=
(SELECTMAX(EFFDT)FROMPS_LOCATION_TBL
WHEREC.LOCATION=LOCATION
ANDEFFDT<=A.EFFDT)
ANDD.EFFDT=
(SELECTMAX(EFFDT)FROMPS_K_CO_WCOMP_TBL
34
WHERED.COMPANY=COMPANY
ANDD.WORKERS_COMP_CD=WORKERS_COMP_CD
ANDEFFDT<=A.EFFDT)
ANDF.EFFDT=
(SELECTMAX(EFFDT)FROMPS_PAYGROUP_TBL
WHEREF.COMPANY=COMPANY
ANDF.PAYGROUP=PAYGROUP
ANDEFFDT<=A.EFFDT)
ANDH.EFFDT=
(SELECTMAX(EFFDT)FROMPS_EARNINGS_TBL
WHEREH.ERNCD=ERNCD
ANDEFFDT<=SYSDATE)
)
ExecutionPlanfortheabovequery:
QueryPlan
CONCATENATION
FILTER
NESTEDLOOPS
NESTEDLOOPS
NESTEDLOOPS
NESTEDLOOPS
NESTEDLOOPS
NESTEDLOOPS
NESTEDLOOPS
NESTEDLOOPS
NESTEDLOOPS
NESTEDLOOPS
35
TABLEACCESSBYINDEXROWID
PS_SCRTY_TBL_SUPRV
INDEXRANGESCAN
PSASCRTY_TBL_SUPRV
PS_PERSONAL_DATA
TABLEACCESSBYINDEXROWIDPS_PAYGROUP_TBL
INDEXRANGESCANPS#PAYGROUP_TBL
TABLEACCESSBYINDEXROWIDPS_K_CO_WCOMP_TBL
INDEXRANGESCANPS_K_CO_WCOMP_TBL
TABLEACCESSBYINDEXROWIDPS_EARNINGS_BAL
INDEXRANGESCANPSBEARNINGS_BAL
TABLEACCESSBYINDEXROWIDPS_EARNINGS_TBL
INDEXRANGESCANPS#EARNINGS_TBL
TABLEACCESSBYINDEXROWIDPS_LOCATION_TBL
INDEXRANGESCANPS#LOCATION_TBL
TABLEACCESSBYINDEXROWIDPS_COMPANY_TBL
INDEXRANGESCANPS#COMPANY_TBL
SORTAGGREGATE
INDEXRANGESCANPS_EARNINGS_BAL
SORTAGGREGATE
INDEXRANGESCANPSDJOB
SORTAGGREGATE
SORTAGGREGATE
36
INDEXRANGESCANPS_COMPANY_TBL
SORTAGGREGATE
INDEXRANGESCANPS_LOCATION_TBL
SORTAGGREGATE
SORTAGGREGATE
INDEXRANGESCANPS_PAYGROUP_TBL
SORTAGGREGATE
INDEXRANGESCANPS_EARNINGS_TBL
SORTAGGREGATE
SORTAGGREGATE
FILTER
TABLEACCESSBYINDEXROWIDPSTREENODE
INDEXRANGESCANPSCPSTREENODE
INDEXRANGESCANPS#SCRTY_TBL_SUPRV
FILTER
SORTAGGREGATE
SORTAGGREGATE
SORTAGGREGATE
SORTAGGREGATE
SORTAGGREGATE
SORTAGGREGATE
37
SORTAGGREGATE
SORTAGGREGATE
SORTAGGREGATE
SORTAGGREGATE
FILTER
NESTEDLOOPS
INDEXRANGESCANPS#COMPANY_TBL
NESTEDLOOPS
INDEXRANGESCANPS#LOCATION_TBL
NESTEDLOOPS
INDEXRANGESCANPS#EARNINGS_TBL
NESTEDLOOPS
INDEXRANGESCANPSBEARNINGS_BAL
NESTEDLOOPS
NESTEDLOOPS
38
NESTEDLOOPS
NESTEDLOOPS
NESTEDLOOPS
NESTEDLOOPS
PS_SCRTY_TBL_SUPRV
INDEXRANGESCAN
PSASCRTY_TBL_SUPRV
PS_PERSONAL_DATA
Note: This plan is truncated for printing here since it repeats itself for every employee in the
IN clause. The complete execution plan, if printed goes onto about 5-6 pages. The RULE based
optimizer is not able to handle an IN clause too well.
Figure 17: Details of Query2 (with hinted PS_PERSONAL_SRCH_Q

view)
SELECTA.EMPLID,PS_RAM_NAME_LOOKUP(A.EMPLID),E.SSN,A.COMPANY,
B.DESCR,C.STATE,A.WORKERS_COMP_CD,D.DESCR,A.PAYGROUP,F.DESCR,G.ERNCD,
H.DESCR,G.BALANCE_YEAR,G.BALANCE_PERIOD,G.GRS_MTD,B.COMPANY,
TO_CHAR(B.EFFDT,'YYYYMMDD'),F.COMPANY,F.PAYGROUP,TO_CHAR(F.EFFDT,'YYYYMMDD'),
H.ERNCD,TO_CHAR(H.EFFDT,'YYYYMMDD')
FROMPS_K_JOB_QUERY_VWA,
PS_PERSONAL_SRCH_QA1,
PS_COMPANY_TBLB,PS_LOCATION_TBLC,
39
PS_K_CO_WCOMP_TBLD,PS_PERSONAL_DATAE,
PS_PAYGROUP_TBLF,PS_EARNINGS_BALG,PS_EARNINGS_TBLH
WHEREWHEREA.EMPLIDIN
('00156','00871','00959','01203','01706','01749','01915','02316','02475','02628','02967','
03403','03426','03476','03583','04267','05130','05423','06107','06108','06303','06576','06
771','07252','07282','07422','07463','08380','09165','09340','09373','09442','09531','0992
6','10576','11496','11886','12028','12038','12336','12530','13050','14336','14471','14972'
,'16956','18069','18070','18362','18442','18816','19243','19762','19857','19858','20560','
21517','21687','21791','22589','22718','22910','23097','23936','23960','24033','24046','24
108','24115','24168','24308','24404','24444','24768','24921','25028','25137','25171','2546
9','25480','26239','26980','27160','27217','27218','27410','27569','27729','28065','29750'
,'30950','31508','31960','32249','32470','32825','32952','33803','33835','37430','37865','
38266','38543','45433','49184','54100','54840','63550','68060','68690','70137','71818','75
420','76390','78090','80500','81220','83696','83985','89930','93892','97288')
ANDB.COMPANY=A.COMPANY
ANDC.LOCATION=A.LOCATION
ANDD.WORKERS_COMP_CD=A.WORKERS_COMP_CD
ANDD.COMPANY=A.COMPANY
ANDE.EMPLID=A.EMPLID
ANDF.PAYGROUP=A.PAYGROUP
ANDF.COMPANY=A.COMPANY
ANDH.ERNCD=G.ERNCD
ANDA.EMPLID=A1.EMPLID
ANDA.EMPL_RCD#=A1.EMPL_RCD#
ANDA1.OPRID='BATRAR'
ANDE.EMPLID=A1.EMPLID
ANDG.EMPLID=A.EMPLID
ANDG.BALANCE_YEARIN(1998,1999)
ANDG.BALANCE_PERIODIN(1,2,3,8,9,10,11,12)
ANDG.BALANCE_PERIOD=(SELECTMAX(BALANCE_PERIOD)FROMPS_EARNINGS_BALGG
WHEREGG.COMPANY=G.COMPANY
ANDGG.ERNCD=G.ERNCD
ANDGG.EMPLID=G.EMPLID
ANDGG.BALANCE_ID=G.BALANCE_ID
40
ANDGG.BALANCE_YEAR=G.BALANCE_YEAR)
AND(A.EFFDT=
(SELECTMAX(EFFDT)FROMPS_K_JOB_QUERY_VW
ANDEFFDT<=SYSDATE)
ANDA.EFFSEQ=
(SELECTMAX(EFFSEQ)FROMPS_K_JOB_QUERY_VW
ANDA.EFFDT=EFFDT)
ANDB.EFFDT=
(SELECTMAX(EFFDT)FROMPS_COMPANY_TBL
WHEREB.COMPANY=COMPANY
ANDEFFDT<=A.EFFDT)
ANDC.EFFDT=
(SELECTMAX(EFFDT)FROMPS_LOCATION_TBL
WHEREC.LOCATION=LOCATION
ANDEFFDT<=A.EFFDT)
ANDD.EFFDT=
(SELECTMAX(EFFDT)FROMPS_K_CO_WCOMP_TBL
WHERED.COMPANY=COMPANY
ANDD.WORKERS_COMP_CD=WORKERS_COMP_CD
ANDEFFDT<=A.EFFDT)
ANDF.EFFDT=
(SELECTMAX(EFFDT)FROMPS_PAYGROUP_TBL
WHEREF.COMPANY=COMPANY
ANDF.PAYGROUP=PAYGROUP
ANDEFFDT<=A.EFFDT)
ANDH.EFFDT=
(SELECTMAX(EFFDT)FROMPS_EARNINGS_TBL
41
WHEREH.ERNCD=ERNCD
ANDEFFDT<=SYSDATE)
)
ExplainPlanfortheaboveSQL:
QueryPlan
FILTER
NESTEDLOOPS
NESTEDLOOPS
NESTEDLOOPS
NESTEDLOOPS
NESTEDLOOPS
NESTEDLOOPS
NESTEDLOOPS
NESTEDLOOPS
NESTEDLOOPS
NESTEDLOOPS
INLISTITERATORCONCATENATED
SORTAGGREGATE
SORTAGGREGATE
INDEXRANGESCAN
PS_COMPANY_TBL
42
SORTAGGREGATE
INDEXRANGESCANPS#K_CO_WCOMP_TBL
SORTAGGREGATE
SORTAGGREGATE
SORTAGGREGATE
TABLEACCESSBYINDEXROWIDPS_SCRTY_TBL_SUPRV
INDEXRANGESCANPSASCRTY_TBL_SUPRV
FILTER
SORTAGGREGATE
SORTAGGREGATE
43
TABLEACCESSFULLPS_K_CO_WCOMP_TBL
SORTAGGREGATE
SORTAGGREGATE
Note: The IN statement is handled very well by the Cost-Based Optimizer and the INLIST
Iteration works wonders .
7.4 Final Results

The results were excellent. All adhoc queries referencing the 7 views started to use the Cost
Based Optimizer and run-times improved across the board. A total of 7 views were modified with
the same hint. At the time of this writing, Raman is working on tuning the payroll jobs. Once
that is complete, he plans to implement the Cost-Based Optimizer for the instance rather than just
hinting the views.
8 Conclusions and Summary

Implementing the Oracle Cost Based Optimizer can be of significant benefit to Peoplesoft
systems. The first_rows method will generally provide significant performance improvements for
online users. In addition, with careful tuning and indexing, the results of batch and DSS jobs can
also experience great improvements. Finally, one must realize that the CBO is the way of the
future. Oracle has set a clear path toward making the CBO the heart of future performance
enhancements for its flagship database product. As such, it will be a minimum requirement that
systems implement the CBO in order to reap the benefits of improvements in the Oracle RDBMS.
44
9 Appendix A: Scripts
allActive.sql
Thisroutinewillproducealist
ofallactiveuserscurrentlyconnectedtothe
system.Thelistwillindicatevarious
thingsincludingthestatusoftheconnection,
thetypeofcommandcurrentlybeingexecuted,
andwhetherthesessionisblockedbyanother
user.
authorJamesE.Strange
settimingoff
setpauseoff
setpagesize9000
setlinesize100
columnsidheadingSIDformat999
columnspidheadingOSPIDformat999999
columnusernameheadingUSERformatA10
columnosuserheadingOSUSERformatA10
columnmachineheadingMACHINEformatA22
columnstatusheadingSTATUSformatA8
columnnameheadingCOMMANDformatA15
columnlockwaitformatA8
selects.sid,proc.spid,s.username,s.osuser,
s.machine,s.status,a.name,nvl(s.lockwait,'NO')BLOCKED
fromv$sessions,audit_actionsa,
v$processproc
wherea.action=s.command
45
andproc.addr=s.paddr
ands.usernameisnotnull
ands.status='ACTIVE'
orderbys.status,s.sid
/
sql.sql
Thisroutineliststhesqlforaselected
sid.Thesecondsecondquerywillindicate
theoptimizermodeandthenumberofrows
processedbythequery.
selecttext.sql_text
from
v$sqltext_with_newlinestext,v$sessionsess
where
text.address=sess.sql_addressand
text.hash_value=sess.sql_hash_valueand
sess.sid=&sid
orderbypiece
/
checkLatches.sql
Author:JamesE.Strange
Thisscriptdisplaysthestatusoflatches.Itlooksatthemfrom3differentways.
Thefirstsectionsimplyliststhelatchstatistics.
Thesecondsectionlooksatthewillingtowaitstats.
Thelastsectionlooksatthenotwillingtowaitpointofview.
46
setpagesize900
setlinesize120
columnnameformatA40
selectl.latch#latch#,l.missesmisses,l.getsgets,n.namenamefromv$latchl,
v$latchnamenwheren.latch#=l.latch#
orderbyname
/
selectnamelatch_name,gets,misses,
round((getsmisses)/decode(gets,0,1,gets),3)hit_ratio,
sleeps,round(sleeps/decode(misses,0,1,misses),3)
"SLEEPS/MISS"
fromv$latch
wheregets!=0
orderbyname
/
selectnamelatch_name,immediate_getsnowait_gets,
immediate_missesnowait_miss,
round((immediate_gets/(immediate_gets+immediate_misses)),3)nowait_hit_ratio
fromv$latch
whereimmediate_gets+immediate_misses!=0
orderbyname
/
makeExplain.sql
Thisroutineliststhesqlforaselected
sidintoafile.Theoutputisformattedto
makeitrelativelysimpletocreateafileto
useforgettinganexplainplan.Thisscript
47
assumesthefilenameshouldbeoftheform'test#'
wherethe#isanumber.Thescriptwillprompt
foranumbertosubstituteforthe#.
setechooff
setheadingoff
setlinesize64
spooltest.sql
select'explainplansetstatement_id=''test&1''for'fromdual;
select
rtrim(text.sql_text)
from
v$sqltext_with_newlinestext,v$sessionsess
where
text.address=sess.sql_addressand
text.hash_value=sess.sql_hash_valueand
sess.sid=&sid
orderbypiece
;
select';'fromdual;
spooloff
48
10 Appendix B: New Indexes

What follows is a partial list of the custom indexes added to Peoplesoft at Jims employer in order
to support various batch and reporting jobs:
TABLE_NAME
INDEX_NAME
COLUMN_NAME
PS_EMPLOYEES
PS_EMPLOYEES1_NDX
GL_PAY_TYPE
PS_EMPLOYEES
EMPL_STATUS
PS_EMPLOYEES
EMPLID
PS_EMPLOYEES
EMPL_RCD#
PS_EMPLOYEES
EFFDT
PS_EMPLOYEES
EFFSEQ
PS_JOB
COMPANY
PS_JOB1_NDX
PS_JOB
PAYGROUP
PS_JOB
GL_PAY_TYPE
PS_JOB2_NDX
PS_JOB
EMPLID
PS_JOB
GRADE
PS_JOB
SAL_ADMIN_PLAN
PS_JOB
POSITION_NBR
PS_JOB
EMPL_RCD#
PS_JOB
EFFDT
PS_PAY_DEDUCTION
PS_PAY_DEDUCTION1_NDX
DEDCD
PS_PAY_LINE
PS_PAY_LINE1_NDXEMPLID
PS_PAY_LINE
EMPL_RCD#
PS_PAY_LINE
COMPANY
PS_PAY_LINE
PAYGROUP
PS_PAY_LINE
PAY_END_DT
PS_PAY_LINE
OFF_CYCLE
49
PS_PAY_LINE
PS_PAY_LINE2_NDXEMPL_RCD#
PS_PAY_LINE
EMPLID
PS_PAY_LINE
COMPANY
PS_PERSONAL_DATA
PER_STATUS
PS_PERSONAL_DATA1_NDX
PS_PERSONAL_DATA
PER_TYPE
PS_PERSONAL_DATA
EMPLID
PS_W2_EE
EMPLID
PS_W2_EE1_NDX
PS_W2_EE
PROCESS_FLAG
11 Appendix C: Test Environments

Production Online Environment for Batch and Online Case Studies
DatabaseServer
HPUX10.20
T600system
6CPUs
3.5GigabytesofRAM
Oracle7.3.4.2
SGA=500MB
OptimizerMode
First_Rows
ClientMachine
CompaqArmada
333MZCPU
96MBRAM
ApplicationServer
Coresidentondatabaseserver
DatabaseSize
30Gig17000employees
PSApplications
PSVersion
HR,Payroll,Benefits,TimeandLabor,SelfService
7.02
Production Batch Environment for DSS Case Study

DatabaseServer
SunSolaris2.5.1
E5000
4CPUs
3.0GigabytesofRAM
50
Oracle8.0.5
SGA=680MB
OptimizerMode
Rule
ClientMachine
DellLaptop
300IIMZCPU
128MBRAM
ApplicationServer
Tuxedo,Logicalthreetier,Coresidentondatabaseserver
DatabaseSize
26Gigapprox.13000employees
PSApplications
HR,Payroll,Benefits
PSVersion
7.03
12 Appendix D: Recommended References

1. "Tuning Oracle for PeopleSoft Applications", Jacob C. Thomas, IOUG(A)-Live
1998, Orlando, Florida.
2. Oracle Performance Tuning, Second Edition, Gurry and Corrigan, O'Reilly
Books (1996)
3. Oracle 8 Server Tuning Manual, Oracle 8 Documentation, Oracle
Corporation (June 1998)
4. Oracle Performance Tuning Tips and Techniques, Niemic Rich et. al, Oracle
Press,McGraw Hill (199
51

2000-01-28 Strange - Perf. For Peoplesoft Apps

Hochgeladen von

Dokumentinformationen

Originaltitel

Copyright

Verfügbare Formate

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

Verfügbare Formate

2000-01-28 Strange - Perf. For Peoplesoft Apps

Hochgeladen von

Copyright:

Verfügbare Formate

Performance Enhancement Techniques for

Peoplesoft Applications on Oracle Databases

General Tuning and Architectural Considerations:..................................................................4

Identifying SQL and Performance Bottlenecks:......................................................................4

The Oracle Optimizer..............................................................................................................5

Driving Tables, Join Orders and Optimizer Modes:..........................................................6

Choosing an Oracle Optimizer:......................................................................................10

Setting the Optimizer Mode:...................................................................................11

Ongoing Maintenance with the Cost-Based Optimizer:..........................................13

Preparation Before Tuning:.....................................................................................13

Payroll Calc Case Study.................................................................................................15

Decision Support System.......................................................................................................30

The Infamous Security Tree...........................................................................................31

Conclusions and Summary....................................................................................................44

Appendix B: New Indexes.................................................................................................48

Appendix C: Test Environments........................................................................................50

Appendix D: Recommended References............................................................................51

2 General Tuning and Architectural Considerations:

3 Identifying SQL and Performance Bottlenecks:

4 The Oracle Optimizer

Available access path for the statement

Ranks of the access paths

4.1 Driving Tables, Join Orders and Optimizer Modes:

Figure 1: A simple query joining three-tables and the RBO execution

In the figure, the first table processed is PS_PERSONAL_DATA, followed by

Figure 3: Query with cost-based optimizer

4.2 Choosing an Oracle Optimizer:

4.2.1 Setting the Optimizer Mode:

4.2.2 Collecting Statistics:

4.2.3 Ongoing Maintenance with the Cost-Based Optimizer:

4.2.4 Preparation Before Tuning:

5.2 Case Study

employee contact info

current job- benefits partic 1st

6.2 Payroll Calc Case Study

6.2.1 Actions taken

Figure 7: Calc Tables

Figure 8: Initial Explain

Figure 12: PS_PAY_LINE1_NDX

Figure 13: Final Tkprof for CALC

6.3 Final Results

7 Decision Support System

7.2 The Infamous Security Tree

7.3 Case Studies

Results of 12 famously "bad" queries

Query Query Query Query Query Query Query Query Query

5.44 122.53 33.15 2.21

3.09 36.25 2.85 228.00 0.51

0.15 14.00 0.09 128.00 0.11

7.00 48.00 6.54

Figure 14: Run-time comparison of 12 queries between Rule-Based

Figure 15:PS_PERSONAL_SRCH View

Figure 16: Details of Query2 (without hinted PS_PERSONAL_SRCH_Q

Figure 17: Details of Query2 (with hinted PS_PERSONAL_SRCH_Q

7.4 Final Results

8 Conclusions and Summary

10 Appendix B: New Indexes

11 Appendix C: Test Environments

Production Batch Environment for DSS Case Study

12 Appendix D: Recommended References

Das könnte Ihnen auch gefallen