Sie sind auf Seite 1von 27

An Oracle White Paper

June, 2012

Siebel CRM BI Publisher Integration


Performance
Oracle White Paper Siebel CRM BI Publisher Integration Performance

Executive Overview ................................................................................................ 2


Tests 1 & 2 - Service Request Statistics report ....................................................... 8
Tests 3 & 4 - Service Request Statistics report ..................................................... 12
Tests 5 & 6 - Account Detail report ....................................................................... 14
Tests 7 & 8 - Account List report........................................................................... 18
Tests 9 - Account List Report ............................................................................... 22
Tests 10 & 11 - Account List Report (Normal Query v Selected Records Query) .. 24
Oracle White Paper Siebel CRM BI Publisher Integration Performance

Executive Overview
The purpose of the technical document is to detail the performance information for Siebel
8.1.1.7 Reporting BI Publisher integration.

Test Criteria

Goal is for < 10 seconds response time with simple reports (1 page, < 1000 records)

Complex reports with large integration objects (IO), aggregation for small data set
would expect 30-60secs.

Large data set would expect no failures, but realistic generation time.

For extremely large, long running report tests the goal is for no report failures (large
datasets or many small reports and large concurrency).

Test Cases

The test scenarios are designed to test the following common customer scenarios:

Impact on response times for both simple and more complex reports, when scaling
up large numbers of concurrent users running reports for a period of time.

Impact of response times for both simple and more complex reports, when running
with large data sets for a period of time.

Difference between immediate and scheduled report generation when running a


report with many records (50,000).

Impact on the BI Publisher server and response times when running a report that
contains many aggregations. Also running the same test and increasing the data
size.

2
Oracle White Paper Siebel CRM BI Publisher Integration Performance

Difference between a normal query and selected records query when running a
report.

Impact on response times when generating more complex reports with a larger
Integration Object and data shape.

Reference tests using a standard Account List report with different levels of
concurrency and data set.

The report templates used for the tests are included with this technical document.

Conclusion

Following performance testing are the following conclusions:

Average response times of less than 5 seconds were achieved running simple
reports with 100 records, with many aggregations and also with 100 user
concurrency.

Average response times of less than 10 seconds were achieved running simple
reports with 1000 records, with many aggregations and also with 50 user
concurrency.

Average response time degrades as the high levels of concurrency are increased.
Increasing the number of XMLP Components improves response times in this high
concurrency scenario.

Response time for report generation with large data sets is affected more by the
number of pages than the number of aggregations in the report template.

Time consuming reports with high concurrency will possibly require increasing
WinInet settings. This should be specified on the operating system of the machine
where the XMLP components are running. WinInet was set to 2 during tests
(recommended).

3
Oracle White Paper Siebel CRM BI Publisher Integration Performance

The Average response time has improved considerably from Siebel 8.1.1 - for short
Accounts List reports for 10, 25, and 100 concurrent users times are 5.17, 5.05, 5.79
seconds respectively in Siebel 8.1.1.7.

The time to generate a Selected Records query for a report generation is higher
than a normal query for report generation. Additional memory usage was observed
generating reports in scheduled mode as compared to non-scheduled mode (~450
mb / ~50% more) was observed when testing a large data set report.

Notes

During the testing a SQL performance issue was encountered (bug # 13972140).
With the fix, Database CPU improved by ~60% and Average Response Time
improved by ~19% for 300 user concurrency with 100 SR records scenario. Note that
this bug might only be encountered running reports over large user concurrency. In
general this bug is unlikely to be encountered in a typical deployment.

During the testing a memory leak issue was encountered (bug # 13824469). The fix
improved stability when running large data set reports under concurrency. Note that
this bug might only be encountered running reports with a large data set (50,000
records) and high concurrency. In general this bug is unlikely to be encountered in a
typical deployment.

References

For more information on Siebel CRM BI Publisher integration performance


recommendations please refer to:

Improving the performance of Siebel BI Publisher Report Generation (Doc ID


1392449.1)

Oracle BI Publisher Best Practices

4
Oracle White Paper Siebel CRM BI Publisher Integration Performance

http://www.oracle.com/technetwork/middleware/bi-publisher/overview/oracle-bi-
publisher-best-practices-133345.pdf

Environment Details

Component Product CPU(s) & Memory OS (version / Patch)


Application / Siebel 8.1.1.7 2 X 3GHz HT enabled , W2K3 Server Enterprise
Gateway [21238]SIA Intel Xeon CPU; 8 GB Edition SP2 ;
Server RAM MSSQL Native client -
2005.90.3042
Web Server Siebel 2 X 3GHz HT enabled , W2K3 Server Enterprise
8.1.1.7 [21238]SIA Intel Xeon CPU; 8 GB Edition SP2 ;
RAM IIS 6.0.3790.3959
Database Microsoft SQL 2 X 3GHz HT enabled , W2K3 Server Enterprise
Server Server Intel Xeon CPU; 8 GB Edition SP2 ;
2005 (811_dump_r RAM SQL Server
eindexed) 9.00.4060.00 - SP2
(Enterprise Edition)
BI Publisher BI Publisher 2 X 3GHz HT enabled , W2K3 Server Enterprise
Server 10.1.3.4.1 Intel Xeon CPU; 8 GB Edition SP2 ;
RAM BIP 10.1.3.4.1
Oracle 2 x 2.93 GHz, Intel W2K3 Server Enterprise
Application Test Xeon CPU; 6 GB RAM Edition SP2 ;
Suite Server OATS Version 9.3.1

Test Plan Overview

# Scenario Concurrency Report & UI Flow Data Volumes


1 Small complex 1, 10, 50, Service Request 100 SRs, some with Status =
report (1 page) 100, 200, 300 View > Service Closed/Open. Date Created
for 100 Request Statistics required.
records report Actual Data Shape Used: 100
containing 10 SRs
aggregations
2 Small complex 1, 10, 50, 100 Service Request 1000 SRs, some with Status =
report (1 page) View > Service Closed/Open. Date Created
for 1000 Request Statistics required.
records report Actual Data Shape Used: 1000
containing 10 SRs
aggregations

5
Oracle White Paper Siebel CRM BI Publisher Integration Performance

3 Large complex 1 Service Request 50000 SRs, some with Status =


report (1 page) View > Service Closed/Open. Date Created
for 50000 Request Statistics required.
records report Actual Data Shape Used: 50000
containing 10 SRs
aggregations
4 Large complex 1 Service Request 50000 SRs, some with Status =
report (1 page) View > Service Closed/Open. Date Created
for 50000 Request Statistics required.
records report Actual Data Shape Used: 50000
containing 10 Sanity test SRs
aggregations for scheduling. No
with scheduled duration or
mode concurrency. Just
run a few single
user iterations.
5 Simple report 1, 10, 25, 50, Account List View 100 Accounts, with 10 Contacts
with IO 100 > Account Detail and 10 Activities and 10
containing report Opportunities for each Account.
multiple ICs. Actual Data Shape Used: 100
Test with Accounts with 10/10/10
standard associations
Master/Detail (Contacts/Activities/Opportunitie
report layout s)
for 100
Account
records
6 Simple report 1, 10 Account List View 1000 Accounts, with 10
with IO > Account Detail Contacts and 10 Activities and
containing report 10 Opportunities for each
multiple ICs. Account.
Test with Actual Data Shape Used: 1000
standard Accounts with 10/10/10
Master/Detail associations
report layout (Contacts/Activities/Opportunitie
for 1000 s)
Account
records
7 Report with 1, 10, 25, Account List View 100 Accounts. The related IO
simplified IO 100 > Account List will only draw from Accounts.
(Account list). report Actual Data Shape Used: 100
Test with Accounts with 10/10/10
standard associations
Account List (Contacts/Activities/Opportunitie
report layout s)
for 100

6
Oracle White Paper Siebel CRM BI Publisher Integration Performance

records
8 Report with 1, 10, 25, 50, Account List View 1000 Accounts. The related IO
simplified IO 100 > Account List will only draw from Accounts.
(Account list). report Actual Data Shape Used: 1000
Test with Accounts with 10/10/10
standard associations
Account List (Contacts/Activities/Opportunitie
report layout s)
for 1000
records
9 Report with 1 Account List View 10000 Accounts. The related IO
simplified IO > Account List will only draw from Accounts.
(Account list). report
Test with
standard
Account List
report layout
for 10000
records
10 Report with 1 Account List View 50 Accounts. The related IO will
simplified IO > Account List only draw from Accounts.
(Account list) - report Actual Data Shape Used: 50
Normal report Accounts
query. Test
with standard
Account List
report layout
for 50 rows
11 Report with 1 Account List View 50 Accounts. The related IO will
simplified IO > Account List only draw from Accounts. We
(Account list) - report just set the flag to Selected
Selected Records = True
Records report Actual Data Shape Used: 50
query. Test Accounts
with standard
Account List
report layout
for 50 rows

7
Oracle White Paper Siebel CRM BI Publisher Integration Performance

Tests 1 & 2 - Service Request Statistics report

The purpose of this test was to validate the impact on the response time and the BI Publisher
server resources running a report with varied concurrency, small dataset and with a report that
contains many complex aggregations.

Notes
A 1000 SRs data shape was generated with 1 Contact and 1 Account associated to each SR.
Service Request Statistics report is a single page layout with 14 aggregations.
Both 100 and 1000 records report tests produced only a single page (PDF files size is ~4KB).
No slow SQL statements were found for both 100 and 1000 records scenarios.
All statistics were calculated for a 30 mins steady state (45 mins - 1 hour 15 mins).

Key Observations
The Average Response Time for Service Request Statistics Report with 100 records for 10,
50, and 100 users is less than 5 seconds. For 200 and 300 concurrent users average response
times are 20 and 31 seconds respectively.
The Average Response Time for Service Request Statistics Report with 1000 records for 10
and 50 users are 8.47 and 9.67 seconds respectively. The average response time for 100
concurrent users is 21.26 seconds.
The Average Response Time for Service Request Statistics Report with 100 and 1000 records
degrades with an increase in user concurrency. However, increasing the number of XMLP
Components improves the Average Response Time dramatically. This can be seen by
comparing the Average Response Time with Test # 5 and Test # 12.
With 100 SRs and 300 concurrent users the average Application CPU was ~56%. Beyond 300
users the Average Response Time degraded to ~31 seconds.
With 1000 SRs and 100 concurrent users the average Application CPU was ~81%. Beyond
100 users the Average Application CPU reached ~81%.
There were no hangs observed during the test.
The biggest impact on the Average Response Time was due to concurrency. With regards to
report aggregations, the impact on the BI Publisher resources was minimal.

8
Oracle White Paper Siebel CRM BI Publisher Integration Performance

Increasing the number of shared database connections doesnt have much impact on the
response time.
The Average Response Time for Service Request Statistics report with 100 records for 10, 50,
100 and 300 users (with 5 and 6 XMLP OMs) are below 10 seconds while the Average
Response Time for 200 users (with 2 XMLP OMs) is above 10 seconds.
The Average Response Time for Service Request Statistics report with 1000 records for 10
and 50 users are below 10 seconds while the Average Response Time for the 100 users is
above 10 seconds.
The Average Response Time for Service Request Statistics Report with 100 and 1000 records
degrades with increase in user load.
During the testing a SQL performance issue was found in the XMLP report generation, with
the SQL performance fix, Database CPU improved by ~60% for 300 users with 100 SR
records scenario. Average Response Time improved by ~19% (Tests #9 vs #12).

Summary

Test Generate Report Avg CPU Usage (%) Siebel -


# Txn Total
users Avg Used
# Key Configuration
# KPI Memory
RT App Web DB BIP
SRs (Sec) (MB)
(Sec)

1 10 100 10 4.51 1.97% 0.46% 2.28% 0.69% 959.88 2 XMLP OM, 20 shared DB connections

2 50 100 10 4.42 8.91% 0.76% 4.63% 2.07% 1298.51 2 XMLP OM, 20 shared DB connections

3 100 100 10 4.84 18.06% 0.96% 8.03% 3.72% 1700.13 2 XMLP OM, 20 shared DB connections

4 200 100 10 20.41 35.67% 1.53% 14.55% 6.44% 2598.53 2 XMLP OM, 20 shared DB connections

5 300 100 10 91.48 37.19% 1.61% 13.98% 6.47% 3269.95 2 XMLP OM, 20 shared DB connections

6 300 100 10 31.78 56.73% 2.21% 19.36% 8.97% 3644.77 3 XMLP OM, 30 shared DB connections

7 300 100 10 9.71 63.39% 2.05% 53.88% 10.73% 2941.42 5 XMLP OM, 50 shared DB connections

8 300 100 10 7.78 61.68% 2.22% 54.50% 10.97% 2864.92 5 XMLP OM, 20 shared DB connections
6 failures

9 300 100 10 7.62 63.84% 2.64% 55.59% 11.51% 3073.95 6 XMLP OM, 30 shared DB connections
1 failure

10 300 100 10 256.54 18.05% 2.02% 43.30% 3.21% 2354.11 2 XMLP OM, 50 shared DB connections
The XMLP report output table was not
cleaned before this run.

11 300 100 10 92.96 38.04% 1.63% 26.71% 6.05% 2532.13 2 XMLP OM, 50 shared DB connections
No failures, consistency run after XMLP
report output table cleanup.

9
Oracle White Paper Siebel CRM BI Publisher Integration Performance

12 300 100 10 6.14 62.23% 2.25% 21.49% 11.35% 3063.02 6 XMLP OM, 30 shared DB connections
With SQL Performance fix:
DB CPU improved by ~60%
Avg RT is improved by 19%

13 10 1000 10 8.47 6.01% 0.39% 4.61% 2.21% 972.29 2 XMLP OM, 20 shared DB connections
No failures

14 50 1000 10 9.67 35.16% 0.79% 15.29% 10.13% 1312.10 2 XMLP OM, 20 shared DB connections
No failures

15 100 1000 10 21.26 81.07% 0.99% 28.95% 19.75% 1734.22 2 XMLP OM, 20 shared DB connections
No failures

Settings

# Component Configurations Value

1 EAI component - MaxMT, MinMT, MaxTasks, MinSharedDBCon, MaxSharedDbCon 1, 1, 110, 10, 10

2 XMLP component - MaxMT, MinMT, MaxTasks, MinSharedDBCon, MaxSharedDbCon 2, 2, 110, 20, 20

3 eHospitality OM component - MaxMT, MinMT, MaxTasks, MinSharedDBCon, MaxSharedDbCon 1, 1, 110, 10, 10

4 EAI HTTP Sleep Time XMLP Driver Service 20 Minutes

5 JVM heapsize on BIP 1.4 GB

6 DB polling Interval 1 second

7 Scalable Mode False

8 DSMaxCachedCursors for ServerDataSrc 0

9 DSMaxCachedDatasets for ServerDataSrc 0

10 EAI HTTP Sleep Time EAI HTTP Transport 20 Minutes

11 Minimal Tracing were enabled to get the statistics for the Siebel Server components

12 DSMaxFetchArraySize Default (0)

13 BIP Report Wait Time (System Preferences) 1000 seconds

14 Think time used in the script between the transactions/step-groups is 30 seconds (Dynamic Think Time
- 15% to 185%).

Setting EAI Sleep Time:

Add the following line in the PreInvokeMethod() of the Server script of "XMLP Driver
Service" business service.
Inputs.SetProperty("HTTPSleepTime", "1200000");
Add the following line in the PreInvokeMethod() of the Server script of "EAI HTTP
Transport" business service.

10
Oracle White Paper Siebel CRM BI Publisher Integration Performance

Inputs.SetProperty("HTTPSleepTime", "1200000");

JVM Heap Size:


Increase JVM Heap Size on the BI Publisher server. NOTE that this can be increased to
different levels depending on server. Windows = 1.4gb, Linux (can set > 2gb), 64 bit OS (can
set large 10-16gb).

11
Oracle White Paper Siebel CRM BI Publisher Integration Performance

Tests 3 & 4 - Service Request Statistics report

The purpose of this test was to validate the impact on the response time and the BI Publisher
server resources running a report with large data set of 50,000 records and with a report that
contains many complex aggregations. The goal was for the report generation to be stable without
any failures. The test was for both immediate report and scheduled report generation.

Notes
A 50000 Service Request data shape was used.
Service Request Statistics report contains a single page layout with 14 aggregations.
50000 records report contains a single page (PDF files size is ~4 kb).
No Slow SQL statements were found.
All statistics were calculated for a 30 mins steady state (15 mins - 45 mins).

Key Observations
The Average Response Time for the Service Request Statistics report with 50000 records
(non-Scheduled Mode and Scheduled Mode) for 1 user is less than 30 minutes.
There were no crashes or hangs noticed during this run.
Memory leak fix was applied

Summary
Generate
Avg CPU Usage (%)
Report Txn
Siebel -
# # Sched KPI Avg Total
We
user SR uled (Se RT App DB BIP Used Key Configuration
b
s s Mode c) (Sec) Memory
(MB)
1 XMLP OM, 10 shared
180 7.31 0.40 6.12 6.76 DB connections
1 50K No 284.12 674.30
0 % % % % No failures, with
memory leak fix
1 XMLP OM, 10 shared
180 6.77 0.55 5.00 4.30 DB connections
1 50K Yes 288.00 1015.72
0 % % % % No failures, with
memory leak fix

Settings

12
Oracle White Paper Siebel CRM BI Publisher Integration Performance

# Component Configurations Value

1 EAI component - MaxMT, MinMT, MaxTasks, MinSharedDBCon, MaxSharedDbCon 1, 1, 110, 10, 10

2 XMLP component - MaxMT, MinMT, MaxTasks, MinSharedDBCon, MaxSharedDbCon 1, 1, 55, 10, 10

3 eHospitality OM component - MaxMT, MinMT, MaxTasks, MinSharedDBCon, MaxSharedDbCon 1, 1, 110, 10, 10

4 EAI HTTP Sleep Time XMLP Driver Service 20 Minutes

5 JVM heapsize on BIP 1.4 GB

6 DB polling Interval 1 second

7 Scalable Mode True

8 DSMaxCachedCursors for ServerDataSrc, RptDataSrc 0

9 DSMaxCachedDatasets for ServerDataSrc , RptDataSrc 0

10 EAI HTTP Sleep Time EAI HTTP Transport 20 Minutes

11 Minimal Tracing were enabled to get the statistics for the Siebel Server components

12 DSMaxFetchArraySize -1

13 BIP Report Wait Time (System Preferences) 1000 seconds

14 Think time used in the script between the transactions/step-groups is 30 seconds (Dynamic Think
Time - 15% to 185%).

13
Oracle White Paper Siebel CRM BI Publisher Integration Performance

Tests 5 & 6 - Account Detail report

The purpose of this test was to validate the impact on the response time running a large report
over varied concurrency with a more complex data shape and increased Integration Object size.
Statistics were obtained at individual component level for analysis.

Notes
A 1000 Accounts data shape was generated, in which 10 Contacts, 10 Opportunities and 10
Activities are associated to each Account.
Account Detail report is a standard Master/Detail report layout. The master section shows
the Account information and detail part contains the list of child records (Contacts, Activities
and Opportunities)
100 Accounts detail report contains 102 pages (PDF files size is ~300 kb).
1000 Accounts detail report contains 1002 pages (PDF files size is ~3 mb).
No slow SQL statements were found for both 100 and 1000 records scenarios.
All statistics were calculated for a 30 mins steady state (15 mins - 45 mins for 1 user, 45 mins
- 1 hour 15 mins for 10, 25 and 50 users scenario).
A memory leak issue was resolved during reliability testing.

Key Observations
The Average Response Time for 1, 10, 25 users is below 60 seconds, the Average Response
Time for 50 users is ~66 seconds.
The Average Response Time degrades with increase in user load.
For 100 accounts with 50 concurrent users the Average Application CPU ~89%.
For 1000 accounts with 10 concurrent users the Average Application CPU ~64%.
There were no hangs during this run. No crash files found during the run.

14
Oracle White Paper Siebel CRM BI Publisher Integration Performance

Summary
Generate
Avg CPU Usage (%)
Report Txn
Siebel -
# # KPI Avg Total
Key
user Account (Se RT App Web DB BIP Used
Configuration
s s c) (Sec) Memory
(MB)
1 XMLP OM, 10
0.35 shared DB
1 100 60 19.10 2.52% % 1.35% 1.42% 706.41 connections
1 XMLP OM, 10
15.57 0.47 shared DB
10 100 60 20.63 % % 5.19% 9.86% 856.52 connections
2 XMLP OM, 20
43.91 0.58 18.54 26.09 shared DB
25 100 60 25.62 % % % % 1489.87 connections
2 XMLP OM, 20
89.48 2.00 19.70 49.21 shared DB
50 100 60 66.75 % % % % 1795.36 connections
1 XMLP OM, 10
165.7 0.35 shared DB
1 1000 60 7 7.45% % 4.05% 5.46% 827.34 connections
2 XMLP OM, 20
296.8 64.83 0.60 35.66 37.31 shared DB
10 1000 60 6 % % % % 1549.73 connections

Timing Breakup:

Timing Breakup - Consolidated Comparison Table

No. of Query Time Datac Datac Datac Datac Total Repor Total File Total RT
Acco Time taken hunk hunk hunk hunk time t Elaps Save Time from
unts for 1 uploa 2 uploa taken Gener ed Time taken OLT
Recor Data query d1 query d2 for ation Time for run
ds uploa time Time time Time dataq Time on the report
d to uery BIP report
BIP and Serve
serve Uploa r
r d

100 7 1 NA NA NA NA 8 13 6.563 NA 21 22.32


9

1000 NA NA 40 13 42 12 107 56 NA 3 167 169.3


2

15
Oracle White Paper Siebel CRM BI Publisher Integration Performance

Reliability Tests - Account Detail report

Notes
A memory leak of ~159KB/Iteration was resolved during initial investigation for eHospitality
OM.
For the first test runs (10 users, 1000 records) the statistics were calculated for 20 hours
steady state.
For the second and third test runs without the memory leak fix, the statistics were calculated
for 2 hours 30 mins steady state.
For the last test run (40 users, 100 records) the statistics were calculated for 18 hours steady
state.

Summary
Generate Report
Avg CPU Usage (%)
Txn
Siebel -
Memory
# # Total
KPI Avg RT Leak
use Accou App Web DB BIP Used Key Configuration
(Sec) (Sec) KB /
rs nts Memory
Iteration
(MB)
2 XMLP OM, 20 shared
61.41 0.66 36.91 38.08 DB connections
10 1000 60 305.48 2100.38 159.18
% % % % Bug# 13824469 for
memory leak
2 XMLP OM, 20 shared
91.21 0.94 34.29 47.07 DB connections
50 100 60 60.81 2223.21 159.76
% % % % Bug# 13824469 for
memory leak
2 XMLP OM, 20 shared
77.58 1.19 30.15 41.17 DB connections
40 100 60 38.92 2051.09 156.00
% % % % Bug# 13824469 for
memory leak
2 XMLP OM, 20 shared
44.80 0.71 60.03 23.73
40 100 60 147.65 2021.19 No leak DB connections
% % % %
With Memory Leak fix

Settings

# Component Configurations Value

1 EAI component - MaxMT, MinMT, MaxTasks, MinSharedDBCon, MaxSharedDbCon 1, 1, 110, 10, 10

2 XMLP component - MaxMT, MinMT, MaxTasks, MinSharedDBCon, MaxSharedDbCon 2, 2, 110, 20, 20

3 eHospitality OM component - MaxMT, MinMT, MaxTasks, MinSharedDBCon, MaxSharedDbCon 1, 1, 110, 10, 10

4 EAI HTTP Sleep Time XMLP Driver Service 20 Minutes

16
Oracle White Paper Siebel CRM BI Publisher Integration Performance

5 JVM heapsize on BIP 1.4 GB

6 DB polling Interval 1 second

7 Scalable Mode False

8 DSMaxCachedCursors for ServerDataSrc 0

9 DSMaxCachedDatasets for ServerDataSrc 0

10 EAI HTTP Sleep Time EAI HTTP Transport 20 Minutes

11 Minimal Tracing were enabled to get the statistics for the Siebel Server components

12 DSMaxFetchArraySize Default (0)

13 BIP Report Wait Time (System Preferences) 1000 seconds

14 Think time used in the script between the transactions/step-groups is 30 seconds (Dynamic Think
Time - 15% to 185%).

17
Oracle White Paper Siebel CRM BI Publisher Integration Performance

Tests 7 & 8 - Account List report

The purpose of this test was to provide a reference test using a standard and simple application
report. The test was run with varied concurrency and data size.

Notes
A 1000 accounts data shape was generated where 10 contacts, 10 opportunities and 10
activities are associated to each Account. The integration object used for the report only
contains the Account entity.
Account list report contains standard OOTB list report layout.
100 accounts list report contains 16 pages (PDF files size is ~47 kb).
1000 accounts list report contains 160 pages (PDF files size is ~447 kb).
No slow SQL statements were found for both 100 and 1000 records scenarios.
All statistics were calculated for 30 mins steady state (45 mins to 1 hour 15 mins).

Key Observations
The Average Response Time for Accounts List Report with 100 records for 10, 25, and 100
users are 5.17, 5.05, 5.79 seconds respectively.
The Average Response Time for Accounts List Report with 1000 records for 10, 25 and 50
users are 15.3, 16.9, 36.3 seconds respectively.
The Average Response Time for Accounts List Report with 1000 records degrades with
increase in user load.
For 100 accounts with 100 concurrent users the Average Application CPU ~21%.
For 1000 accounts with 100 concurrent users the Average Application CPU ~40%.
There were no hangs during this run.

Summary
Generate Report Txn Avg CPU Usage (%)

# # KPI Avg App Web DB BIP Siebel - Key


users Accounts (Sec) RT Total Used Configuration
(Sec) Memory
(MB)

18
Oracle White Paper Siebel CRM BI Publisher Integration Performance

10 100 60 5.17 2.58% 0.51% 2.27% 1.66% 964.97 2 XMLP OM, 20


shared DB
connections

25 100 60 5.05 5.33% 0.60% 3.09% 3.82% 1092.61 2 XMLP OM, 20


shared DB
connections

100 100 60 5.79 21.08% 1.10% 7.44% 13.89% 1697.82 2 XMLP OM, 20
shared DB
connections
Failures after
~1.5 hours due to
memory leak

100 100 60 5.77 22.15% 1.05% 7.48% 13.91% 1694.59 2 XMLP OM, 20
shared DB
connections
Failures after
~1.5 hours due to
memory leak

10 1000 60 15.30 8.20% 0.70% 4.39% 12.38% 1038.16 2 XMLP OM, 20


shared DB
connections

25 1000 60 16.93 20.53% 0.73% 8.10% 31.35% 1221.29 2 XMLP OM, 20


shared DB
connections

50 1000 60 36.33 40.09% 0.97% 13.80% 60.33% 1518.39 2 XMLP OM, 20


shared DB
connections

Settings
# Component Configurations Value

1 EAI component - MaxMT, MinMT, MaxTasks, MinSharedDBCon, MaxSharedDbCon 1, 1, 110, 10, 10

2 XMLP component - MaxMT, MinMT, MaxTasks, MinSharedDBCon, 2, 2, 110, 20, 20


MaxSharedDbCon

3 eHospitality OM component - MaxMT, MinMT, MaxTasks, MinSharedDBCon, 1, 1, 110, 10, 10


MaxSharedDbCon

4 EAI HTTP Sleep Time XMLP Driver Service 20 Minutes

5 JVM heapsize on BIP 1.4 GB

6 DB polling Interval 1 second

7 Scalable Mode False

8 DSMaxCachedCursors for ServerDataSrc 0

9 DSMaxCachedDatasets for ServerDataSrc 0

10 EAI HTTP Sleep Time EAI HTTP Transport 20 Minutes

11 Minimal Tracing were enabled to get the statistics for the Siebel Server components

19
Oracle White Paper Siebel CRM BI Publisher Integration Performance

12 DSMaxFetchArraySize Default (0)

13 BIP Report Wait Time (System Preferences) 1000 seconds

14 Think time used in the script between the transactions/step-groups is 30 seconds


(Dynamic Think Time - 15% to 185%).

Siebel 8.1.1 Performance for Accounts List report

Key Observations
It is important to note that in Siebel 8.1.1 performance testing was with a throughput
scenario, meaning there are no delays between the report requests, with continuous hits to the
server.
With 8.1.1.7 performance testing, a UI scenario was used which involves a dynamic think
time between each request.
The hardware used between 8.1.1 and 8.1.1.7 is different and considerably improved in 8.1.1.7
testing.
The Average Response Time for a shorter Accounts List report was 9 seconds in Siebel 8.1.1.
The Average Response Time for a medium size Accounts List report was 22 seconds in
Siebel 8.1.1.

Summary
Average response times from Siebel 8.1.1 BI Publisher integration with 10 user concurrency:
# of Call XMLP BIP Total (in
Account List report
Pages Center Component Server Seconds)

Shorter (2-5 pages) 3 7 1 1 9

Medium (50-100 pages) 82 6 5 11 22

Larger (800-1000 pages) 861 9 34 92 135

Hardware used in 8.1.1 performance testing:

Server CPU(s) Memory OS (version/Patch)

DB Server (Microsoft SQL Server 2005) 4 x 701 Mhz 3.65 GB Windows Server 2K3

Application / Gateway Server 4 x 701 Mhz 3.65 GB Windows Server 2K3

20
Oracle White Paper Siebel CRM BI Publisher Integration Performance

Web Server 2 x 1266 MHz 2 GB Windows Server 2K3

BI Publisher server 4 x 701 Mhz 3.65 GB Windows Server 2K3

21
Oracle White Paper Siebel CRM BI Publisher Integration Performance

Tests 9 - Account List Report

The purpose of this test was to provide a reference test using a standard and simple application
report. The test was run with a large data size (10000 records).

Notes
We have used a 10000 Accounts data shape.
Account List report contains standard list report layout.
10000 accounts list report contains 1591 pages (PDF files size is ~4.5 MB).
No slow SQL statements were found.

Key Observations
Average Response Time for Account List report with 10,000 records was 173.59 seconds, less
than 3 minutes.

Summary
Generate
Avg CPU Usage (%)
Report Txn
# # KPI Siebel - Total
Avg RT Key
user Account (Se App Web DB BIP Used
(Sec) Configuration
s s c) Memory (MB)
1 XMLP OM, 10
360 3.13 0.39 7.59 6.38
1 10K 173.59 699.83 shared DB
0 % % % %
connections

Settings
# Component Configurations Value

1 EAI component - MaxMT, MinMT, MaxTasks, MinSharedDBCon, MaxSharedDbCon 1, 1, 110, 10, 10

2 XMLP component - MaxMT, MinMT, MaxTasks, MinSharedDBCon, 2, 2, 110, 20, 20


MaxSharedDbCon

3 eHospitality OM component - MaxMT, MinMT, MaxTasks, MinSharedDBCon, 1, 1, 110, 10, 10


MaxSharedDbCon

4 EAI HTTP Sleep Time XMLP Driver Service 20 Minutes

5 JVM heapsize on BIP 1.4 GB

6 DB polling Interval 1 second

22
Oracle White Paper Siebel CRM BI Publisher Integration Performance

7 Scalable Mode True

8 DSMaxCachedCursors for ServerDataSrc, RptDataSrc (DSMaxFetchArraySize = -1) 0

9 DSMaxCachedDatasets for ServerDataSrc , RptDataSrc (DSMaxFetchArraySize = -1) 0

10 EAI HTTP Sleep Time EAI HTTP Transport 20 Minutes

11 Minimal Tracing were enabled to get the statistics for the Siebel Server components

12 DSMaxFetchArraySize Default (0)

13 BIP Report Wait Time (System Preferences) 1000 seconds

14 Think time used in the script between the transactions/step-groups is 30 seconds


(Dynamic Think Time - 15% to 185%).

23
Oracle White Paper Siebel CRM BI Publisher Integration Performance

Tests 10 & 11 - Account List Report (Normal Query v Selected


Records Query)

The purpose of this test was to see the difference running a standard Accounts List report in two
different modes, standard query and selected records query mode. For selected reports query
mode, a Selected Records version of the same report was executed with individual applet
records selected. For normal query mode, a standard applet query was executed.

Notes
Tests were performed using both a Cold/Warm state for the Object Manager to see any
difference on query caching. Different sets of records were also used for comparison.
An individual breakdown of the report generation timings was done to see where the most
time was taken during report generation.

Key Observations
Time to generate a Selected Records query for a report generation is higher than a normal
query for report generation.
Search Specification in Selected Query contains 50 individual Row IDs in the SearchSpec
text. Search Specification in Normal Query contains single SearchSpec text.
For 50 records using Selected Records Query Spec, using Warm OM, even though SQL
remains the same by using different bind variables, the SQL is taking ~15 sec due to multiple
OR conditions.

Summary
Normal Query Selected Records Query

Warm OM Cold OM Warm OM Warm OM - W/ Selection


of new set of records

# of Time # of Time # of Time # of Time


SQLs Taken(Sec) SQLs Taken(Sec) SQLs Taken(Sec) SQLs Taken(Sec)

Query Creation 0.000 4.000 0.000 1.000


End

Query Page Start


for loop index 0

24
Oracle White Paper Siebel CRM BI Publisher Integration Performance

SQLs 2 0.003 5 14.204 2 0.004 2 15.001

Query Page End 0.000 15.000 0.000 15.000


for loop index 0

Binary data 0.000 19.000 0.000 16.000


Generation End

RunBIPReport 4.000 18.000 4.000 3.000


end

Total Time to 4.000 63.000 4.000 19.000


Generate the
Report

Selected Records Query - ColdOM


Tracing done by executing the report with a cold OM (Just after bouncing the Siebel Server)

Selected Records Query - WarmOM


Tracing done by executing the report with a warm OM (from the 2nd Iteration with the same
selection of records)

Selected Records Query - WarmOM-W/New set of records


Tracing done by executing the report with a warm OM (from the 2nd Iteration with a
different selection of records )

Normal Query - WarmOM


Tracing done by executing the report with a warm OM (from the 2nd Iteration with the same
query)

25
Siebel CRM BI Publisher Integration
Performance
June, 2012
Author: John Bedford
Copyright 2009, Oracle and/or its affiliates. All rights reserved. This document is provided for information purposes only and
the contents hereof are subject to change without notice. This document is not warranted to be error-free, nor subject to any other
Oracle Corporation
warranties or conditions, whether expressed orally or implied in law, including implied warranties and conditions of merchantability or
World Headquarters
fitness for a particular purpose. We specifically disclaim any liability with respect to this document and no contractual obligations are
500 Oracle Parkway
formed either directly or indirectly by this document. This document may not be reproduced or transmitted in any form or by any
Redwood Shores, CA 94065
means, electronic or mechanical, for any purpose, without our prior written permission.
U.S.A.

Worldwide Inquiries: Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective

Phone: +1.650.506.7000 owners.

Fax: +1.650.506.7200
oracle.com 0109

Das könnte Ihnen auch gefallen