Oracle EPM Troubleshooting

& Infrastructure Best

Practices
Doug Fiedler
Principal Infrastructure Architect

Copyright 2014 Doug Fiedler, Edgewater Ranzal

About Edgewater Ranzal

15 Years
700+ clients
1000+ projects

Focus
Services
People
Methodology
Customers
Partnership

Copyright 2014 Doug Fiedler, Edgewater Ranzal

We offer a full spectrum of EPM/BI Services

Financial performance, Legal,
Segment & Mgmt Reporting, Financial Close
HFM Optimization, Performance Lab
SOX Compliance Support

Business
Intelligence

Consolidation

Installation, Upgrades,
Migration, System
Monitoring, Backup and
Recovery, Disaster Recovery,
Load Testing, Hardware
Sizing, Exalytics
Benchmarking

Dashboards & Scorecards, Financial Analytics

& Reporting, Operational Analytics, What-if
Analysis, Query & Reporting, Visual Exploration

Enterprise
Planning
Costing &
Profitability
Mgmt

Infrastructure

Data Integration, Financial Data

Management, Data Warehousing,
Master Data Management &DRM,
ETL Services, Automation

Strategic Finance, Planning, Budgeting,

Forecasting, Workforce Planning,
Capital Planning, Project Financial
Planning

Data
Services

Project
Management

Training &
Support Services

Key Teach Course Delivery: Planning, Essbase,

Financial Reporting, Smart View, HPCM, HFM, FDM,
DRM, OBIEE
Custom Training Delivery: Process & Reporting

HPCM Standard & Detailed

Models, Waterfall Allocations,
Activity Based Costing, Customer,
Product & LOB Profitability

Project/Program Mgmt, EPM

Road Maps, Application
Reviews, Business
Requirements, Process
Change, Documentation

Support Services Infrastructure &

Application Support Contracts
Copyright 2014 Doug Fiedler, Edgewater Ranzal

What version of EPM are you running today?

v11.1.2.3
V11.1.2.2
V11.1.2.1
V11.1.1.4/3 or earlier

Copyright 2014 Doug Fiedler, Edgewater Ranzal

Agenda

Monitoring your EPM environment with OEM

EPM log utilities ziplogs, log analysis, validation
ResetConfigTask - revert a product back to an unconfigured state
Maintenance & Housekeeping - common logs to
rotate/archive
When to apply PSUs
Virtualization when it works and when it doesnt

Copyright 2014 Doug Fiedler, Edgewater Ranzal

Monitoring your EPM environment with OEM

What is OEM? OEM stands for Oracle Enterprise Manager,

it is used to manage many Oracle products including
Oracle RDBMS, OBIEE & EPM. It can be used to monitor
performance, monitor system health, restart services and
many other useful features.
Out of the box it only plugs into the weblogic JVMs but it
is possible to add non JVM applications like Essbase.
Upgrade to OEM 12c and check out doc http://www.oracle.com/technetwork/middleware/bifoundation/epm11123-oem12c-monitor-1955582.pdf

Copyright 2014 Doug Fiedler, Edgewater Ranzal

 To get to the OEM login

screen you must first start the
weblogic admin server in
foreground
This can be added as a
windows service https://support.oracle.com/ep
mos/faces/DocumentDisplay?
id=1060058.1

Copyright 2014 Doug Fiedler, Edgewater Ranzal

 Here you have access to all

weblogic deployments on the
current wls domain
Green for jvms that are up, red for
down
Copyright 2014 Doug Fiedler, Edgewater Ranzal

 Shows system health

JVM Performance
Response & Load times
Copyright 2014 Doug Fiedler, Edgewater Ranzal

 Start/stop services
View logs
Setup security

Copyright 2014 Doug Fiedler, Edgewater Ranzal

EPM log utilities

Log Analysis Utility

Located in
\Oracle\Middleware\user_projects\epmsystem1\bin\log
analysis.bat
Collects data from all log EPM log files, allows
searching by key word and/or window of time

Copyright 2014 Doug Fiedler, Edgewater Ranzal

\Oracle\Middleware\user_projects\epmsystem1\diagnostics\
reports\LogAnalysis_Report_2013_10_24_17_58_15.html

Copyright 2014 Doug Fiedler, Edgewater Ranzal

Ziplogs

Script that collects and zips all log files on the server,
great for submitting log files to Oracle support

\Oracle\Middleware\user_projects\epmsystem1\bin\zi
plogs.bat

The zip file will be created here:

\Oracle\Middleware\user_projects\epmsystem1\diagn
ostics\ziplogs\EPM_logs_servername_11.1.2.3.0.zip\
zip
Copyright 2014 Doug Fiedler, Edgewater Ranzal

Validation Tool

\Oracle\Middleware\user_projects\epmsystem1\bin\validat
e.bat
Checks to make sure that the environment is working
properly and produces the following report

Copyright 2014 Doug Fiedler, Edgewater Ranzal

Registry-cleanup

\Oracle\Middleware\user_projects\epmsystem1\
bin\registry-cleanup.bat
If you see errors pertaining to the EPM registry,
after viewing the validation report, they can be
fixed by running the registry-cleanup tool.
It is also helpful to remove a server from your
environment in order to clean it out of the
registry.

Copyright 2014 Doug Fiedler, Edgewater Ranzal

Deployment Report

Provides information on what has been configured, # of

CPUs/Ram by server, OS, Database/JDBC etc.

Copyright 2014 Doug Fiedler, Edgewater Ranzal

Registry Report

Generates an html report of all registry keys

Copyright 2014 Doug Fiedler, Edgewater Ranzal

ResetConfigTask

What does it do?

It allows you to redo a config task
Why would you want to do this?
When a task fails or you need to change
something
First run a registry report
D:\Oracle\Middleware\user_projects\epmsystem1\
bin\epmsys_registry.bat or epmsys_registry.sh
Copyright 2014 Doug Fiedler, Edgewater Ranzal

Here we can see that Planning has been

configured and it was successful
If it had failed it would say failed instead of
configured

Copyright 2014 Doug Fiedler, Edgewater Ranzal

To reconfigure run the following command:

resetConfigTask.bat -product Planning -task
applicationServerDeployment

When you go back into the configtool you will now see
that the Planning deployment will have gone from green
to yellow (unconfigured)

You can do this with any epm configuration task

Copyright 2014 Doug Fiedler, Edgewater Ranzal

Maintenance & Housekeeping

Backups - Nightly
File system (d:\oracle , /app/oracle)
Database (SQL, Oracle)
LCM exports

Log file archiving

Essbase.log & app logs can be archived with the
AGTMAXLOGFILESIZE & APPMAXLOGFILESIZE commands within
the essbase.cfg
HFM & .odl (oracle diagnostic logging) log files should be archived
manually as needed when they reach a size that is too big to open
with windows tools
If you have a log essbase.log file that cant be opened in windows you
can view it in EAS and view by date/time as well
Copyright 2014 Doug Fiedler, Edgewater Ranzal

HFM
Purge task & data audit quarterly

Miscellaneous
Always monitor disk space, especially on the Essbase server. If the
disk becomes full Essbase will become corrupted.

Copyright 2014 Doug Fiedler, Edgewater Ranzal

When to apply Patches

PSUs tend to come out quarterly

PSU = Patch Set Update
PSE = Patch Set Exception
PSEs come out all the time, they are usually
created because of defects specific to a certain
customers environment
I recommend applying PSUs quarterly

Copyright 2014 Doug Fiedler, Edgewater Ranzal

I only recommend applying PSEs if the issue

listed in the defects fixed readme is an issue
that you are experiencing
To find out what the latest EPM patches are go
to the following url:
https://supporthtml.oracle.com/epmos/faces/ui/k
m/DocumentDisplay.jspx?id=1400559.1

Copyright 2014 Doug Fiedler, Edgewater Ranzal

Virtualization

Oracles official policy on Virtualization

The following KB article is Oracles official stance on virtualizing the
Oracle EPM Product stack.
https://support.oracle.com/CSP/main/article?cmd=show&type=NO
T&doctype=HOWTO&id=588303.1
Additional information
https://support.oracle.com/epmos/faces/SearchDocDisplay?_adf.ct
rl-state=1854yc0wrq_9&_afrLoop=14354592882224
In a Nut Shell
Oracle will support VMWare and Microsofts Hyper-V technologies,
but they do not certify them. What this means is that if Oracle
cannot fix an issue they can refer you to the hosting provider for
further troubleshooting, or force you to physicalize the servers and
reproduce the issue. However, Oracle does certify their products
running on Oracles Virtualization Manager (OVM).
Copyright 2014 Doug Fiedler, Edgewater Ranzal

Virtualization

Advantages
Reduced hardware costs (ability to consolidate servers)
Much quicker to stand up a VM vs a physical server
If the hypervisor fails, technologies like vmotion allow you to move
your VM/s to another host, this is also very helpful in DR situations
CPU, disk & memory can be added and removed as needed

Disadvantages
You can over assign resources (CPU, memory) and this can cause
performance issues when multiple VM guests need the resources at
the same time
Certain products run better than others when virtualized: weblogic
JVMs run well while Essbase & HFM may have a performance loss
during intense calculations
Copyright 2014 Doug Fiedler, Edgewater Ranzal

Virtualization Whats important to know

Allocate resources statically : RAM and vCPUs

Dont let the VM Host Servers manage resources on
the fly, use reservations
Dont over subscribe your Virtual host servers
Utilize dedicated VM Host servers for EPM when
possible
Test, Test, Test and then Test again. If you have a test
tool like Load Runner ever better
Have performance issues? We have VMware Certified
Engineers who can find & fix your VM bottlenecks
Copyright 2014 Doug Fiedler, Edgewater Ranzal

Virtual

Foundation and Web

Servers
DRM Server
HSF Server
All servers can be virtual,
really depends on the
application
HFM Application Server
Essbase Server
Integration Server

Physical

Depending on the
application, the following
servers should perform
better on Physical
hardware
HFM application Server
(Consolidations)
Essbase Server
(Calculations)
Integration Server (Data
Mapping and Loads)
Copyright 2014 Doug Fiedler, Edgewater Ranzal

Questions?

Copyright 2014 Doug Fiedler, Edgewater Ranzal