PL/SQL Enhancements in 11g

Enhanced SQL –
PIVOT and UNPIVOT
SELECT * FROM sales
PIVOT
PROD QUARTER AMOUNT (sum(amount)
Shoes Q1 2000 FOR quarter in
Shoes Q2 1000 ‘Q1’,’Q2’,’Q3’,’Q4’);
Jeans Q1 1000
Jeans Q3 500
PROD Q1 Q2 Q3 Q4
Jeans Q3 100
Shoes 2000 1000 Null Null
Jeans Q4 1000
Jeans 1000 Null 600 1000

Enhanced performance of ETL in your warehouse

PL/SQL Enhancements in 11g
Fine Grained Dependency Tracking
 The challenge

create table t(a number)

/
create view v as select a from t
/
alter table t add(Unheard_Of number)
/
select status from User_Objects
where Object_Name = 'V'
/

• View v ends up invalid in 10.2 because we know only

that its dependency parent has changed – at the
granularity of the whole object
 Fine Grained Dependency Tracking

create table t(a number)

/
create view v as select a from t
/
alter table t add(Unheard_Of number)
/
select status from User_Objects
where Object_Name = 'V'
/

• In 11.1 we track dependencies at the level of element

within unit
• so we know that these changes have no consequence
PL/SQL Enhancements in 11g
Real Native Compilation
 The challenge

• Through 10.2, PL/SQL compiled to a native DLL is

significantly faster than PL/SQL compiled for
interpretation by the PVM

• Oracle translates PL/SQL source to C code and

leaves the last step to a 3rd party
C compiler

• BUT… some customers’ religion forbids a C compiler

on a production box!

• AND… other customers’ religion forbids paying to

license a C compiler when they’ve already paid to
license Oracle Database!
 Real native compilation

• In 11.1, Oracle translates PL/SQL source directly to

the DLL for the current hardware

• Moreover, Oracle does the linking and loading so that

the filesystem directories are no longer needed

• So PL/SQL native compilation will work out of the box

– and without compromising religion

• Only one parameter remains: the on/off switch,

PLSQL_Code_Type
 Real native compilation

• As a bonus, it’s faster!

• Compilation with real native is twice as fast as with C native
• The Whetstone benchmark is 2.5x faster with real native
than with C native at run-time
• Contrived tests have shown 20x run-time speedup
PL/SQL Enhancements in 11g
Sequence in a PL/SQL expression
The challenge

create or replace trigger Trg

before insert on My_Table for each row
declare
s number;
begin
-- Annoying locution
select My_Seq.Nextval into s from Dual;
:New.PK := s;
end;
/
Sequence in a PL/SQL expression

create or replace trigger Trg

before insert on My_Table for each row

begin

:New.ID := My_Seq.Nextval;
end;
/
PL/SQL Enhancements in 11g
Tools support
 PL/Scope
Challenge
y Inherit code, find a problem in one area that links elsewhere, got
lost following reference after reference
y Impact Analysis: how do you determine who references a
particular variable?

Solution
y PL/Scope

Usage
alter session set PLScope_Settings = 'identifiers:all'

create or replace...

Select …. from … User_Identifiers where Object_Name = ...

UI expected in SQL Developer 2.0

 PL/SQL Hierarchical Profiler
Challenge
y Where is time really being spent in PL/SQL code?

Solution
y PL/SQL Hierarchical Profiler
y reports dynamic execution profile organised by subprogram
y accounts for SQL and PL/SQL separately
y generates hypertexted HTML reports

Usage
> EXECUTE DBMS_HPROF.START_PROFILING('PLSHPROF_DIR', 'test.trc');

% plshprof

UI expected in SQL Developer 2.0

XML DB Enhancements in 11g
Fast, better and easier

See:- New Features in Oracle XML DB, for Oracle Database 11g Release 1,
An Oracle White Paper, June 2007
 In Place Schema Evolution

• 10.2 copyEvolve() unloads and reloads the data when the XML
Schema changes
• very flexible but expensive: time to evolve schema is proportionate
to the amount of data being managed
• 11.1 In Place Schema Evolution allows simple changes to
registered XML schemas with zero down-time
• no data copy required
• schema change takes a few seconds regardless of amount of data.
• changes to the XML must not invalidate existing documents
• add optional elements and attributes
• adding new values to enumerations
• increase in length
 XMLType Partitioning

Challenge
<PurchaseOrder> • repeating elements typify XML documents and are
<Date>…</Date>
managed using nested tables
<Ref>…</Ref>
<LineItems> Solution
<LineItem>… • leverage 11.1 REF partitioning
<LineItem>…
• nested Tables are partitioned using the same key
<LineItem>
as the top level XMLType table
<LineItems>
• supports “partition maintenance” operations
• supports “partition pruning”
XML Misc Improvements

• Significant performance improvements

• Large node handling eliminates current 64K limit on
size of a text-node.
• Stream based replication support for LOB-based
XML storage
• Events: trigger support for the XML DB repository.
• e.g. inserting a purchase order into “approved” folder triggers
ordering process
 Binary XML

Challenge
• 10.2 schema-less XML not well optimized
Solution
• 11.1 binary XML optimized for schema-less and
flexible schema-based XML
• Post Parse compact representation of XML
• Flexible XML Schema Support
• High performance fragment access and extraction
with new XML Index
Binary XML
• Single format for “on-disk”, “in-memory” and “on-the-wire”
• Reduced CPU and Memory overhead
• single format avoids parse and serialize issue between tiers
• Reduced Network Overhead
• post-parse format used when XML moves between tiers
Database App Web
Server Cache Client

Binary XML Binary XML Binary XML

Oracle Binary XML

 XML Index
• New universal index for Binary and LOB based
XMLType
• All possible paths are indexed by default
• XML index optimization
• asynchronous operation possible
• path sub-setting controls what is indexed
• partial re-indexing (binary XML and securefiles only)
• Repository integration with XML Index on XML DB
repository
Java Enhancements in 11g
Doing it inside like you do outside

See:- Java Developer Center on OTN

http://www.oracle.com/technology/tech/java/index.html
 JDBC Performance
Saving 1 Million Roundtrips per Day

Prefetch in First Roundtrip (JDBC-Thin)

• saves 1 database roundtrip per SELECT Statement
• benefit for ad-hoc queries

New Native AQ protocol (JDBC-Thin)

• AQ/OJMS operations 40%-300% Faster
• reduced server-side CPU Utilization
 Java Standards
JDBC 4.0
• SQLXML
• RowId
• NChar & Nclob
• Wrapper Interface
• SQLException Hierarchy
• conn.createBlob, createClob, etc

Disconnect Java VM upgrade from database

• Java 5 (i.e. JDK 1.5) in base 11.1 release
• Java 6 (i.e. JDK 1.6) and newer JDKs, in patch sets

From Black Box to White Box Monitoring

• JMX Instrumentation
• Support standard JMX-enabled tools(i.e., JConsole)
 Java in the Database
Mass Adoption
Challenge
• Performance
Solution
• New JIT compiler compiles Java in the database natively and transparently (on
the fly) without the need of a C compiler
• Enabled by default, Out-of-the-Box
• Up to an Order of magnitude (~x 10) speed-up for pure Java, 10g versus 11g,
out-of-the-box

Challenge
• Ease of Use
Solution
• JDK-Like interface, with code on filesystem, using familiar notation,
e.g. –{classpath}, -D system properties
• Eases the migration for developers from JDK VM world
Enhancements for PHP developers
Accessing the database will never be the same

See:- PHP Developer Center on OTN

http://www.oracle.com/technology/tech/php/index.html
 PHP users have asked for
Database Connection Pooling
• Web apps issue hundreds connects per second
• Connection establishment is expensive
• PHP does not allow real multi-threading
• persistent vs non persistent connections
• Existing client-side connection pooling not possible
• Options:
• apache process can repeatedly connect/disconnect
• apache process can hold onto private connection
• Results in inefficient use of the database resources
Database Resident Connection Pool

No Connection Pooling 11g Database Resident Connection Pooling

 Native Database Web Service

DBA -
u to ate WSDL
A er
en
G

PL/SQL SOAP

DB WS
? SOA
SQL

XQuery

Easy: a few steps, no other software installation and no coding

Secure: HTTP authentication and easily integrate with WS Security Products
High performance : C-based engine, natively build in to the Oracle DB
 Oracle Scheduler
enhancements in Oracle Database 11g
Schedule
Jobs
Execute local OS
• Local OS jobs 11g job A as OS user X &
• run as any OS user
job B as OS user Y
• Remote OS jobs
• no Oracle database required
on node
• run as any OS user

Agent
Agent

Execute remote Execute remote

OS job as OS user X OS Job as OS user Z
Business Intelligence

• OLAP
• Data Mining

See:-
Oracle Database 11g for Data Warehousing
and Business Intelligence
An Oracle White Paper
July 2007
 What is Oracle OLAP Option?

• A multidimensional calculation and aggregation ‘engine’

embedded in the database
• multidimensional data types (Cubes and Dimensions)
• OLAP API for cube definition and multidimensional query

and coming in Oracle Database 11g ….

• transparent SQL interface to OLAP Cubes and dimensions
• cube-organized materialized views
• OLAP metadata in the Oracle Data Dictionary
 Oracle OLAP 11g
New Features in Oracle Database 11g

• Data accessibility: Enhanced SQL access to OLAP

cubes
• SQL Optimizer awareness of OLAP cubes
• Cube Organized Materialized Views
• System-maintained generation of cube and dimension
views
• Feature accessibility: Shift the focus from the skilled
OLAP specialist to the database generalist
• Cube and dimensions in the Oracle Data Dictionary
• Cost-based aggregation
• Dimensionally aware SQL-like calculation syntax
 Cube-Organized Materialized Views

• Materialized view with data stored in an OLAP cube

• Automatic query write of summary queries into the cube
for improving performance of summary queries
• Materialized view managed refresh of cubes and
dimensions
• Similar to materialized view on pre-built table; data is
stored in cube and MV is a meta data object
 Cube-Organized Materialized Views

• Detail data is stored in

relational tables
SQL Query
• Application queries
relational tables with SQL
• Summary data is
Query
Tables
Rewrite Cube Organized
managed by OLAP cube
MV representing
Cube MV
summary data
• Access to summary data
occurs via automatic
Detail data managed in
relational tables query rewrite to the cube
• Application is unchanged,
but updates and queries
Database-
Database-managed
refresh are faster
Summary data
managed in cubes
 Query Rewrite to Cube
select a13.FAMILY_ID FAMILY_ID,
a13.FAMILY_DSC FAMILY_DSC,
a12.CALENDAR_YEAR_ID CALENDAR_YEAR_ID,
a12.CALENDAR_YEAR_DSC CALENDAR_YEAR_DSC,
sum(a11.SALES) WJXBFS1 Í SUM(SALES)
from UNITS_FACT a11
join TIME_DIM a12
on (a11.MONTH_ID = a12.MONTH_ID) Select from relational tables
join PRODUCT_DIM a13
on (a11.ITEM_ID = a13.ITEM_ID)
group by a13.FAMILY_ID,
a13.FAMILY_DSC, GROUP BY
a12.CALENDAR_YEAR_ID,
a12.CALENDAR_YEAR_DSC;
 Query Rewrite to Cube

OPERATION
---------------------------------------
SELECT STATEMENT
HASH
HASH JOIN
HASH JOIN
CUBE SCAN CB$UNITS_CUBE Í Cube Access
VIEW
HASH
TABLE ACCESS PRODUCT_DIM
VIEW
HASH
TABLE ACCESS TIME_DIM
 What is Oracle Data Mining?
• In-Database Mining Engine
• wide range of algorithms
• anomaly detection
• attribute importance
• association rules
• clustering
• classification & regression
• nonnegative matrix factorization
• structured & unstructured data (text mining)
• BLAST (life sciences similarity search algorithm)
• Oracle Data Miner (GUI)
• simplified, guided data mining
• Spreadsheet Add-In for Predictive Analytics
• “1-click data mining” from a spreadsheet
• PL/SQL API & Java (JDM) API
• develop advanced analytical applications
 Oracle Data Mining 11g
New Features in Oracle Database 11g

• Simplified development and deployment of models

• supermodels: data preparation combined with mining model
• tighter database integration – DMSYS schema eliminated; improved
sharing of objects, security, manageability
• New algorithms: “General Linear Models”
• multivariate linear regression; logistic regression
• basic tool for microeconomics analysis and business decision-making. Indicate
the impact of different attributes on a key performance indicator.
“What is the impact on SUVs sales if the price of gas goes up by a $1.00?”
Why In-Database Analytics ?

• Fewer moving parts : data remains in the database

at all times … with appropriate access security
control mechanisms
• Straightforward inclusion within interesting
and arbitrarily complex queries
• Real-world scalability and performance
• Fast scoring:
• 2.5 million records scored in 6 seconds on a single CPU
system
• Real-time scoring:
• 100 models on a single CPU: 0.085 seconds
Integration with Oracle BI EE ODM provides likelihood
of fraud and other
important questions.
Security

• User Management
• Access Control
• Data Protection
• Monitoring

See:- Oracle Database 11g Release 1 Transparent

Solutions for Security and Compliance
An Oracle White Paper
June 2007
Key Drivers for Data Security

Regulatory Compliance
• Sarbanes-Oxley (SOX),
• Foreign Exchange Instruments and
Exchange Law (J-SOX)
• EU Privacy Directives, CA SB 1386….
• Payment Card Industry (PCI)
• Adequate IT controls, COSO, COBIT
• Separation of duty, Proof of compliance,
Risk Assessment and Monitoring

Insider Threats
• Large percentage of threats go undetected
• Outsourcing and off-shoring trend
• Customers want to monitor insider/DBA
 Oracle Database Security Products

User Management Access Control

• Oracle Identity Management • Oracle Database Vault
• Enterprise User Security • Oracle Label Security

Core
Platform
Security

Monitoring Data Protection

• Oracle Audit Vault • Oracle Advanced Security
• EM Configuration Pack • Oracle Secure Backup
 Oracle Database 11g
Enterprise User Security

• User Management for Compliance

HR Financial Customer
• Centralized User Management Database Database Database
• Consolidate database accounts with
shared database schemas EUS EUS
• Centrally managed DBAs
• Validated with Oracle Virtual Directory
• Enterprise Strong Authentication
• Kerberos (MSFT, MIT)
• PKI (x.509v3)
• Password
• SYSDBA Strong Authentication
• Database Enterprise Edition Feature
• Requires Oracle Identity Management
• Available since Oracle 8.1.6 Oracle Identity
Management
 Oracle Database Security Products

User Management Access Control

• Oracle Identity Management • Oracle Database Vault
• Enterprise User Security • Oracle Label Security

Core
Platform
Security

Monitoring Data Protection

• Oracle Audit Vault • Oracle Advanced Security
• EM Configuration Pack • Oracle Secure Backup
 Oracle Label Security
Manageability

• Policy based model

• Multiple policies
supported
• ACME, HR, Legal
• Policies are umbrellas
applying to one or
more tables, schemas,
users
• Web based management

• Integrated with Oracle

Identity Management
 Oracle Database Security Products

User Management Access Control

• Oracle Identity Management • Oracle Database Vault
• Enterprise User Security • Oracle Label Security

Core
Platform
Security

Monitoring Data Protection

• Oracle Audit Vault • Oracle Advanced Security
• EM Configuration Pack • Oracle Secure Backup
 Oracle Advanced Security
Tablespace Encryption

• Define a new tablespace as ‘encrypted’

• cannot convert existing, un-encrypted tablespaces
• however, content can be moved into encrypted tablespaces
• Always salted for higher security
• Overcomes limitation of column-based TDE:
• supports indexes other than b-tree
• supports foreign keys
• No additional management overhead
• integrated into TDE key management, same wallet used as
for column based Transparent Data Encryption
• No storage overhead (!)
 Oracle Advanced Security
Master key stored in HSM device

• Store the Master key in an external hardware device

• Master key never leaves the device
• Standard PKCS #11 API allows customers to choose from
a wide range of HSM vendors
• Encryption and decryption done on the database server
• Simplifies key management in distributed environments
(data guard, RAC)
 Oracle Advanced Security
SECUREFILE LOB Encryption

• SECUREFILE LOB encryption

• All SECUREFILE LOBs in an encrypted column are
encrypted
• In-line (in table) and out-of-line (in tablespace) are both
encrypted
• BFILEs are not encrypted
• Always salted for higher security
 Oracle Database Security Products

User Management Access Control

• Oracle Identity Management • Oracle Database Vault
• Enterprise User Security • Oracle Label Security

Core
Platform
Security

Monitoring Data Protection

• Oracle Audit Vault • Oracle Advanced Security
• EM Configuration Pack • Oracle Secure Backup
 Oracle Database 11g
Security Manageability

• Integrated with EM
• Label Security
• Virtual Private Database
• Application Context
• Enterprise Security
Manager
• Transparent Data
Encryption
11g Database Manageability

• Memory Management
• AWR Baselines
• Automatic SQL Tuning
• Advisors
• Upgrade Considerations

See:- Oracle Database 11g:

Manageability Overview
An Oracle White Paper
August 2007
11g Database Manageability
Automatic Memory Management
Memory Management in 10g/11g
11g Automatic Memory Management
• MEMORY_TARGET
• MEMORY_MAX_TARGET
• Improves memory utilization
• Eases memory management

10g Automatic Shared Memory 10g Automatic PGA Memory

Management (ASMM) Management
• SGA_TARGET • PGA_AGGREGATE_TARGET

• Key shared memory components • Various PGA sort areas tuned

tuned automatically automatically
• Single parameter controls all • Single parameter controls all PGA
shared memory memory
 Automatic Memory Management in 11g

• Unifies system (SGA) and process

O/S Memory (PGA) memory management
O/S Memory • Dynamic parameters for all database
SGA memory:
MEMORY_TARGET
MEMORY_MAX_TARGET
SGA • Automatically adapts to workload
changes
• Maximizes memory utilization
• Available on:
PGA • Linux
PGA • Windows
• Solaris
• HPUX
• AIX
 Memory Target Benefits

• Balances dynamic memory needs of SGA and PGA

• Previously either or both could end up undersized
• PGA can grow but not to the detriment of SGA performance
• Best-effort memory transfers
• Tunable PGA can help relieve 4031 from SGA

• Built-in safety nets:

• Decides victim components
• Paced memory donation avoids thrashing
• Run-away workload cannot starve resources
• Component-specific settings still honored
11g Database Manageability
AWR Baseline and Adaptive Thresholds
 Performance Monitoring and Baselines

• Performance Page shows certain metrics against 99th

percentile over selected baseline

• Adaptive Thresholds over select system metrics

• Enabled by System Moving Window baseline

• AWR Compare Period Report

• Use Baseline as one of the periods
 Other Baselines Features

• Baseline Templates enable future Baseline creation

• Pre-specify known interesting times, e.g. Thanksgiving

• System Moving Window Baseline

• Always available
• Defined by AWR retention or user specified
• Adaptive thresholds computed over this baseline
Creating Single AWR Baseline
Quick Configure Adaptive Thresholds
Adaptive Thresholds

Adaptive threshold values

follow baseline values

AWR Baseline
11g Database Manageability
Automatic SQL Tuning
 Manual SQL Tuning Challenges

• Complex - requires expertise in several domains

• SQL optimization: adjust the execution plan
• Access design: provide fast data access
• SQL design: use appropriate SQL constructs

• Time consuming
• Each SQL statement is unique
• Potentially large number of statements to tune

• Never ending task

• SQL workload always evolving
• Plan regressions can happen
SQL Tuning in Oracle Database 10g

Evaluate
Implement Recommendations

Workload DBA DBA

Generate
Recommendations

one hour
DBA

ADDM Invoke
Advisor
SQL Tuning
AWR SQL Tuning Candidates Advisor

Some meaningful automation,

but the DBA is still required
 SQL Tuning Automation in 11g

• Fully automatic SQL Tuning task

• Runs automatically under Autotask framework:
• Maintenance Window, CPU resource controlled, on/off switch
• Identifies, ranks and tunes candidate SQL
• Leverages SQL Tuning Advisor

• Candidate SQL automatically chosen

• Excluded: parallel queries, DML/DDL, recursive, ad-hoc (infrequent)

• Tests and (optionally) implements SQL profiles

• Only implements significantly improved plans (3x)
• Time budgeted to avoid run-away plans
 Automatic SQL Tuning in Oracle
Database 11g
Implement Test SQL Profiles
SQL Profiles

Workload
Generate
Recommendations

Choose
Candidate
SQL SQL Tuning
one Candidates
week
It’s Automatic!

View Reports /
AWR Control
DBA Process
Result Summary
Result Details
Result Recommendations
11g Database Manageability
SQL Access Advisor
 SQL Access Advisor: Partition Advice

SQL Access
Advisor Hash
partitions?
SQL cache
Interval
Complete partitions?
Hypothetical Workload

STS
Filter
Options

Indexes Materialized Materialized Partitioned

views views log objects
SQL Access Advisor Results
Access Advisor Recommendations
Editable implementation script
11g Database Manageability
Fault Diagnostic Automation
 Oracle Database 11g R1
Fault Diagnostic Automation

Goal: Reduce Time to Problem Resolution

Realistic Testing
and Automatic Proactive
Intelligent
Diagnostic Patching
Automatic Health Resolution
Workflow
Checks

Diagnostic Solution Delivery

Prevention Resolution
Automatic Diagnostic Workflow
Automatic
Critical Diagnostic
Error Repository

DBA

Alert DBA
Auto incident creation
1 2 Targeted health checks
First failure capture Assisted SR filling

No Known
DBA bug?

Yes
EM Support Workbench:
4 Package incident info EM Support Workbench:
Data Repair
Apply patch / Data Repair DBA
3
 Problems and Incidents

• Problems are fundamental code or configuration

issues that can cause execution failures
• They exist until they are corrected, e.g by patch
• They are assigned a signature-based key value

• An incident is a single occurrence of a problem

• They happen at point(s) in time and thus have timestamps
• They induce diagnostic actions like dumps and traces
• They are managed by ADR and packaged by IPS
 Automatic Diagnostic Repository
DIAGNOSTIC_DEST
Support Workbench
$ORACLE_BASE BACKGROUND_DUMP_DEST
CORE_DUMP_DEST
USER_DUMP_DEST
$ORACLE_HOME/log
ADR
Base

diag

rdbms

DB
Name

ADR SID metadata

Home

alert cdump incpkg incident hm trace (others)

incdir_1 … incdir_n
ADRCI
log.xml
V$DIAG_INFO
alert_SID.log
 Important points about ADR

• Location of traces and dumps has changed

• See V$DIAG_INFO for new locations
• Symbolic links can provide work around for script
compatibility

• Format of alert.log is changed

• New XML format more parse-able
• Old style still available for backward compatibility

• Space management
• Incident flood control minimizes number of traces
• ADR auto-purging reduces disk footprint
 Incident Packaging Service (IPS)

• IPS automates packaging of diagnostic data for given problem

• Creates zip files from incident package contents for sending to

Oracle Support

• Intelligently correlates and includes dumps and traces from

related incidents
• Correlation rules based on time proximity, client id, error code

• Recommends further diagnostic actions for DBA

• For example build SQL test case

• DBAs can explicitly add/edit or remove files before packaging

• Including non-ADR managed files
 EM Support Workbench Overview

• Primary customer-facing interface into IPS and ADR

• Task-oriented interface over problems and incidents

• View problem and incident details
• Create and manage IPS packages
• Run health checks or generate additional diagnostic data
• Run advisors to help resolve problems

• Linkages to Oracle Support

• Generate and send incident package files using OCM
• Create and track Metalink service requests from problem
• Close problem SR once resolved
EM Support Workbench
Package details and IPS in EM
 EM Support Workbench
Quick Package
• Simple wizard workflow from problem through
package and send to support
• Single-problem package
• Good for rapid first fault reporting

• Limitations
• Package zip must be 2GB or less
• Cannot scrub traces or customize package
Prevention: Health Monitoring

• Proactive Health Checks

• E.g. Dictionary Validation

• Reactive Checks
• Trigger on error via Intelligent dumping Rules
• E.g. Checks blocks around corrupted block

• All Checks can be Activated on Demand

• Incident Meter for EM
 Intelligent Repair Advisors

• Data Recovery Advisor

• Guided expert data recovery system using diagnostic data
and Health Check output

• SQL Test Case Builder

• Automatically retrieves exact environment information from
ADR to build SQL test cases and replicate SQL issues

• SQL Repair Advisor

• Analyzes failing SQL statements to isolate bug
• May recommend SQL Patch as work around
Data Recovery Advisor
 Data Recovery Advisor
• Oracle Database tool that automatically diagnoses data failures,
presents repair options, and executes repairs at the user's request
• Determines failures based on symptoms
• E.g. an “open failed” because datafiles f045.dbf and f003.dbf are missing
• Failure Information recorded in diagnostic repository (ADR)
• Flags problems before user discovers them, via automated health
monitoring
• Intelligently determines recovery strategies
• Aggregates failures for efficient recovery
• Presents only feasible recovery options
• Indicates any data loss for each option
• Can automatically perform selected recovery steps

Reduces downtime by eliminating confusion

Data Recovery Advisor
Enterprise Manager Support
SQL Test Case Builder
 SQL Test Case Builder

Business Requirement
• Bug resolution
• Test case required for fast bug resolution
• Not always easy to provide a test case
• What information should be provided?
• How much data is need?
• Getting the test case to Oracle can be tricky
Solution
• Oracle automatically creates a test case
• Collects necessary information relating to a SQL incident
• Collected data is packaged to be sent to Oracle
• Collected data allows a developer to reproduce the problem
SQL Repair Advisor
 SQL Repair Advisor

Business Requirement
y The most common types of SQL problems - exception, performance
regression etc., are hard to diagnose
y A lot of time is spent trying to reproduce the problem
y If a workaround is found it has to be applied to entire system

Solution
• Advisor
• Investigates the incident locally
• Automatically determines the root cause
• Provides a workaround (SQL Patch) for just the effected SQL
• If not, sends necessary diagnostic information to Oracle
 SQL Repair Advisor Flow
SQL Generate
statement Execute Statement incident in ADR
crashes automatically

Trace files

DBA run DBA gets

SQL Repair Advisor

SQL Repair Advisor alerted

investigates

DBA
DBA accept Statement
executes
SQL patch successfully
again
Execute

SQL patch
generated SQL statement
patched
 DB Management Pack Enhancements

• Diagnostic Pack
• ADDM for RAC
• AWR Baselines and Adaptive Thresholds
• Transportable AWR

• Tuning Pack
• Automatic SQL Tuning Advisor
• Partition Advisor
• Real-time SQL Monitoring
• SQL Plan Management and Plan Evolution
Upgrade Considerations
 New Background Processes in 11g
Process
Description Found
Name
In an RAC environment, this per-instance process (Atomic Controlfile to Memory Service) is an agent
which contributes to ensuring a distributed SGA memory update is either globally committed on success
ACMS or globally aborted in the event of a failure. RAC

Responsible for hang detection and deadlock resolution.

DIA0 ALL

Performs diagnostic dumps and executes global oradebug commands.

DIAG ALL

The Resource Manager process is responsible for setting Resource Plans and other Resource Manager
DBRM ALL
related tasks.

ALL with any

The Event Monitor Coordinator coordinates the event management and notification activity in the registered event
EMNC / e0xx database which includes Streams Event Notifications, Continuous Query Notifications and Fast activity
Application Notifications.
(optional non-fatal)

The process archives historical rows for tracked tables into flashback data archives and manages the
FBDA ALL.
flashback archives.

GMON Maintains disk membership in ASM disk groups ASM

Provides transparent support for XA global transactions in a RAC environment. The database
GTX0-j RAC only
autotunes the number of these processes based on the workload of XA global transactions.
 New Background Processes in 11g
Process
Description Found
Name
ASM, spawned on
KATE Performs proxy I/O to an ASM metafile when a disk becomes offline.
demand.

ASM, spawned on
MARK Marks ASM Allocation Units as stale following a missed write to an offline disk.
demand.

Spawns Oracle processes.

PSP0 ALL

The RAC Management Processes perform manageability tasks for RAC, e.g. creation of RAC related
RMSn RAC only
resources when new instances are added to the clusters

In a RACenvironment, this process manages background slave process creation and communication on
RMSN remote instances. These background slave processes perform tasks on behalf of a coordinating process RAC only
running in another instance.

The space management coordinator process coordinates the execution of various space management
ALL (optional non-
SMCO / Wnnn related tasks, such as proactive space allocation and space reclamation. It dynamically spawns slave
fatal)
processes (Wnnn) to implement the task.

?
The Virtual Keeper of TiMe is responsible for providing a Wall-Clock time (updated every second) and
VKTM ALL
Reference-Time Counter (updated every 20ms and available only when running at elevated priority).
Oracle Database 11g Release 1
Upgrade Paths
• Direct upgrade to 11g is supported from 9.2.0.4 or higher,
10.1.0.2 or higher, and 10.2.0.1 or higher.
• If you are not at one of these versions you need to perform a
“double-hop” upgrade
• For example:
• 7.3.4 -> 9.2.0.8 -> 11.1
• 8.1.7.4->9.2.0.8->11.1
Choose an Upgrade Method

• Database Upgrade Assistant (DBUA)

• Automated GUI tool that interactively steps the user through
the upgrade process and configures the database to run with
Oracle Database 11g Release 1
• Manual Upgrade
• Use SQL*Plus to perform any necessary actions to prepare
for the upgrade, run the upgrade scripts and analyze the
upgrade results
• Export-Import
• Use Data Pump or original Export/Import
• CREATE TABLE AS SQL statement
Oracle Database 11g Installation
Changes
• Addition of new products to the install
• SQL Developer
• Movement of APEX from companion CD to main CD
• Warehouse Builder (server-side pieces)
• Oracle Configuration Management (OCM)
• New Transparent Gateways
• Removal of certain products and features from the installation:
• OEM Java Console
• Oracle Data Mining Scoring Engine
• Oracle Workflow
• iSQL*Plus
 Fine Grained Access Control
for Utl_TCP and its cousins
Challenge
• Oracle Database provides packaged APIs for PL/SQL
subprograms to access machines (specified by host and port)
using bare TCP/IP and other protocols built on it (SMTP and
HTTP)
• Utl_TCP, Utl_SMTP, Utl_HTTP…
• If you have Execute on the package, you can access ANY
host-port

Solution
• an Access Control List (ACL) specifies a set of users and roles
• you assign an ACL to a host and port range
• you may need to explicitly grant this access in 11.1
 Case Sensitive Password
• By default:
• Default password profile is enabled
• Account is locked after 10 failed login attempts
• In upgrade:
• Passwords are case insensitive until changed
• Passwords become case sensitive by ALTER USER
• On creation:
• Passwords are case sensitive
• Review:
• Scripts
• Database links with stored passwords
• Consider backward compatibility parameter,
SEC_CASE_SENSITIVE_LOGON
 Log files changes

Automatic Diagnostic Repository

• $ORACLE_BASE/diag

alert.log
• xml format
• $ORACLE_BASE/diag/rdbms/orcl/orcl/alert/log.xml
• adrci> show alert –tail
• $ORACLE_BASE/diag/rdbms/<dbname>/trace/alert_SID.log
 Moving On

RBO
• Still there, same behavior as 10.2, but even move
tools to ease the move
• SQL performance analyzer
• SQL plan management
 Moving On
These deprecated features / privileges are still
available for backward compatibility but the
recommendation is to move on

• Stored Outline SQL Plan Management

• Ultrasearch Secure Enterprise Search
• CTXPath Index XML Index
• SYSDBA for ASM administration
SYSASM

No longer supported
• Workflow BPEL
Real Application Testing

• Real Application Testing

• Enterprise Manager
Management Packs
Enhancements

See:- Oracle Database 11g: Real Application

Testing & Manageability Overview
An Oracle White Paper
June 2007
 What is Real Application Testing?
• New database option available with EE only
• Includes two new features
• Database Replay (DB Replay)
• SQL Performance Analyzer (SPA)
• SQL Tuning Set (STS) functionality is shared between
Real Application Testing option and EM Tuning Pack
• Advanced functionality requires additional packs
• Diagnostic Pack for DB Replay
• Tuning Pack for SPA
• License
• Requires licenses for both capture and replay system CPUs
 Real Application Testing

• Value
• Rapid technology adoption Deploy
• Higher testing quality
• Business Benefit Test
Change
• Lower cost
• Lower risk
Remediate

Solution for the Agile Business

Real Application Testing
Database Replay
The Need for Database Replay

• Businesses want to adopt new technology that adds value

• Extensive testing and validation is expensive in time and cost
• Despite expensive testing success rate low
• Many issues go undetected
• System availability and performance negatively impacted
• Cause of low success rate
• Existing tools provide inadequate testing
• Simulate synthetic workload instead of replaying actual
production workload
• Provide partial workflow coverage

Database Replay makes real-world testing possible

 Database Replay
• Replay actual production database workload in test
environment
• Identify, analyze and fix potential instabilities before making
changes to production
• Capture Workload in Production
• Capture full production workload with real load, timing &
concurrency characteristics
• Move the captured workload to test system
• Replay Workload in Test
• Make the desired changes in test system
• Replay workload with full production characteristics
• Honor commit ordering
• Analyze & Report
• Errors
• Data divergence Analysis & Reporting
• Performance divergence
 Database Replay: Supported Changes

Client Client Client

…
Changes
Unsupported
Middle Tier

Changes Supported
•Database Upgrades, Patches
•Schema, Parameters Recording of
External Client
•RAC nodes, Interconnect Requests
•OS Platforms, OS Upgrades Storage
•CPU, Memory
•Storage
•Etc.
 Comparison of LoadRunner & DB Replay
Testing e-Business Suite

80
80
Time Taken (Days)

60

40 24
20 20
20 5 LoadRunner
4 0 0 2 5 DB Replay
0
Install & Setup Understand Identify Key Generate Run Test
Application Transactions Workload
Usage

Total Testing Time

DB Replay: 2 weeks
LoadRunner: 30 weeks
Why DB Replay?

From: To:
Artificial workloads Production workloads

Partial workflows Complete workflows

Months of development Days of development

Manual intensive Automated

150 High risk Low risk 10

Days Days
 Database Replay Workflow
Production (10.2.0.4) Test (11.1)
Clients
… Replay Driver

Mid-Tier

Storage
Storage

Analysis &
Capture Process Replay
Reporting
 Step 1: Workload Capture
Production System
• All external client requests
captured in binary files
Client Client Client
• System background, internal …
activity excluded
File System
• Minimal performance
overhead for capture Middle Tier

• For RAC, shared and local File 1

file system supported
File 2
• Specify interesting time
period for capture, e.g., peak …
workload, month-end
processing, etc. File n

Storage
• Can capture on 10.2.0.4 and
replay on 11g
 Capture Options
• Workload can be filtered to customize what is captured
• Filter Types
• Inclusion Filters: Specifies which sessions should be captured
• Exclusion Filters: Specifies which sessions should NOT be
captured
• Filter Attributes: Workload capture can be filtered using any of the
following session attributes
• User
• Program
• Module
• Action
• Service
• Session ID
• Workload capture can be run on-demand or scheduled to run at
later time
 Step 2: Process Workload Files
• Setup test system
Test System
• Test DB is at same point in time
as before production capture
• Use RMAN to physically restore
production db from backup File 1
File 1
• Use Snapshot standby
File 2
• Use imp/exp, Data Pump, etc.
File 2
…
• Processing transforms captured
data into replayable format … File n

• Once processed, workload can be File n

replayed many times
• For RAC copy all capture files to Metadata
single location for processing Capture Files Replay Files
 Step 3: Replay Workload
Test System
• Replays workload preserving
timing, concurrency and
dependencies of the capture
system Replay Driver

• Replay Driver is a special

client program that
consumes processed File 1
workload and sends requests
to the replay system File 2

…
• Replay Driver consists of one
or more clients. For File n
workloads with high
concurrency, it may be Metadata
necessary to start multiple Replay Files
clients to drive workload
 Replay Options
• Synchronized Replay (Default)
• Workload is replayed in full synchronized mode
• Exact same concurrency and timing as production workload
• Transaction commit order is honored
• Ensures minimal data divergence
• Unsynchronized Replay
• Workload can be replayed in unsynchronized mode
• Useful for load/stress testing
• High data divergence
• Three (3) parameters provided to control degree of synchronization
• Think time synchronization
• Connect (logon) time synchronization
• Commit order synchronization
 Replay Options
• Number of Replay Clients
• Configurable by user
• Client Calibration Advisor recommends number of replay
clients needed for specific workload
• Replay clients are multithreaded clients that can drive multiple
workload sessions each
 Analysis & Reporting

• Comprehensive reports are provided for analysis purposes

• There (3) types of divergences are reported
• Data Divergence: Number of rows returned by each call are
compared and divergences reported
• Error Divergence: For each call error divergence is reported
• New: Error encountered during replay not seen during capture
• Not Found: Error encountered during capture not seen during
replay
• Mutated: Different error produced in replay than during capture
• Performance Divergence
• Capture and Replay Report: Provides high-level performance
information
• ADDM Report: Provides in-depth performance analysis
• AWR, ASH Report: Facilitates comparative or skew analysis
Database Replay Summary Report
Performance Page – Database Replay
Top Activity Page: Database Replay
Performance Page – Database Replay
Performance Page – Database Replay
 Current Restrictions

• Database Replay does not support the following

features in the current release
• Direct path load, import/export
• OCI based object navigation (ADTs) and REF binds
• Streams, non-PL/SQL based AQ
• Distributed txns, remote describe/commit operations
• Flashback
• Shared Server
 Best Practices
• Capture
• Provide adequate disk space for captured workload (binary files)
• Database restart (Optional): Recommended to minimize divergence
• For RAC, use shared file system
• Test System Setup
• Ensure data in test is identical to production as of capture start time to
minimize data divergence during replay
• Use RMAN backup/restore or Snapshot Standby to setup test system
• For performance analysis test system capacity should be similar to
production
• Reset system clock to same time as production if application logic involves
SYSDATE usage
• Process Workload
• Processing workload has performance overhead and can possibly take a
long time
• Process workload on test system instead of production
• Replay
• Use Client Calibration Advisor to identify number of replay clients needed to
replay workload properly
Real Application Testing
SQL Performance Analyzer (SPA)
The Need for SQL Performance Analyzer (SPA)

• Businesses want systems that are performant and

meet SLA’s
• SQL performance regressions are #1 cause of poor
system performance
• Solution for proactively detecting all SQL regressions
resulting from changes not available
• DBA’s use ineffective and time-consuming manual
process to identify problems

SPA automates identification of all SQL performance

regressions resulting from changes
Why SQL Performance Analyzer?

From: To:
Manual workload capture Automated workload capture

Synthetic workload Production workload

Months of manual analysis Automated analysis in minutes

Partial workload Complete workload

High risk Low risk

 SQL Performance Analyzer
• Test impact of change on SQL query performance
• Capture SQL workload in production including statistics & bind
variables
• Re-execute SQL queries in test environment
• Analyze performance changes – improvements and regressions

Client Client Client

… Production Test

Middle Tier Re-execute SQL Queries

Capture SQL
… … Use SQL Tuning
Oracle DB Advisor to tune
regression

Storage
 SPA Benefits
• Enables identification of SQL performance regressions before
end-users can be impacted

• SPA can help with any change that impacts SQL execution plan
• DB upgrades
• Optimizer statistics refresh
• New indexes, Materialized Views, Partitions, etc.

• Automates SQL performance tracking of hundreds of thousands of

SQL statements – impossible to do manually

• Captures SQL workload with low overhead

• Integrated with SQL Tuning Advisor and SQL Plan Baselines for
regression remediation
 SQL Performance Analyzer Workflow
Production (10.2)
Clients
…
Test (11.1)

Mid-Tier

Storage

Storage

Capture Transport Execute SQL Execute SQL Compare

SQL SQL Pre-change Post-change Perf
 Step 1: Capture SQL Workload

• SQL Tuning Set (STS) used to store

SQL workload
• STS includes:
• SQL Text
• Bind variables
• Execution plans
Cursor Cache
• Execution statistics

Incremental Capture • Incremental capture used to populate

STS from cursor cache over a time
period
• SQL tuning set’s filtering and ranking
SQL Tuning Set capabilities filters out undesirable SQL
• SQL workload captured in 10.2.0.1 and
Production Database higher can be used for SPA tasks in 11g
 Step 2: Move SQL Workload to Test System

Cursor Cache

SQL Tuning Set

Export/Import SQL Tuning Set

Production Database Test Database

• Copy SQL tuning set to staging table (“pack”)

• Transport staging table to test system (datapump, db link, etc.)
• Copy SQL tuning set from staging table (“unpack”)
 Step 3: Execute SQL Before Making Change

SQL Tuning Set • Establishes SQL workload performance

baseline
Fetch Next SQL

• SQL execution plan and statistics

captured

Test Execute • SQL executed serially (no concurrency)

Execution Plan & • Each SQL executed only once

Statistics

Save Result • DDL/DML skipped

• Option to do Explain Plan only analysis

nge
ef ore Cha
B

SQL Performance Analyzer

 Step 4: Execute SQL After Making Change

SQL Tuning Set

Fetch Next SQL • Manually implement the planned change

• Database upgrade, patches
• Optimizer statistics refresh
• Schema changes
Test Execute • Database parameter changes
• Tuning actions, e.g., SQL Profile creation
Execution Plan &
Statistics

Save Result
• Re-execute SQL after change
Completed • Gathers new SQL execution plans
and statistics
nge e
e Cha Chang
Bef or After

SQL Performance Analyzer

 Step 5: Compare & Analyze Performance
Completed Completed
• Compare performance using
nge Chang
e
different metrics, e.g.,
e Cha After
Befor
• Elapsed Time
• CPU Time
• Optimizer Cost
• Buffer Gets
Compare
SQL Performance
• SPA Report shows impact of
change for each SQL
• Improved SQL
Analysis Report • Regressed SQL
• Unchanged SQL

• Fix regressed SQL using SQL

Tuning Advisor or SQL Plan
Baselines
SQL Performance Analyzer
SPA Report
SPA Report – Regressed SQL
Statements
 Best Practices
• Workload Capture
• Use incremental STS capture
• Peak or representative workloads can be more accurately captured
with this mechanism
• Test system
• Run SPA on test system instead of production as analysis can be
resource intensive
• Ensure test system has similar configuration and comparable
optimizer statistics as production
• Performance comparison
• Use several different metrics, e.g., elapsed time, CPU time, etc., to
compare pre- and post-change performance for reliable results
• Regression remediation
• Use SQL Tuning Advisor and SQL Plan Baselines to fix regressions
 Real Application Testing Summary

• Provides cost-effective and easy-to-use solution of assessing

impact of change on production systems
• Holistic and actual workload testing results in lower risk
• Brings down testing cycle from months down to days
• Lowers hardware costs by eliminating need for mid-tier and
application setup on test system
• Maximizes ROI by leveraging Diagnostics and Tuning Pack to
remediate regressions

• With Real Application Testing businesses can

• Stay competitive
• Improve profitability
• Be compliant
Q U E S T I O N S
A N S W E R S
Thank you for your attention !