Real Application Testing

Kevin Patterson, Senior Sales Consultant

Systems and Applications Management (SAM)
The following is intended to outline our general
product direction. It is intended for information
purposes only, and may not be incorporated into any
contract. It is not a commitment to deliver any
material, code, or functionality, and should not be
relied upon in making purchasing decisions.
The development, release, and timing of any
features or functionality described for Oracle’s
products remains at the sole discretion of Oracle.
 Oracle Enterprise Manager
Top-Down Application Management
Application Application
Operations Quality
Management Management
Real User
Experience Monitoring Load Testing and
Accelerators for
Service Level Packaged Apps &
Management Web Services
Application
Packaged Application Testing Suite Functional Testing and
Management Accelerators for
Packaged Apps &
SOA Application Management Web Services

Diagnostic and Tuning Test Management

Provisioning, Configuration
Management Real Application Testing

Change Tracking Data Masking

and Synchronization
Real Application Testing
Load and SQL impact testing solution providing
highest quality testing for the database tier
• Value
• Rapid technology adoption
Deploy
• Higher testing quality
• Business Benefit Test
• Lower cost Change
• Lower risk
• Features
• Database Replay Remediate
• SQL Performance Analyzer (SPA)

Business Agility through Superior

Testing
 Testing Today
Production – 1,000s of Real Online Users

Application
Servers

Database
Servers

PRODUCTION
 Testing Today
Test – 1-2 testers trying to be 1,000s of users

Application
Servers

Database Database
Servers Servers

PRODUCTION TEST
 Database Replay
Real workload for 1,000s of online users captured

Application
Servers

Database
Servers
Capture

Workload

PRODUCTION
 Database Replay
Real workload for 1,000s of online users replayed

Application
Servers

Database Database
Servers Servers
Capture Replay

Workload

PRODUCTION TEST
 Database Replay
Test your system changes at production levels

Application Application
Servers Servers

Database Database
Servers Servers
Capture Replay 11g ONLY
9.2.0.8,
10.2.0.3+, 11g
Workload

PRODUCTION TEST
 Real Application Testing with
Database Replay
• Replay production workload in test environment
• Identify, analyze and fix potential instabilities before making
changes to production
• Capture Workload in Production
• Capture full production workload with real load, timing &
concurrency characteristics
• Move the captured workload to test system
• Replay Workload in Test
• Make the desired changes in test system
• Replay workload with full production characteristics
• Honor commit ordering
• Analyze & Report
• Errors
• Data divergence Analysis & Reporting
• Performance divergence
Why DB Replay?

From: To:
Artificial workloads Production workloads

Partial workflows Complete workflows

Months of development Days of development

Manual intensive Automated

150 High risk Low risk 10

Days Days
 Supported Changes
with Real Application Testing

Changes
Unsupported Application
Servers

Changes Supported
Database
•Database Upgrades, Patches
Servers
•Schema, Parameters Recording of
External Client
•RAC nodes, Interconnect Requests
•OS Platforms, OS Upgrades
•CPU, Memory
•Storage
•Etc.
 Database Replay Workflow
Production
(9.2.0.8, 10gR2, 11g) Test (11g)
Clients
… Replay Driver

Mid-Tier

Storage
Storage

Analysis &
Capture Process Replay
Reporting
 <Insert Picture Here>

Application Quality
Management

Oracle Real Application Testing

Database Replay
Demonstration
 Database Replay Restrictions

• Database Replay does not support the following

features in the current release
• Direct Path Load (SQL Loader)
• Shared server (Oracle MTS)
• Oracle Streams & Advanced Replication Streams
• Non-PL/SQL based Advanced Queuing (AQ)
• Flashback queries
• OCI-based Object Navigation
• Non SQL-Based Object Access
• Distributed transactions, remote describe/commit
operations (will be replayed as local transactions)
 Real Application Testing with
SQL Performance Analyzer (SPA)
• Test impact of change on SQL query performance
• Capture SQL workload in production including statistics & bind variables
• Re-execute SQL queries in test environment
• Tune regressed SQL and seed SQL plans for production

Client Client Client

… Production Test

Middle Tier Re-execute SQL Queries

Capture SQL
… … Use SQL Tuning
Oracle DB Advisor to tune
regression
9i, 10g, 11g 10g, 11g

Storage
Why SQL Performance Analyzer (SPA)?

• Businesses want systems that are performant and

meet SLA’s
• SQL performance regressions are #1 cause of poor
system performance
• Solution for proactively detecting all SQL regressions
resulting from changes not available
• DBA’s use ineffective and time-consuming manual
scripts to identify problems

SPA identifies all changes in SQL performance before

impacting users
 SQL Performance Analyzer Workflow
Production
(9.2, 10.1, 10.2, 11g)
Clients
…
Test (10.2 or 11g)

Mid-Tier

Storage

Storage

Capture Transport Execute SQL Execute SQL Compare

SQL SQL Pre-change Post-change Perf
 SQL Performance Analyzer
Restrictions
• SQL Performance Analyzer does not support the
following features in the current release
• Shared server (Oracle MTS)
• 9i Parallel Query (SQL Trace Capture)
 <Insert Picture Here>

Application Quality
Management

Oracle Real Application Testing

SQL Performance Analyzer
Demonstration
 Real Application Testing: Tools of the Trade
SQL Performance Analyzer
What is it? • Predicts SQL performance deviations
before end-users can be impacted
What Purpose? • Assess impact of change on SQL
response time
How it works? • Executes each SQL, stored in SQL
Tuning Set, in isolation using
production context and then
compares before and after execution
plans and run-time statistics
When to use? • Unit testing of SQL with the goal to
identify the set of SQL statements
with improved/regressed performance
SQL Performance Analyzer and Database Replay are complementary
Database Replay
• Replays real database workload
on test system
• Assess impact of change on
workload throughput
• Captures workloads and replays it
with production characteristics
including concurrency,
synchronization &
dependencies
• Comprehensive testing of all
sub-systems of the database
server using real production
workload
 Why Consider Oracle 11g for Upgrades?
• Oracle Database 9i Release 2 Premium Support ended Jul 30, 2007
• 1 yr free Extended Support offer ends Jul 30, 2008
• Help enable upgrades from 9.2.0.8
Today

August 2012 August 2015

July 2010 July 2013

R2

January 2009 January 2012

Premier Support Extended Support Sustaining Support

R2 July 2007 July 2010
Jul 2008
t
2011

2013

2014
2002

2003

2004

2005

2006

2007

2008

2009

2010

2012

2016
2015
http://www.oracle.com/support/library/data-sheet/oracle-lifetime-support-policy-datasheet.pdf
 David Mitchell
Senior Vice President, OVUM

“Oracle Real Application Testing reduces the time

required to test changes by as much as 80%, lower
testing costs by as much as 70%, mitigate risks by
reducing the number of unexpected outages, and
improve the quality of service for their IT operations.”

Source: Oracle Real ApplicationTesting – business agility through superior testing, Jan 2008
 Further Information

• Oracle Real Application Testing Homepage on OTN

http://www.oracle.com/technology/products/manageability/database/index.html
• Data Sheets
• Technical and Customer Whitepapers
• Analyst Reports
• Tech Notes on support for Earlier Releases
• Examples and Scripts for Command Line Interface Usage
• Demo Viewlets

• Oracle Real Application Testing User Guide

http://download.oracle.com/docs/cd/B28359_01/server.111/e12253.pdf
 Further Information (Continued)
• 11G Top Features for DBAs and Developers
• http://www.oracle.com/technology/pub/articles/oracle-database-11g-
top-features/index.html
• 11G Oracle By Example Series:
• http://www.oracle.com/technology/obe/11gr1_db/index.htm
• Database Replay
• http://www.oracle.com/technology/obe/11gr1_db/manage/dbrepla
y/dbreplay.htm
• SQL Performance Analyzer
• http://www.oracle.com/technology/obe/11gr1_db/manage/spa/spa_g
uide.htm
• Metalink Notes for Patches and Steps to Implement:
• 560977.1: Real Application Testing Now Available for Earlier
Releases
• 562899.1: Oracle Real Application Testing: Test the SQL
Performance Impact of an Oracle 9i to Oracle Database 10g
Release 2 upgrade with SQL Performance Analyzer
 Database Replay for Earlier Releases

• Help customers smoothly transition to Oracle Database 11g

• “Workload Capture” portion of Database Replay is now available
with Oracle 9i and Oracle Database 10g
• “Workload Replay” only possible on Oracle Database 11g and above
• For more details:
• Metalink Note: 560977.1 – “Real Application Testing for Earlier
Releases”

Upgrade Upgrade Release / Patches Comments

From To needed
10gR2 11g • 10.2.0.4 Patchset • Need to apply one-off patch for 10.2.0.2 and
• 10.2.0.x + one-off patch 10.2.0.3

9iR2 11g • 9.2.0.8 + one-off patch • Limited performance data available for
captured workload – can use STATSPACK
 SQL Performance Analyzer
for Earlier Releases
• Help customers smoothly transition to Oracle Database 10g and 11g
• SQL Test Execution feature of SPA backported to 10gR2
• For more details:
• Metalink Note: 560977.1 – “Real Application Testing for Earlier Releases”
• Metalink Note: 562899.1 – “Testing Performance Impact of an Oracle 9i to
Oracle Database 10g Release 2 Upgrade with SQL Performance Analyzer”

Upgrade Upgrade Release / Patches Comments

From To needed
10gR2 10gR2 or • 11.1.0.6 + one-off patch • Uses STS to capture SQL stats
11g or 11.1.0.7
• 10gR2 + one-off patch

9i 10gR2 or • 11.1.0.6 + one-off patch • Uses SQL*Trace to capture SQL stats

10gR1 11g or 11.1.0.7
• 10gR2 + one-off patch