<Insert Picture Here>

Oracle Real Application Clusters

NAME, Title

Agenda
What is RAC? RAC for Scalability RAC for High Availability Consolidation with Oracle Clusterware and RAC RAC as the Grid Infrastructure
<Insert Picture Here>

Real Application Clusters

Benefits Highest Availability On-demand flexible scalability Lower computing costs World record performance

Database

Storage

RAC Adoption Rate Growing

15,000 12,000 9,000 6,000 3,000 0 FY02 FY03 FY04 FY05 FY06 FY07 FY08 FY09 (Q2)
Source: Oracle Internal Reporting, FY09

RAC: The Cluster Database

Network

Centralized Management Console Interconnect High Speed Switch or Interconnect

Users
No Single Point Of Failure

Clustered Database Servers

Hub or Switch Fabric Mirrored Disk Subsystem

Storage Area Network Drive and Exploit Industry Advances in Clustering

ASM & RAC, Understand the Architecture

public network

VIP1

VIP2

VIP3

Node1
ASM instance 1 Database instance 1 CRS Operating System
cluster interconnect

Node 2
ASM instance 2 Database instance 2 CRS Operating System
cluster interconnect

Node3
ASM instance 3 Database instance 3 CRS Operating System

...

shared storage

Managed by ASM

redo logs all instances

database & control files OCR & voting disk (oracle_home)

Single system command: srvctl

Start/stop the database and all enables instances srvctl start|stop|status database -d prod Start the instance PROD2 srvctl start instance -d prod -i PROD2 Display the current status srvctl status|config database -d prod Start/stop VIP, GSD, TNS listener etc. srvctl start|stop|status nodeapps -n <host> You can do a lot with srvctl, see manual for more info

<Insert Picture Here>

Single Client Access Name

Connection Load Balancing

Application Server

Oracle RAC Database SCAN Listeners Clients Local Listeners

Connection Load Balancing

Application Server

Oracle RAC Database SCAN Listeners Clients Listeners

Oracle Database 11g Release 2

Single Client Access Name (SCAN)

DW

2009 Oracle Corporation Proprietary and Confidential

RAC ONE A RAC ONE B RAC ONE C RAC ONE D RAC ONE E RAC ONE F RAC ONE G RAC ONE H RAC ONE I RAC ONE J RAC ONE K RAC ONE L

mycluster.myco.com

App Servers Front Office Back Office

Used by clients to connect to any database in the cluster Removes the requirement to change the client connection if cluster changes Load balances across the instances providing a service Provides failover between moved instances

Free

Single Client Access Name (SCAN)

A DNS entry is required (unless using GNS)
A single name defined to resolve to 3 addresses (the scanvips)

Each cluster will have 3 scan listeners, each having a scanvip defined as cluster resources A SCAN VIP/LISTENER will failover to another node in cluster
Cluster Resources -------------------------------------------ora.LISTENER_SCAN1.lsnr 1 ora.LISTENER_SCAN2.lsnr 1 ora.LISTENER_SCAN3.lsnr 1 ONLINE ONLINE ONLINE ONLINE ONLINE ONLINE node1 node2 node3

Oracle Clusterware SCAN Resources

Created by root script during Grid Infrastructure Installation $srvctl config scan_listener
SCAN Listener LISTENER_SCAN1 exists. Port: TCP:1521 SCAN Listener LISTENER_SCAN2 exists. Port: TCP:1521 SCAN Listener LISTENER_SCAN3 exists. Port: TCP:1521

$srvctl config scan

SCAN SCAN SCAN SCAN name: sales1-scan, Network: 1/192.87.58.0/255.255.255.0/ VIP name: scan1, IP: /sales1-scan.mycompany.com/192.87.58.143 VIP name: scan2, IP: /sales1-scan.mycompany.com/192.87.58.99 VIP name: scan3, IP: /sales1-scan.mycompany.com/192.87.58.100

Single Client Access Name (SCAN)

SCAN listener is the load balancer for the cluster. Allows clients to use EZConnect or simple JDBC connections Instance registers with local listener on its node Database remote_listener registers instances with all SCAN listeners
sqlplus system/manager@ sales1-scan:1521/oltp

jdbc:oracle:thin:@sales1-scan:1521/oltp

SQL*NET and SCAN

SQL*NET will retrieve the IP addresses from DNS or GNS, it will then load balance and failover across the IP addresses. For MAA implementations, if client uses both primary and standby in address list, SQL*NET will retrieve all 6 IPs, it will then load balance and failover across the 6 IP addresses
sales.mycompany.com =(DESCRIPTION= (CONNECT_TIMEOUT=10)(RETRY_COUNT=3) (ADDRESS_LIST= (LOAD_BALANCE=on)(FAILOVER=ON) (ADDRESS=(PROTOCOL=tcp)(HOST=scan1)(PORT=1521)) (ADDRESS=(PROTOCOL=tcp)(HOST=scan2)(PORT=1521))) (CONNECT_DATA= (SERVICE_NAME= sales.mycompany.com)))

SQL*NET and SCAN

If load_balance is OFF, SQL*NET will retrieve the IP addresses for the SCAN and sequentially go through the list. For EZConnect, Load_balance is OFF
sales.mycompany.com =(DESCRIPTION=(ADDRESS_LIST= (LOAD_BALANCE=OFF)(FAILOVER=ON) (ADDRESS=(PROTOCOL=tcp)(HOST=scan1)(PORT=1521)) (CONNECT_DATA= (SERVICE_NAME= sales.mycompany.com)))

<Insert Picture Here>

Oracle RAC Services

Enterprise Grid Services

Service is an abstraction that customers use to divide work into logical workloads
globally unique name service quality levels (performance, Priority, availability) Unit of Management Location Transparency Traditional HA(stop/start/relocate) operations Workload Management Dependencies and notification

Fast, lights-out recovery operation

Database services
tnsnames.ora GL=(DESCRIPTION= (ADDRESS= (PROT=TCP) (HOST=BLEU) PORT=1521) ) (CONNECT_DATA= (SERVICE_NAME=GL) ) ) tnsnames.ora CRM=(DESCRIPTION= (ADDRESS= (PROT=TCP) (HOST=BLEU) PORT=1521) ) (CONNECT_DATA= (SERVICE_NAME=CRM) ) )

GL

CRM

PROD GL CRM

GL

PROD

CRM

Instance parameter :
SERVICE_NAMES=PROD,GL,CRM

Use EM to Define Services

Published services

<Insert Picture Here>

Oracle RAC for Scalability

Real Application Clusters

Flexible Scalability for Applications

Order Entry Supply Chain

Application scalability by load balancing work across cluster As workload increases, easily add new node(s) to the cluster Cache Fusion performance improvements with 11g

Batch

Order Entry

Supply Chain

RAC Provisioning
Simple, Accurate, Reliable Capacity on Demand

Automate RAC provisioning while providing service

Convert DB to RAC One-click-extend RAC to meet increased demand De-provision RAC nodes

Simplified way to scale up or down RAC

Integrated provisioning of RAC, ASM, and Clusterware Bare metal provisioning integrated with RAC provisioning

Gold image based initial RAC and Clusterware provisioning Flexible and Adaptable to your IT environment

ADDM for Oracle RAC

Database-Level ADDM 11g

Self-Diagnostic Engine

New in 11g

Performance expert in a box Identify the most Globally Significant performance problems for the entire Oracle RAC database Database-wide analysis of:
Global cache interconnect issues Global resource contention, e.g. IO bandwidth, hot blocks Globally high-load SQL Skew in instance response times

Instance-Level ADDM

Inst 1
AWR 1

Inst 2
AWR 2

Inst 3 AWR 3

Runs proactively every hour when taking AWR snapshots (default)

Load Balancing Advisory

Load Balancing Advisory is an advisory for balancing work across Oracle RAC instances. Load balancing advice
Is available to ALL applications that send work. Directs work to where services are executing well and resources are available. Adjusts distribution for different power nodes, different priority and shape workloads, changing demand. Stops sending work to slow, hung, failed nodes early.

Client connection pool is integrated with Oracle RAC load balancing advisory When application does getConnection, the connection given is the one that will provide the best service.

Automatic Workload Management

Runtime Connection Load Balancing Supported by JDBC, ODP.NET, OCI Client connection pool is integrated with Oracle RAC load balancing advisory When application does getConnection, the connection given is the one that will provide the best service. Policy defined by setting GOAL on Service Need to have Connection Load Balancing

PQ integration with Services

Parallel Query slaves will only execute on nodes where the service of the query owner is active. No longer have to code instance_groups
Node-1 Node-2 Node-3 Node-4 Node-5 Node-6

OLTP 1 OLTP 2 OLTP 3

OLTP 4
Repor ting Batch

DW

Mercado Libre
Load Distribution

Marketplace Searches, Listings, Bidding, Reporting

Customer Service

Newer nodes are faster and have more memory Marketplace workload is not partitioned

Amazon Data Warehouse Superstack 3rd Cluster

Having a 3rd copy allows us to buy 1.5x the amount of processing power instead of 2x. We replace the oldest cluster every year to keep up with growing requirements

1. Extract from source systems

2. Integrate, transform, de-normalize, query and analyze

4. Data access and publishing

Extract Servers
17nodes, 8CPUs, 16-32 GB RAM, 100+ TB, 1Gb IC

ETL Manager Amazon S3

Amazon DCS 16 nodes, 16 CPUs, 64GB RAM, 100+ TB, IB IC

<Insert Picture Here>

Oracle RAC for High Availability

High Availability
Removes Server as Single Point of Failure
Integral part of Oracles Maximum Availability Architecture Enables applications to mask component failures in infrastructure
Order Entry Spare Supply Chain

Server Failure

Automatic Workload Management

Service fails over to surviving Node

Order Entry

Supply Chain

Re-allocate spare to Order Entry

Oracle: High Availability Without the High Cost

Traditionally High Quality = High Cost
High quality systems were built by combining high quality, high cost parts Mainframe model

Oracle enables a new model: High Quality + Low Cost

Oracles vision is to attain the highest possible availability using software that can also easily scale out to meet additional load requirements Fully integrated and simple to manage
Enable widespread deployment of HA configurations

Fast Application Notification (FAN)

Immediate Notification to Clients
FAN Event

c c c c c c cc c c
Application

Connection Pool Oracle Real Application Clusters

Fast Connection Failover

Supports multiple connection caches Datasource for each cache mapped to a service Keeps track of service and instance for each connection Cleans up connections when failures occur Distributes new work requests across available instances
Mid-Tier
CACHES SERVICE 1 SERVICE 2 SERVICE 3 INST Z INST X

Database Tier

INST Y

JDBC, ODP.NET, OCI

APPLICATION TODAY
Trading Application

Order Entry Bus

Pool A
FAN

Pool C

Pool B

Connection Cache

Service A Service C

Service B Service C

2 node RAC

ASM

Notification Callouts
Users can write callout programs that are invoked when notifications occur
Notifications include node up/down, instance up/down, or service up/down

Example usages:
Send email, page Log status info Start / stop programs

EM driven Oracle RAC Rolling Patching

a
1 Both nodes running 2
Node 1 shut down/patched/started

EM

a
4

a
3 Nodes in mixed mode

Node 2 shut down & patched

<Insert Picture Here>

Consolidation with Oracle Clusterware and Oracle RAC

Cluster Consolidation
listener database cluster database 2

Protected App A listener cluster database

Protected App B listener cluster database

Protected App C listener cluster database 2

Consolidated Pool of Storage with Automatic Storage Management

Be careful to size your servers to fit the required capacity.

Oracle Clusterware Licensing

See Oracle Database Licensing Information B14199-01

At a minimum, one of the machines involved in the cluster must be licensed using the appropriate metric for any Oracle Server Technology product in either its Standard Edition or Enterprise Edition, or all of the machines in the cluster must be supported via Oracle Unbreakable Linux support. A cluster is defined to include all the machines that share the same Oracle Cluster Registry (OCR) and Voting Disk

Managing a Single Instance in a Cluster

1 Single Instance
OS/Clusterware Upgrade ? Add Instance

a
4 Move Back to Original Node 3 Failover Clients to New Instance, shutdown original

Consolidated Clusters

8 SKU databases consolidated into two 18 node Linux clusters. ASM provides pooled storage Database consolidation next

Consolidated Clusters

Instances are brought up on nodes as workload or maintenance requires.

Consolidated Clusters

In case of outage, Data Guard recovers, and work is adjusted over remaining nodes

Grid Architecture
Corporate DW (9i + raw) SAP SEM/BI (9i + ocfsv1) Electricity Dispat. (10g + asm) SAP ISU (10g + asm) Siebel Brazil (10g + asm)
Provisioning Between Systems

Combined OLTP & DW

Production Servers

Development/Test

Siebel Brazil planned production Feb 07, SAP ISU Mid 07 Provisioning Nodes is Future Plans

<Insert Picture Here>

Oracle Real Application Clusters One Node

Better Virtualization for Databases

Oracle RAC One Node

New single-instance Oracle RAC configuration :
Database Relocation migrate database to another server, supports:
Oracle RAC One Node Oracle RAC One Node Oracle RAC One Node

Different processor generation, Different OS and DB levels.

Rolling patches for single instance and OS Built-in high availability Online upgrade to Oracle RAC

Oracle Clusterware Automatic Storage Management

Allows customer to standardize on one database deployment and management model

Supported in both physical and virtual environments

Oracle RAC One Node

Always On Single Instance Database

Online replacement of servers and storage Online patching and upgrades of operating system and infrastructure software Online database patching Automated cluster failover

Better consolidation
Extreme consolidation of servers & storage Load balancing to protect service levels

Enhanced virtualization
Extends and improves database availability and flexibility when running in a virtual server

Oracle RAC One Node

Key Attributes:
Instance Caging Single Cluster Shared storage

Server 1

Server 2

Server 3

DB-A

DB-B

DB-C

DB- D

DB- E

Common Shared Storage

Single Cluster

Oracle RAC One Node: Database Relocation

Client Connections
Server 1 Server 2 Server 3

DB-A

DB-B

DB-C

DB-D

DB-E

Common Shared Storage

Single Cluster

Oracle RAC One Node: Database Relocation

Client Connections
Server 1 Server 2 Server 3

DB-A

DB-B

DB-B

DB-C

DB-D

DB-E

Common Shared Storage

Single Cluster

Oracle RAC One Node: Database Relocation

Client Connections
Server 1 Server 2 Server 3

DB-A

DB-B

DB-B

DB-C

DB-D

DB-E

Common Shared Storage

Single Cluster

Oracle RAC One Node: Database Relocation

Client Connections
Server 1 Server 2 Server 3

DB-A

DB-B

DB-B

DB-C

DB-D

DB-E

Common Shared Storage

Single Cluster

Oracle RAC One Node: Database Relocation

Client Connections
Server 1 Server 2 Server 3

DB-A

DB-B

DB-C

DB-D

DB-E

Common Shared Storage

Single Cluster

Oracle RAC One Node: Online Rolling Patches

Client Connections
Server 1 Server 2 Server 3

DB-A

DB-B

DB-C

DB-D

DB-E

Database Binaries

Common Shared Storage

Single Cluster

Oracle RAC One Node: Online Rolling Patches

Client Connections
Server 1 Server 2 Server 3

Patch DB-A DB-B DB-C DB-D DB-E

Database Binaries

Common Shared Storage

Single Cluster

Oracle RAC One Node: Online Rolling Patches

Client Connections
Server 1 Server 2 Server 3

DB-A

DB-B

DB-C

DB-D

DB-E

Patched Database Binaries

Common Shared Storage

Single Cluster

Oracle RAC One Node: Online Rolling Patches

Client Connections
Server 1 Server 2 Server 3

DB-A

DB-B

DB-B

DB-C

DB-D

DB-E

Patched Database Binaries

Common Shared Storage

Single Cluster

Oracle RAC One Node: Online Rolling Patches

Client Connections
Server 1 Server 2 Server 3

DB-A

DB-B

DB-B

DB-C

DB-D

DB-E

Patched Database Binaries

Common Shared Storage

Single Cluster

Oracle RAC One Node: Online Rolling Patches

Client Connections
Server 1 Server 2 Server 3

DB-A

DB-B

DB-B

DB-C

DB-D

DB-E

Patched Database Binaries

Common Shared Storage

Single Cluster

Oracle RAC One Node: Online Rolling Patches

Client Connections
Server 1 Server 2 Server 3

DB-A

DB-B

DB-C

DB-D

DB-E

Patched Database Binaries

Common Shared Storage

Single Cluster

Oracle RAC One Node: Cluster Failover

Server 1

Server 2

Server 3

DB-A

DB-B

DB-C

DB-D

Common Shared Storage

Single Cluster

Oracle RAC One Node: Cluster Failover

Server 1

Server 2

Server 3

DB-A

DB-B

DB-C

DB-D

Common Shared Storage

Single Cluster

Oracle RAC One Node: Cluster Failover

Key Capabilities:
Database failover Fast storage failover

Server 1

Server 2

Server 3

DB-A

DB-B

DB-C

DB-D

Common Shared Storage

Single Cluster