Beruflich Dokumente
Kultur Dokumente
Application Clusters
D17276GC10
Edition 1.0
December 2004
D40139
Authors Copyright © 2004, Oracle. All rights reserved.
All other products or company names are used for identification purposes only, and
may be trademarks of their respective owners.
Contents
I Introduction
Overview I-2
What Is a Cluster? I-3
What Is Oracle Real Application Clusters? I-4
Why Use RAC? I-5
Clusters and Scalability I-6
Levels of Scalability I-7
Scaleup and Speedup I-8
Speedup/Scaleup and Workloads I-9
A History of Innovation I-10
Course Objectives I-11
Typical Schedule I-12
iii
2 RAC Installation and Configuration (Part I)
Objectives 2-2
Oracle Database 10g RAC Installation: New Features 2-3
Oracle Database 10g RAC Installation: Outline 2-5
Preinstallation Tasks 2-6
Hardware Requirements 2-7
Network Requirements 2-8
RAC Network Software Requirements 2-9
Package Requirements 2-10
hangcheck-timer Module Configuration 2-11
Required UNIX Groups and Users 2-12
The oracle User Environment 2-13
User Shell Limits 2-14
Configuring User Equivalency 2-15
Required Directories for the Oracle Database Software 2-17
Linux Kernel Parameters 2-19
Cluster Setup Tasks 2-21
Obtaining OCFS 2-22
Installing the OCFS RPM Packages 2-23
Starting ocfstool 2-24
Generating the ocfs.conf File 2-25
Preparing the Disks 2-26
Loading OCFS at Startup 2-27
Mounting OCFS on Startup 2-28
Using Raw Partitions 2-30
Binding the Partitions 2-31
Raw Device Mapping File 2-33
Installing Cluster Ready Services 2-35
Specifying the Inventory Directory 2-36
File Locations and Language Selection 2-37
Cluster Configuration 2-38
Private Interconnect Enforcement 2-39
Oracle Cluster Registry File 2-40
Voting Disk File 2-41
Summary and Install 2-42
Running the root.sh Script on All Nodes 2-43
Verifying the CRS Installation 2-44
Summary 2-46
Practice 2: Overview 2-47
iv
3 RAC Installation and Configuration (Part II)
Objectives 3-2
OUI Database Configuration Options 3-3
Install the Database Software 3-4
Specify File Locations 3-5
Specify Cluster Installation 3-6
Select Installation Type 3-7
Products Prerequisite Check 3-8
Select Database Configuration 3-9
Check Summary 3-10
The root.sh Script 3-11
Launching the VIPCA with root.sh 3-12
VIPCA Network Interface Discovery 3-13
VIP Configuration Data and Summary 3-14
Installation Progress 3-15
End of Installation 3-16
Database Preinstallation Tasks 3-17
Creating the Cluster Database 3-19
Node Selection 3-20
Select Database Type 3-21
Database Identification 3-22
Cluster Database Management Method 3-23
Passwords for Database Schema Owners 3-24
Storage Options for Database Files 3-25
Database File Locations 3-27
Flash Recovery Area 3-28
Database Components 3-29
Database Services 3-30
Initialization Parameters 3-31
Database Storage Options 3-32
Create the Database 3-33
Monitor Progress 3-34
Manage Default Accounts 3-35
Postinstallation Tasks 3-36
Patches and the RAC Environment 3-37
Inventory List Locks 3-38
Summary 3-39
Practice 3: Overview 3-40
v
4 RAC Database Instances Administration
Objectives 4-2
The EM Cluster Database Home Page 4-3
Cluster Database Instance Home Page 4-5
Cluster Home Page 4-6
The Configuration Section 4-7
Operating System Details Page 4-8
Performance and Targets Pages 4-9
Starting and Stopping RAC Instances 4-10
Starting and Stopping RAC Instances with EM 4-11
Starting and Stopping RAC Instances with SQL*Plus 4-12
Starting and Stopping RAC Instances with SRVCTL 4-13
RAC Initialization Parameter Files 4-14
SPFILE Parameter Values and RAC 4-15
EM and SPFILE Parameter Values 4-16
RAC Initialization Parameters 4-18
Parameters Requiring Identical Settings 4-20
Parameters Requiring Unique Settings 4-21
Adding a Node to a Cluster 4-22
Adding a Node to an Existing Cluster 4-23
Adding the RAC Software to the New Node 4-25
Reconfigure the Listeners 4-27
Add an Instance by Using DBCA 4-28
Deleting Instances from a RAC Database 4-29
Node Addition and Deletion and the SYSAUX Tablespace 4-31
Quiescing RAC Databases 4-32
How SQL*Plus Commands Affect Instances 4-33
Administering Alerts with Enterprise Manager 4-34
Viewing Alerts 4-35
Blackouts and Scheduled Maintenance 4-37
Summary 4-39
vi
Oracle Linux ASMLib Installation 5-11
ASM Library Disk Creation 5-13
ASM Administration 5-15
ASM Instance Functionalities 5-16
ASM Instance Creation 5-17
ASM Instance Initialization Parameters 5-18
RAC and ASM Instances Creation 5-19
ASM Instance Initialization Parameters and RAC 5-20
Discovering New ASM Instances with EM 5-21
Accessing an ASM Instance 5-22
Dynamic Performance View Additions 5-23
ASM Home Page 5-24
ASM Performance Page 5-25
ASM Configuration Page 5-26
Starting Up an ASM Instance 5-27
Shutting Down an ASM Instance 5-28
ASM Administration 5-29
ASM Disk Group 5-30
Failure Group 5-31
Disk Group Mirroring 5-32
Disk Group Dynamic Rebalancing 5-33
ASM Administration Page 5-34
Create Disk Group Page 5-35
ASM Disk Groups with EM in RAC 5-36
Disk Group Performance Page and RAC 5-37
Create or Delete Disk Groups 5-38
Adding Disks to Disk Groups 5-39
Miscellaneous Alter Commands 5-40
Monitoring Long-Running Operations Using V$ASM_OPERATION 5-42
ASM Administration 5-43
ASM Files 5-44
ASM File Names 5-45
ASM File Name Syntax 5-46
ASM File Name Mapping 5-48
ASM File Templates 5-49
Template and Alias: Examples 5-50
Retrieving Aliases 5-51
SQL Commands and File Naming 5-52
DBCA and Storage Options 5-53
Database Instance Parameter Changes 5-54
vii
Summary 5-56
Practice 5 Overview 5-57
7 Services
Objectives 7-2
Traditional Workload Dispatching 7-3
Grid Workload Dispatching 7-4
What Is a Service? 7-5
High Availability of Services in RAC 7-6
Possible Service Configuration with RAC 7-7
Service Attributes 7-8
Service Types 7-9
Creating Services 7-10
Creating Services with DBCA 7-11
Creating Services with SRVCTL 7-13
Preferred and Available Instances 7-14
Everything Switches to Services 7-15
Using Services with Client Applications 7-16
Using Services with Resource Manager 7-17
Services and Resource Manager with EM 7-18
Services and Resource Manager: Example 7-19
Using Services with Scheduler 7-20
viii
Services and Scheduler with EM 7-21
Services and Scheduler: Example 7-23
Using Services with Parallel Operations 7-24
Using Services with Metric Thresholds 7-25
Changing Service Thresholds Using EM 7-26
Services and Metric Thresholds: Example 7-27
Service Aggregation and Tracing 7-28
Cluster Database: Top Services 7-29
Service Aggregation Configuration 7-30
Service Aggregation: Example 7-31
The trcsess Utility 7-32
Service Performance Views 7-33
Managing Services 7-34
Managing Services with EM 7-36
Managing Services: Example 7-38
Summary 7-39
Practice 7 Overview 7-40
ix
TAF Verification 8-24
FAN Connection Pools and TAF Considerations 8-25
Restricted Session and Services 8-26
Summary 8-27
Practice 8 Overview 8-28
x
Typical Latencies for RAC Operations 10-6
Wait Events for RAC 10-7
Wait Event Views 10-8
Global Cache Wait Events: Overview 10-9
2-way Block Request: Example 10-11
3-way Block Request: Example 10-12
2-way Grant: Example 10-13
Considered “Lost” Blocks: Example 10-14
Global Enqueue Waits: Overview 10-15
Session and System Statistics 10-16
Most Common RAC Tuning Tips 10-17
Index Block Contention Considerations 10-19
Oracle Sequences and Index Contention 10-20
Undo Block Considerations 10-21
High-Water Mark Considerations 10-22
Cluster Database Performance Page 10-23
Cluster Cache Coherency Page 10-25
Database Locks Page 10-26
Automatic Workload Repository: Overview 10-27
AWR Tables 10-28
AWR Snapshots in RAC 10-29
Generating and Viewing AWR Reports 10-30
AWR Reports and RAC: Overview 10-31
Statspack and AWR 10-33
Automatic Database Diagnostic Monitor 10-34
ADDM Problem Classification 10-35
RAC-Specific ADDM Findings 10-36
ADDM Analysis: Results 10-37
ADDM Recommendations 10-38
Summary 10-39
Practice 10: Overview 10-40
xi
Data Guard Broker (DGB) and CRS Integration 11-11
Data Guard Broker Configuration Files 11-12
Hardware Assisted Resilient Data 11-13
Rolling Patch Upgrade Using RAC 11-14
Rolling Release Upgrade Using SQL Apply 11-15
Database High Availability Best Practices 11-16
Extended RAC: Overview 11-17
Extended RAC Connectivity 11-18
Extended RAC Disk Mirroring 11-19
Additional Data Guard Benefits 11-20
Using Distributed Transactions with RAC 11-21
Using a Test Environment 11-22
Summary 11-23
Appendix A: Practices
Appendix B: Solutions
xii
Introduction
Overview
The material in this course is designed to provide basic information that is needed to plan
or manage Oracle Database 10g for Real Application Clusters.
The lessons and practices are designed to build on your knowledge of Oracle used in a
nonclustered environment. The material does not cover basic architecture and database
management; these topics are addressed by the Oracle Database 10g administration courses
offered by Oracle University. If your background does not include working with a current
release of the Oracle database, then you should consider taking such training before
attempting this course.
The practices provide an opportunity for you to work with the features of the database that
are unique to Real Application Clusters.
Node
• Interconnected nodes
act as a single server.
• Cluster software
hides the structure.
• Disks are available
for read and
write by all nodes.
Disks
Interconnect
Cluster ware
on each node
What Is a Cluster?
A cluster consists of two or more independent, but interconnected, servers. Several
hardware vendors have provided cluster capability over the years to meet a variety of
needs. Some clusters were only intended to provide high availability by allowing work to
be transferred to a secondary node if the active node fails. Others were designed to provide
scalability by allowing user connections or work to be distributed across the nodes.
Another common feature of a cluster is that it should appear to an application as if it were a
single server. Similarly, management of several servers should be as similar to the
management of a single server as possible. The cluster management software provides this
transparency.
For the nodes to act as if they were a single server, files must be stored in such a way that
they can be found by the specific node that needs them. There are several different cluster
topologies that address the data access issue, each dependent on the primary goals of the
cluster designer.
The interconnect is a physical network used as a means of communication between each
node of the cluster.
In short, a cluster is a group of independent servers that cooperate as a single system.
Memory Shared
storage
Level of Scalability
Successful implementation of cluster databases requires optimal scalability on four levels:
• Hardware scalability: Interconnectivity is the key to hardware scalability, which
greatly depends on high bandwidth and low latency.
• Operating system scalability: Methods of synchronization in the operating system
can determine the scalability of the system. In some cases, potential scalability of the
hardware is lost because of the operating system’s inability to handle multiple
resource requests simultaneously.
• Database management system scalability: A key factor in parallel architectures is
whether the parallelism is affected internally or by external processes. The answer to
this question affects the synchronization mechanism.
• Application scalability: Applications must be specifically designed to be scalable. A
bottleneck occurs in systems in which every session is updating the same data most of
the time. Note that this is not RAC specific and is true on single-instance system too.
It is important to remember that if any of the above areas are not scalable (no matter how
scalable the other areas are), then parallel cluster processing may not be successful. A
typical cause for the lack of scalability is one common shared resource that must be
accessed often. This causes the otherwise parallel operations to serialize on this bottleneck.
A high latency in the synchronization increases the cost of synchronization, thereby
counteracting the benefits of parallelization. This is a general limitation and not a RAC-
specific limitation.
Oracle Database 10g: Real Application Clusters I-7
Scaleup and Speedup
Original system
Hardware Time up to
200% Hardware
of up to
100%
task 300%
Hardware of task
Time of
Hardware
task Time/2
Hardware
Time
Automatic Enterprise
Storage
Management Grids
RAC Automatic
Workload
Data management
Low-cost
Guard
commodity
clusters
Nonblocking Resource
queries Manager
OPS
A History of Innovation
Oracle Database 10g and the specific new manageability enhancements provided by Oracle
RAC 10g enable RAC for everyone—all types of applications and enterprise grids, the
basis for fourth-generation computing. Enterprise grids are built from large configurations
of standardized, commodity-priced components: processors, network, and storage. With
Oracle RAC’s cache fusion technology, the Oracle database adds to this the highest levels
of availability and scalability.
Also, with Oracle RAC 10g, it becomes possible to perform dynamic provisioning of
nodes, storage, CPUs, and memory to maintain service levels more easily and efficiently.
Enterprise grids are the data centers of the future and enable business to be adaptive,
proactive, and agile for the fourth generation.
The next major transition in computing infrastructure is going from the era of big SMPs to
the era of grids.
Course Objectives
This course is designed to give you the necessary information to successfully administer
Real Application Clusters.
You install Oracle Database 10g with Oracle Universal Installer (OUI) and create your
database with the Database Configuration Assistant (DBCA). This ensures that your RAC
environment has the optimal network configuration, database structure, and parameter
settings for the environment that you selected. As a DBA, after installation your tasks are to
administer your RAC environment at three levels:
• Instance administration
• Database administration
• Cluster administration
Throughout this course you use various tools to administer each level of RAC:
• Oracle Enterprise Manager 10g Database Control to perform administrative tasks
whenever feasible
• Task-specific GUIs such as the Database Configuration Assistant (DBCA) and the
Virtual Internet Protocol Configuration Assistant (VIPCA)
• Command-line tools such as SQL*Plus, Recovery Manager, Server Control
(SRVCTL).
3-4 2
7-8-9 4
Typical Schedule
The lessons in this guide are arranged in the order that you will probably study them in
class, and are grouped into the topic areas that are shown in the slide. The individual
lessons are ordered so that they lead from more familiar to less familiar areas. The related
practices are designed to let you explore increasingly powerful features of a Real
Application Clusters database.
In some cases, the goals for the lessons and goals for the practices are not completely
compatible. Your instructor may, therefore, choose to teach some material in a different
order than found in this guide. However, if your instructor teaches the class in the order in
which the lessons are printed in this guide, then the class should run approximately as
shown in this schedule.
Applications
Applications/RAC
Services framework
System Management
Cluster control
Management APIs
Event Services
Event Services
Cluster control/Recovery APIs
Volume Manager
file system Automatic Storage Management
Messaging and Locking
Messaging and Locking
Membership Membership
Connectivity Connectivity
Cluster
Node1 Noden
Instance1 Instancen
Cache Cache
… Global …
LMON resources LMON
LMD0 LMD0
LMSx LMSx
LCK0 LCK0
DIAG DIAG
Client Client
process process
Shared storage
OCR
file
OCR Architecture
Cluster configuration information is maintained in Oracle Cluster Registry. OCR relies on a
distributed shared-cache architecture for optimizing queries against the cluster repository.
Each node in the cluster maintains an in-memory copy of OCR, along with an OCR process
that accesses its OCR cache. Only one of the OCR processes actually reads from and writes
to the OCR file on shared storage. This process is responsible for refreshing its own local
cache, as well as the OCR cache on other nodes in the cluster. For queries against the
cluster repository, the OCR clients communicate directly with the local OCR process on
the node from which they originate. When clients need to update the OCR, they
communicate through their local OCR process to the OCR process that is performing
input/output (I/O) for writing to the repository on disk.
The OCR client applications are Oracle Universal Installer (OUI), SRVCTL, Enterprise
Manager (EM), Database Configuration Assistant (DBCA), Database Upgrade Assistant
(DBUA), NetCA, and Virtual Internet Protocol Configuration Assistant (VIPCA).
Furthermore, OCR maintains dependency and status information for application resources
defined within CRS, specifically databases, instances, services, and node applications.
The name of the configuration file is ocr.loc, and the configuration file variable is
ocrconfig_loc. The location for the cluster repository is not restricted to raw devices.
You can put OCR on shared storage that is managed by a Cluster File System.
Note: OCR also serves as a configuration file in a single instance with the ASM, where
there is one OCR per node.
Node1 Noden
…
Instance1 Instancen
Archived Archived
log files log files
Local storage Local storage
Shared storage
Operating system
• Using CFS:
– Simpler management
– Use of OMF with RAC
– Single Oracle software installation
– Autoextend
• Using raw:
– Performance
– Use when CFS not available
– Cannot be used for archivelog files (on UNIX)
Raw or CFS?
As already explained, you can either use a cluster file system or place files on raw devices.
Cluster file systems provide the following advantages:
• Greatly simplify the installation and administration of RAC
• Use of Oracle Managed Files with RAC
• Single Oracle software installation
• Autoextend enabled on Oracle data files
• Uniform accessibility to archive logs in case of physical node failure
Raw devices implications:
• Raw devices are always used when CFS is not available or not supported by Oracle.
• Raw devices offer best performance without any intermediate layer between Oracle
and the disk.
• Autoextend fails on raw devices if the space is exhausted.
• ASM, Logical Storage Managers, or Logical Volume Managers can ease the work
with raw devices. Also, they can enable you to add space to a raw device online, or
you may be able to create raw device names that make the usage of this device clear
to the system administrators.
Servers
Interconnect
1008 1008
1 2
Lost
updates!
1008 1008
4 3
Cluster
Node1 Noden
Instance1 Instancen
GRD Master Cache GRD Master Cache
… Global …
LMON GES resources GES LMON
LMD0 LMD0
LMSx GCS GCS LMSx
LCK0 Interconnect LCK0
DIAG DIAG
Cluster
Node1 Node2
Instance1 Instance2
Cache 1009 1009 Cache
… 3 …
LMON LMON
LMD0 LMD0
LMSx 4 LMSx
LCK0 LCK0
DIAG DIAG
2 1
Instance two has
Block mastered the current version of the block Which instance
by instance one GCS masters the block?
Cluster
Node1 Node2
Instance1 Instance2
Cache 1009 1010 Cache
… 3 …
LMON LMON
LMD0 LMD0
LMSx LMSx
LCK0 5 4 LCK0
DIAG DIAG
1 2
Block flushed, make room
Need to make room
Instance two owns it.
in my cache.
GCS Instance two, flush the block
Who has the current version
to disk
of that block?
Only one
disk I/O
1010
Use information
Remaster for other caches
enqueue LMON
resources Remaster recovers
1 cache GRD
resources
2
Build re-
SMON covery set
recovers Resource
3 claim
the
database Merge failed 4 Roll forward
redo threads recovery set
5
Recovery time
Full A G H
Database availability
Partial B F
2
1 2 3
None C D E
Elapsed time
UPDATE UPDATE
1 2
No block-level
COMMIT UPDATE
lock
Node1 Node2 Node1 Node2
Instance1 Instance2 Instance1 Instance2
3
4
SELECT resource_name,
current_utilization,max_utilization
FROM v$resource_limit
WHERE resource_name like 'g%s_%';
Execution
coordinator
Cluster
Node1 Noden
GV$INSTANCE
Instance1 Instancen
V$INSTANCE V$INSTANCE
Application server
ERP Run-time load balancing CRM
Service location transparency
Stop/Start service connections
Service connections
RAC Instances
ERP ERP Backup
ERP ERP
Priority
CRM CRM Alerts CRM CRM
Tuning
CRS
Up and down events notification engine
Restart failed components
clnode-2vip
2 Clients 2
ERP=(DESCRIPTION= ERP=(DESCRIPTION=
4 1 ((HOST=clusnode-1)) 5 1 ((HOST=clusnode-1vip))
6 ((HOST=clusnode-2)) 6 ((HOST=clusnode-2vip))
(SERVICE_NAME=ERP)) (SERVICE_NAME=ERP))
Timeout 7
5 wait 7
3
3
clnode-2vip
clnode-1 clnode-2 4 clnode-1vip
Cluster DB Control
Cluster Database DB
&
Node2 Rep
Cluster Database Home
Cluster Database Performance Instance2
Preinstallation Tasks
Several tasks must be completed before CRS and Oracle Database 10g software can be
installed. Some of these tasks are common to all Oracle database installations and should
be familiar to you. Others are specific to Oracle Database 10g RAC.
Attention to details here simplifies the rest of the installation process. Failure to complete
these tasks can certainly affect your installation and possibly force you to restart the
process from the beginning.
Hardware Requirements
The system must meet the following minimum hardware requirements:
• At least 512 megabytes of physical memory is needed. To determine the amount of
physical memory, enter the following command: grep MemTotal
/proc/meminfo
• A minimum of one gigabyte of swap space or twice the amount of physical memory
is needed. On systems with two gigabytes or more of memory, the swap space can be
between one and two times the amount of physical memory. To determine the size of
the configured swap space, enter the following command: grep SwapTotal
/proc/meminfo
• At least 400 megabytes of disk space must be available in the /tmp directory. To
determine the amount of disk space available in the /tmp directory, enter the
following command: df -k /tmp
• Up to four gigabytes of disk space is required for the Oracle Database 10g software,
depending on the installation type. The df command can be used to check for the
availability of the required disk space.
Network Requirements
Each node must have at least two network adapters: one for the public network interface
and the other for the private network interface or interconnect. In addition, the interface
names associated with the network adapters for each network must be the same on all
nodes.
For the public network, each network adapter must support TCP/IP. For the private
network, the interconnect must support UDP using high-speed network adapters and
switches that support TCP/IP. Gigabit Ethernet or an equivalent is recommended.
Before starting the installation, each node requires an IP address and an associated host
name registered in the DNS or the /etc/hosts file for each public network interface.
One unused virtual IP address and an associated virtual host name registered in the DNS or
the /etc/hosts file that you configure for the primary public network interface is
needed for each node. The virtual IP address must be in the same subnet as the associated
public interface. After installation, you can configure clients to use the virtual host name or
IP address. If a node fails, its virtual IP address fails over to another node.
For the private IP address and optional host name for each private interface, Oracle
recommends that you use private network IP addresses for these interfaces, for example,
10.*.*.* or 192.168.*.*. You can use the /etc/hosts file on each node to associate
private host names with private IP addresses.
Package Requirements
Depending on the products that you intend to install, verify that the packages listed in the
slide above are installed on the system. The OUI performs checks on your system to verify
that it meets the Linux package requirements of the cluster database and related services.
To ensure that these checks succeed, verify the requirements before you start the OUI. To
determine whether the required packages are installed, enter a command similar to the
following:
# rpm -q package_name
# rpm –qa |grep package_name_segment
For example, to check the gcc compatability packages, run the following command:
# rpm –qa |grep compat
compat-db-4.0.14.5
compat-gcc-7.3-2.96.122
compat-gcc-c++-7.3-2.96.122
compat-libstdc++-7.3-2.96.122
compat-libstdc++-devel-7.3-2.96.122
If a package is not installed, install it from your Linux distribution media as the root user
by using the rpm –i command. For example, to install the compat-db package, use the
following command:
# rpm –i compat-db-4.0.14.5.i386.rpm
$ cd
$ vi .bash_profile
umask 022
ORACLE_BASE=/u01/app/oracle; export ORACLE_BASE
TMP=/u01/mytmp; export TMP
TMPDIR=$TMP; export TMPDIR
Obtaining OCFS
Download OCFS for Linux in a compiled form from the following Web site:
http://oss.oracle.com/projects/ocfs/
In addition, you must download the following RPM packages:
• ocfs-support-1.0-n.i686.rpm
• ocfs-tools-1.0-n.i686.rpm
Also, download the RPM kernel module ocfs-2.4.21-4typeversion.rpm, where the
variable typeversion stands for the type and version of the kernel that is used. Use the
following command to find out which Red Hat kernel version is installed on your system:
uname -a
The alphanumeric identifier at the end of the kernel name indicates the kernel version that
you are running. Download the kernel module that matches your kernel version. For
example, if the kernel name that is returned with the uname command ends with -21.EL,
download the ocfs-2.4.9-21-EL-1.0.11-1.rpm kernel module.
Note: Ensure that you use the SMP or enterprise kernel that is shipped with Red Hat
Advanced Server 3.0 without any non-Red Hat supplied patches or customization. If you
modify the kernel, Oracle Corporation cannot support it.
# /usr/bin/ocfstool&
Starting ocfstool
Use the ocfstool utility to generate the /etc/ocfs.conf file. The ocfstool
utility is a graphical application. Therefore, you must be sure that your DISPLAY variable
is properly set. Start up ocfstool as shown in the following example:
# DISPLAY=:0.0
# export DISPLAY
# /usr/bin/ocfstool&
The OCFS Tool window appears in a new window. Click in the window to make it active,
and select the Generate Config option from the Tasks menu. The OCFS Generate Config
window is displayed.
# more S24ocfs
...
case "`basename $0`" in
*ocfs)
MODNAME=ocfs
FSNAME=OCFS
LOAD_OCFS=/sbin/load_ocfs
;;
*ocfs2)
MODNAME=ocfs2
FSNAME=OCFS2
LOAD_OCFS=/sbin/load_ocfs2
;;
...
esac...
2. Edit the
$ORACLE_BASE/oradata/dbname/dbname_raw.conf
file.
# cd $ORACLE_BASE/oradata/dbname/
# vi dbname_raw.conf
3. Set the DBCA_RAW_CONFIG environment variable to
specify the full path to this file.
$ /cdrom/crs/runInstaller
# cd /u01/app/oracle/oraInventory
# ./orainstRoot.sh
Cluster Configuration
The Cluster Configuration page displays predefined node information if the OUI detects
that your system has vendor clusterware. Otherwise, the OUI displays the Cluster
Configuration page without the predefined node information.
Node Names
• Vendor: Use vendor node names.
• Oracle: Use host names as returned by /bin/hostname:
# hostname
raclin01
• Private interconnect
Names or IP Addresses
• Names must be resolvable by every node by the DNS or /etc/hosts.
• Names must exist and be on the same subnet.
# cat /etc/inittab
# Run xdm in runlevel 5
x:5:respawn:/etc/X11/prefdm -nodaemon
h1:35:respawn:/etc/init.d/init.evmd run >/dev/null
2>&1 </dev/null
h2:35:respawn:/etc/init.d/init.cssd fatal >/dev/null
2>&1 </dev/null
h3:35:respawn:/etc/init.d/init.crsd run >/dev/null
2>&1 </dev/null
$ id
oracle
$ /cdrom/dbs/runInstaller
Check Summary
The Summary page is displayed next. Review the information on this page. Node information
and space requirements can be viewed here, as well as selected software components. If you are
satisfied with the summary, click the Install button to proceed. If you are not, you can click the
Back button to go back and make the appropriate changes.
On the Install page, you can monitor the progress of the installation. During the installation, the
OUI copies the software first to the local node and then to the remote nodes.
# cd /u01/app/oracle/product/10.1.0/db_1
# ./root.sh
Installation Progress
When you click the Finish button, a progress dialog box appears while the VIPCA configures
the virtual IP addresses with the network interfaces that you selected. The VIPCA then creates
and starts the VIPs, GSD, and ONS node applications. When the configuration completes, click
OK to view the VIPCA configuration results. Review the information on the Configuration
Results page, taking care to review any errors that may have been reported. After reviewing the
configuration results, click the Exit button to exit the VIPCA.
End of Installation
After running root.sh on all the nodes as described in previous slides, click the OK button in
the OUI dialog box to continue the installation. This enables the remaining Oracle configuration
assistants to run, so that the assistants can perform postinstallation processing.
The Network Configuration Assistant (NETCA) runs next to configure listeners for each node in
the cluster. If you have chosen to create the database as described earlier in the Select Database
Configuration slide, the DBCA is automatically launched to perform database creation. When
the configuration assistants stop running, the End of Installation page appears. You can now
click the Exit button to exit the OUI and start the DBCA to create your database.
$ cd /u01/app/oracle/product/10.1.0/db_1/bin
$ ./dbca –datafileDestination /ocfs/oradata
Node Selection
The Node Selection page is displayed next. Because you are creating a cluster database, choose
all the nodes. Click the Select All button to choose all the nodes of the cluster. Each node must
be highlighted before continuing. If all the nodes do not appear, you must stop the installation
and troubleshoot your environment.
The most common reason for encountering an error here is related to problems with the GSD
node application. If no problems are encountered, click the Next button to proceed.
Database Identification
On the Database Identification page, you must enter the database name in the Global Database
Name field. A global database name includes the database name and database domain such as
racdb.oracle.com. The name that you enter on this page must be unique among all the
global database names used in your environment. The global database name can be up to 30
characters in length and must begin with an alphabetical character.
A system identifier (SID) prefix is required, and the DBCA suggests a name based on your
global database name. This prefix is used to generate unique SID names for the two instances
that make up the cluster database. For example, if your prefix is RACDB, the DBCA creates two
instances on node 1 and node 2, called RACDB1 and RACDB2, respectively. This example
assumes that you have a two-node cluster. If you do not want to use the system-supplied prefix,
enter a prefix of your choice. The SID prefix must begin with an alphabetical character and
contain no more than 5 characters on UNIX-based systems or 61 characters on Windows-based
systems. Click the Next button to continue.
Database Components
The Database Content page has two tabs. On the Database Components page, you can select
the components to configure for use in your database. If you choose the Custom Database
option, you can select or deselect the database components and their corresponding assigned
tablespaces. Select the check box next to each component that you want to install, and select a
tablespace from the drop-down list for the product, if you want to install it somewhere other than
the default tablespace that is shown. For a seed database, you can select whether to include the
sample schemas in your database.
The Custom Scripts page enables you to browse and choose scripts to be executed after your
database has been created. After selecting components, click the Next button to continue.
Database Services
On the Database Services page, you can add database services to be configured during database
creation. To add a service, click the Add button at the bottom of the Database Services section.
Enter a service name in the Add a Service dialog box, and then click OK to add the service and
return to the Database Services page. The new service name appears under the global database
name. Select the service name. The DBCA displays the service preferences for the service on the
right of the DBCA Database Services page. Change the instance preference (Not Used,
Preferred, or Available) as needed.
Go to the Transparent Application Failover (TAF) policy row at the bottom of the page. Make a
selection in this row for your failover and reconnection policy preference as described in the
following list:
• None: Do not use TAF.
• Basic: Establish connections at failover time.
• Pre-connect: Establish one connection to a preferred instance and another connection to a
backup instance that you have selected to be available.
In the example in the slide, the Pre-connect policy has been chosen. When you have finished
adding and configuring services, click the Next button to continue.
Note: For more information about services and TAF, refer to the lessons titled “Services” and
“High Availability of Connections” in this course.
Initialization Parameters
On the Initialization Parameters page, you can set important database parameters. The
parameters are grouped under four tabs:
• Memory
• Sizing
• Character Sets
• Connection Mode
On the Memory page, you can set parameters that deal with memory allocation, including
shared pool, buffer cache, Java pool, large pool, and PGA size. On the Sizing page, you can
adjust the database block size. Note that the default is eight kilobytes. In addition, you can set
the number of processes that can connect simultaneously to the database.
By clicking the Character Sets tab, you can change the database character set. You can also
select the default language and the date format. On the Connection Mode page, you can choose
the connection type that clients use to connect to the database. The default type is Dedicated
Server Mode. If you want to use Oracle Shared Server, click the Shared Server Mode button.
If you want to review the parameters that are not found in the four tabs, click the All
Initialization Parameters button. After setting the parameters, click the Next button to
continue.
Monitor Progress
The Progress Monitor page appears next. In addition to informing you about how fast the
database creation is taking place, it also informs you about the specific tasks being performed by
the DBCA in real time. These tasks include:
• Creating the RAC data dictionary views
• Configuring the network for the cluster database
• Starting the listeners and database instances and the high-availability services
When the database creation progress reaches 100 percent, the DBCA displays a dialog box
announcing the completion of the creation process. It also directs you to the installation log file
location, parameter file location, and the Enterprise Manager URL. By clicking the Password
Management button, you can manage the database accounts created by the DBCA.
$ cd $ORACLE_HOME
$ cp root.sh root.sh.bak
Postinstallation Tasks
After the cluster database has been successfully created, you must run the following command to
verify the Enterprise Manager/Oracle Cluster Registry configuration in your newly installed
RAC environment:
$ srvctl config database -d db_name
Server Control (SRVCTL) displays the name of the node and the instance for the node. The
following example shows a node named raclin01 running an instance named racdb1.
Execute the following command:
$ srvctl config database -d racdb
raclin01 racdb1 /u01/app/.../db_1
raclin02 racdb2 /u01/app/.../db_1
It is also recommended that you back up the root.sh script after you complete an installation.
If you install other products in the same Oracle Home directory, the OUI updates the contents of
the existing root.sh script during the installation. If you require information contained in the
original root.sh script, you can recover it from the root.sh file copy.
You must also add and set up additionally required user accounts. For information about setting
up additional optional user accounts, refer to the Administrator’s Guide for UNIX Systems.
• start/stop syntax:
srvctl start|stop instance -d <db_name> -i <inst_name_list>
[-o open|mount|nomount|normal|transactional|immediate|abort>]
[-c <connect_str> | -q]
• Examples:
$ srvctl start instance -d RACDB -i RACDB1,RACDB2
SCOPE=MEMORY
SCOPE=SPFILE
SCOPE=BOTH
• ACTIVE_INSTANCE_COUNT
• ARCHIVE_LAG_TARGET
• CLUSTER_DATABASE
• CONTROL_FILES
• DB_BLOCK_SIZE
• DB_DOMAIN
• DB_FILES
• DB_NAME
• DB_RECOVERY_FILE_DEST
• DB_RECOVERY_FILE_DEST_SIZE
• DB_UNIQUE_NAME
• MAX_COMMIT_PROPAGATION_DELAY
• TRACE_ENABLED
• UNDO_MANAGEMENT
• Instance Settings
– THREAD
– ROLLBACK_SEGMENTS
– INSTANCE_NUMBER
– UNDO_TABLESPACE
(When using automatic undo management)
• Environment Variables
– ORACLE_SID
RACDB1
$ cd $ORA_CRS_HOME/oui/bin
$ ./addNode.sh
Viewing Alerts
When an alert that requires a closer look is raised, you can click it for more information.
There is a statistic summary for the metric displayed to the left of the window. Here you
can find information such as high, low, and average values of the metric over the duration
of the polling period, how many times the threshold exceeded the warning and critical
thresholds, and the current values of those thresholds. If you want to adjust the warning or
threshold values, click the Manage Metrics link at the bottom of the page. In addition to
adjusting the threshold values, you can also define a response action that will be used when
the threshold is exceeded.
Operating system
ASM
Database
disk group
File system
Extent
file Allocation unit
or
raw device
Oracle
block Physical
block
DB tom=ant tom=bee DB
Instance dick=ant dick=bee Instance
harry=ant harry=bee
SID=sales SID=sales
DBW0 ASMB ASMB DBW0
ASM disks ASM disks ASM disks ASM disks ASM disks ASM disks
ASM Disk group Tom ASM Disk group Dick ASM Disk group Harry
ASMLibs
ASMLib is a support library for the ASM feature. The objective of ASMLib is to provide a more
streamlined and efficient mechanism for identifying and accessing block devices used by ASM
disk groups. This API serves as an alternative to the standard operating system interface.
The ASMLib kernel driver is released under the GNU General Public License (GPL), and Oracle
Corporation freely delivers an ASMLib for Linux platforms. This library is provided to enable
ASM I/O to Linux disks without the limitations of the standard UNIX I/O API.
The main ASMLib functions are grouped into three collections of functions:
• Device discovery functions must be implemented in any ASMLib. Discover strings usually
contain a prefix identifying which ASMLib this discover string is intended for. For the
Linux ASMLib provided by Oracle, the prefix is ORCL:.
• I/O processing functions extend the operating system interface and provide an optimized
asynchronous interface for scheduling I/O operations and managing I/O operation
completion events. These functions are implemented as a device driver within the
operating system kernel.
• The performance and reliability functions use the I/O processing control structures for
passing metadata between the Oracle database and the back-end storage devices. They
enable additional intelligence on the part of back-end storage.
Note: The database can load multiple ASMLibs, each handling different disks.
• ASM instance
0010
• Files 0010
ASM
instance
Database
instance
INSTANCE_TYPE = ASM
DB_UNIQUE_NAME = +ASM
ASM_POWER_LIMIT = 1
ASM_DISKSTRING = '/dev/rdsk/*s2', '/dev/rdsk/c1*'
ASM_DISKGROUPS = dgroupA, dgroupB
LARGE_POOL_SIZE = 8MB
ASM
AS SYSDBA
instance
All operations
Storage system
V$ASM_TEMPLATE
V$ASM_CLIENT V$ASM_DISKGROUP
V$ASM_FILE
V$ASM_ALIAS
Storage system
V$ASM_DISK
V$ASM_OPERATION
$ sqlplus /nolog
SQL> CONNECT / AS sysdba
Connected to an idle instance.
SQL> STARTUP;
ASM instance started
Total System Global Area 147936196 bytes
Fixed Size 324548 bytes
Variable Size 96468992 bytes
Database Buffers 50331648 bytes
Redo Buffers 811008 bytes
ASM diskgroups mounted
SHUTDOWN DB instance
immediately aborted
DB
ASM ASM
• ASM instance
0010
• Files 0010
Disk group
5
4
3
1 7 13 1 7 13 1 7 13
1 7 13 1 7 13 1 7 13
1 7 13 1 7 13 1 7 13
Disk group A
Failure Group
A failure group is a set of disks, inside one particular disk group, sharing a common resource
whose failure needs to be tolerated. An example of a failure group is a string of SCSI disks
connected to a common SCSI controller. A failure of the controller leads to all of the disks on its
SCSI bus becoming unavailable, although each of the individual disks is still functional.
What constitutes a failure group is site specific. It is largely based on failure modes that a site is
willing to tolerate. By default, ASM assigns each disk to its own failure group. When creating a
disk group or adding a disk to a disk group, administrators can specify their own grouping of
disks into failure groups. After failure groups are identified, ASM can optimize file layout to
reduce the unavailability of data due to the failure of a shared resource.
• Mirror at AU level
• Mix primary and mirror
AUs on each disk
• External redundancy:
Defers to hardware
mirroring
• Normal redundancy:
– Two-way mirroring
– At least two failure groups
• High redundancy:
– Three-way mirroring
– At least three failure groups
• Automatic online
rebalancing whenever
storage configuration
changes
• Only move data
proportional to
storage added
• No need for manual
I/O tuning
• Online migration to
new storage
Disk formatting
• ASM instance
0010
• Files 0010
Database file
RMAN 1 Automatic
2 ASM file
3 management
Mandatory
4
for backups
1 2 3 4
ASM Files
ASM files are Oracle database files stored in ASM disk groups. When a file is created, certain
file attributes are permanently set. Among these are its protection policy and its striping policy.
ASM files are Oracle-managed files. Any file that is created by ASM is automatically deleted
when it is no longer needed. However, ASM files that are created by specifying a user alias are
not considered Oracle-managed files. These files are not automatically deleted.
When ASM creates a data file for a permanent tablespace (or a temp file for a temporary
tablespace), the data file is set to auto-extensible with an unlimited maximum size and 100 MB
default size. An AUTOEXTEND clause may override this default.
All circumstances where a database must create a new file allow for the specification of a disk
group for automatically generating a unique file name.
With ASM, file operations are specified in terms of database objects. Administration of
databases never requires knowing the name of a file, though the name of the file is exposed
through some data dictionary views or the ALTER DATABASE BACKUP CONTROLFILE TO
TRACE command.
Because each file in a disk group is physically spread across all disks in the disk group, a backup
of a single disk is not useful. Database backups of ASM files must be made with RMAN.
Note: ASM does not manage binaries, alert logs, trace files, password files, or Cluster Ready
Services files.
ASM
file name
Single-file Multiple-file
Reference
creation creation
Incomplete
Fully Alias with
Numeric Alias Incomplete with
qualified template
template
1 +<group>/<dbname>/<file_type>/<tag>.<file#>.<incarnation#>
2 +<group>.<file#>.<incarnation#>
3 +<group>/<directory1>/…/<directoryn>/<file_name>
4 +<group>/<directory1>/…/<directoryn>/<file_name>(<temp>)
5 +<group>
6 +<group>(<temp>)
SELECT name
FROM V$ASM_ALIAS
WHERE parent_index = :alias_id;
Retrieving Aliases
Assume that you want to retrieve all aliases that are defined inside the previously defined
directory +dgroupA/mydir. You can traverse the directory tree, as shown in the example.
The REFERENCE_INDEX number can be used only for entries that are directory entries in the
alias directory. For nondirectory entries, the reference index is set to zero. The example retrieves
REFERENCE_INDEX numbers for each subdirectory and uses the last REFERENCE_INDEX as
the PARENT_INDEX of needed aliases.
…
INSTANCE_TYPE = RDBMS
LOG_ARCHIVE_FORMAT
DB_BLOCK_SIZE
DB_CREATE_ONLINE_DEST_n
DB_CREATE_FILE_DEST_n
DB_RECOVERY_FILE_DEST
CONTROL_FILES
LOG_ARCHIVE_DEST_n
LOG_ARCHIVE_DEST
STANDBY_ARCHIVE_DEST
…
CONNECT TARGET
SQL "ALTER TABLESPACE tbsname OFFLINE";
BACKUP AS COPY TABLESPACE tbsname FORMAT '+DGROUP1';
SWITCH TABLESPACE tbsname TO COPY;
SQL "ALTER TABLESPACE tbsname ONLINE";
RMAN
Migrate
RMAN
Migrate
ASM to ASM
DBMS_FILE_TRANSFER
ASM Scalability
ASM imposes the following limits:
• 63 disk groups in a storage system
• 10,000 ASM disks in a storage system
• 4 petabyte maximum storage for each ASM disk
• 40 exabyte maximum storage for each storage system
• 1 million files for each disk group
• 2.4 terabyte maximum storage for each file
Node1 Node2
RAC01 RAC02
Shared storage
Thread 1
ALTER DATABASE ADD LOGFILE THREAD 2 GROUP 4;
ALTER DATABASE ADD LOGFILE THREAD 2 GROUP 5;
Consistent reads
Transaction recovery
Shared storage
… SPFILE
undotbs3 undotbs2
RAC01.UNDO_TABLESPACE=undotbs3
RAC02.UNDO_TABLESPACE=undotbs2
undotbs1
…
Day time
HR DW CRM Batch
Idle
DW HR Batch
CRM
Idle Idle
DW Batch
Batch
HR DW
CRM HR CRM
What Is a Service?
The concept of a service was first introduced in Oracle8i as a means for the listener to do
connection load balancing between nodes and instances of a cluster. However, the concept,
definition, and implementation of services have been dramatically expanded. Services are a
feature for workload management that organizes the universe of work execution within the
database to make that work more manageable, measurable, tunable, and recoverable. A service is
a grouping of related tasks within the database with common functionality, quality expectations,
and priority relative to other services. The notion of service provides a single-system image for
managing competing applications running within a single instance and across multiple instances
and databases.
Using standard interfaces, such as DBCA, Enterprise Manager, and SRVCTL, services can be
configured, administered, enabled, disabled, and measured as a single entity.
Services provide availability. Following outages, a service is recovered fast and automatically at
surviving instances.
Services provide a new dimension to performance tuning. With services, workloads are visible
and measurable. Tuning by “service and SQL” replaces tuning by “session and SQL” in the
majority of systems where sessions are anonymous and shared.
Services are dynamic in that the number of instances a service runs on can be augmented when
load increases, and reduced when load declines. This dynamic resource allocation enables a cost-
effective solution for meeting demands as they occur.
Oracle Database 10g: Real Application Clusters 7-5
High Availability of Services in RAC
Active/Spare
RAC01 RAC02 RAC03
AP AP
GL GL
Active/Symmetric Active/Asymmetric
RAC01 RAC02 RAC03 RAC01 RAC02 RAC03
AP AP AP AP AP AP
GL GL GL GL GL GL
• Single instance:
– Global unique name
– Threshold
– Priority
• RAC:
– Global unique name
– Threshold
– Priority
– High-availability configuration
– Preconnection
Service Attributes
Each service has the following attributes:
• Globally unique name that identifies the service in the local cluster and globally for data
guard
• Quality of service thresholds for response time and CPU consumption
• Priority relative to other services, defined in terms of either ratio of resource consumption
or priority
In a RAC environment, services have two additional attributes:
• High-availability configuration: A description of how to distribute the service across
instances when the system first starts.
• Preconnection: Definition of a corresponding preconnected service, also called a shadow
service. The preconnect service spans the set of instances that are available to support a
service in the event of a failure. When a service is added by using the Database
Configuration Assitant (DBCA) or Server Control (SRVCTL), the preconnect service is
created automatically and is then managed by CRS. This service name
<SERVICE>_PRECONNECT is used in the backup clause for transparent application
failover (TAF) connect descriptors for directly connected applications that are using the
preconnected TAF feature. Preconnect services are non-overlapping with their matching
active services. This feature eliminates sessions looping back to the same instance as the
original, and enables load balancing for both active and preconnected sessions with TAF.
Oracle Database 10g: Real Application Clusters 7-8
Service Types
• Application services
• Internal services:
– SYS$BACKGROUND
– SYS$USERS
• Limit of 64 services per database:
– 62 application services
– 2 internal services
Service Types
Oracle Database 10g supports two broad types of services: application services and internal
services. Application services are mainly functional maps to workloads. Sessions doing work for
a common business function are grouped together. For Oracle Applications, AP, AR, GL, MFG,
WIP, BOM, and so on create a functional division of work within the database and can thus be
categorized as services.
In addition to application services, the RDBMS also supports two internal services.
SYS$BACKGROUND is used by the background processes only. SYS$USERS is the default
service for user sessions that are not associated with any application service. Both internal
services support all the workload management features and neither one can be stopped or
disabled.
There is a limitation of 64 services per database, 62 application services, and 2 internal services.
Also, a service name is restricted to 64 characters.
Note: Shadow services are also included in the application service category. In addition, a
service is also created for each Advanced Queue created.
Creating Services
Like other database objects, services are maintained and tracked through the data dictionary and
dynamic performance views.
Each service has a unique name that identifies it locally in the cluster and globally for Data
Guard.
For single-instance Oracle, services can be created with the DBMS_SERVICE package.
Services are also created implicitly at startup of the instance according to the values set for the
SERVICE_NAMES initialization parameter.
For high-availability features in RAC environments, services should be defined either by the
DBCA, or by the command-line tool SRVCTL. This definition process implicitly creates high-
availability business rules that are managed automatically by CRS to keep the services available.
The high-availability business rules are kept in the OCR.
RAC02
AP GL AP GL
RAC01
1 2
RAC01 RAC02 RAC03 RAC04 RAC01 RAC02 RAC03 RAC04
4 3
ERP=(DESCRIPTION=
(LOAD_BALANCE=on)
(ADDRESS=(PROTOCOL=TCP)(HOST=node-1vip)(PORT=1521))
(ADDRESS=(PROTOCOL=TCP)(HOST=node-2vip)(PORT=1521))
(ADDRESS=(PROTOCOL=TCP)(HOST=node-3vip)(PORT=1521))
(ADDRESS=(PROTOCOL=TCP)(HOST=node-4vip)(PORT=1521))
(CONNECT_DATA=(SERVICE_NAME=ERP)))
url="jdbc:oracle:oci:@ERP"
url="jdbc:oracle:thin:@ERP"
AP Instance resources
Connections AP 75%
exec DBMS_RESOURCE_MANAGER.CREATE_PENDING_AREA;
exec DBMS_RESOURCE_MANAGER.CREATE_CONSUMER_GROUP(
CONSUMER_GROUP => 'HIGH_PRIORITY',
COMMENT => 'High priority consumer group');
exec DBMS_RESOURCE_MANAGER.SET_CONSUMER_GROUP_MAPPING(
ATTRIBUTE => DBMS_RESOURCE_MANAGER.SERVICE_NAME,
VALUE => 'AP',
CONSUMER_GROUP => 'HIGH_PRIORITY');
exec DBMS_RESOURCE_MANAGER.SUBMIT_PENDING_AREA;
exec -
DBMS_RESOURCE_MANAGER_PRIVS.GRANT_SWITCH_CONSUMER_GROUP(-
GRANTEE_NAME => 'PUBLIC',
CONSUMER_GROUP => 'HIGH_PRIORITY',
GRANT_OPTION => FALSE);
Database
Job table
Job1 HOT_BATCH_CLASS HOT_BATCH_SERV
Job2 HOT_BATCH_CLASS HOT_BATCH_SERV
Job3 LOW_BATCH_CLASS LOW_BATCH_SERV
DBMS_SCHEDULER.CREATE_JOB_CLASS(
JOB_CLASS_NAME => 'HOT_BATCH_CLASS',
RESOURCE_CONSUMER_GROUP => NULL ,
SERVICE => 'HOT_BATCH_SERV' ,
LOGGING_LEVEL => DBMS_SCHEDULER.LOGGING_RUNS,
LOG_HISTORY => 30, COMMENTS => 'P1 batch');
DBMS_SCHEDULER.CREATE_JOB(
JOB_NAME => 'my_report_job',
JOB_TYPE => 'stored_procedure',
JOB_ACTION => 'my_name.my_proc();',
NUMBER_OF_ARGUMENTS => 4, START_DATE => SYSDATE+1,
REPEAT_INTERVAL => 5, END_DATE => SYSDATE+30,
JOB_CLASS => 'HOT_BATCH_CLASS', ENABLED => TRUE,
AUTO_DROP => false, COMMENTS => 'daily status');
ERP
ERP ERP
Execution
coordinator
exec DBMS_SERVER_ALERT.SET_THRESHOLD(-
METRICS_ID => dbms_server_alert.elapsed_time_per_call,
WARNING_OPERATOR => dbms_server_alert.operator_ge,
WARNING_VALUE => '500000',
CRITICAL_OPERATOR => dbms_server_alert.operator_ge,
CRITICAL_VALUE => '750000',
OBSERVATION_PERIOD => 15,
CONSECUTIVE_OCCURRENCES => 3,
INSTANCE_NAME => 'I0n',
OBJECT_TYPE => dbms_server_alert.object_type_service,
OBJECT_NAME => 'ERP');
TRCSESS TRCSESS
Report
file
Managing Services
Depending on the type of management tasks that you want to perform, you can use Enterprise
Manager, DBCA, or SRVCTL.
The following is the description of the management tasks related to services in a RAC
environment:
• Disabling a service is used to disable a specified service on all or specified instances. The
disable state is used when a service is down for maintenance to prevent inappropriate
automatic CRS restarts. Disabling an entire service affects all the instances by disabling
each one.
• Enabling a service is used to enable a service to run under CRS for automatic restart and
redistribution. You can enable a service even if that service is stopped. Enable is the
default value when a service is created. If the service is already enabled, then the command
is ignored. Enabled services can be started, and disabled services cannot be started.
Enabling an entire service affects the enabling of the service over all the instances by
enabling the service at each one.
• Starting a service is used to start a service or multiple services on the specified instance.
Only enabled services can be started. The command fails if you attempt to start a service
on an instance and if the number of instances that are currently running the service already
reaches its cardinality.
ERP =
(DESCRIPTION =
(LOAD_BALANCE=ON)
(ADDRESS_LIST =
(ADDRESS=(PROTOCOL=TCP)(HOST=node1vip)(PORT=1521))
(ADDRESS=(PROTOCOL=TCP)(HOST=node2vip)(PORT=1521))
)
(CONNECT_DATA=(SERVICE_NAME=ERP)))
Random
access
node1 node2
ERP =
(DESCRIPTION =
(LOAD_BALANCE=ON)
(FAILOVER=ON)
(ADDRESS_LIST = 3
(ADDRESS=(PROTOCOL=TCP)(HOST=node1vip)(PORT=1521))
(ADDRESS=(PROTOCOL=TCP)(HOST=node2vip)(PORT=1521))
) 4
(CONNECT_DATA=(SERVICE_NAME=ERP)))
node2vip
2 node1vip
ERP = (DESCRIPTION=(LOAD_BALANCE=ON)(FAILOVER=ON)
(ADDRESS_LIST=
(ADDRESS=(PROTOCOL=TCP)(HOST=node1vip)(PORT=1521))
(ADDRESS=(PROTOCOL=TCP)(HOST=node2vip)(PORT=1521)))
(CONNECT_DATA=(SERVICE_NAME=ERP)))
6 4 3 2
ERP started on both instances
Listener Listener
1
5 1 1
PMON PMON
*.REMOTE_LISTENERS=RACDB_LISTENERS
Node1 Node2
RACDB_LISTENERS=
(DESCRIPTION=
(ADDRESS=(PROTOCOL=tcp)(HOST=node1vip)(PORT=1521))
(ADDRESS=(PROTOCOL=tcp)(HOST=node2vip)(PORT=1521)))
Proxy
SNMP HA
SNMP app ONS
console Events
Callout HA HA DB
e-mail SMTP script CRS EMD
Events Events Control
syslog Callout
exec
Remote
System trouble ticket
logs
Node1 repository
<Event_Type>
VERSION=<n.n>
[service=<serviceName.dbDomainName>]
[database=<dbName>] [instance=<sid>]
[host=<hostname>]
status=<Event_Status>
reason=<Event_Reason>
[card=<n>]
timestamp=<eventDate> <eventTime>
Midtier1
localport=6100
remoteport=6200 ONS
useocr=on
ONS ONS
OCR
ons.config 1 1 ons.config
Node1 Node2
Midtier1
localport=6100
remoteport=6200 1
nodes=node1:6200,node2:6200
ONS ONS
OCR
ons.config ons.config
Node1 Node2
Midtier1
Service Service or node
ONS
UP event DOWN event
JDBC ICC
Event
handler
Connections Connections
reconnected Connection Cache
marked down
Connections Connections
using Listeners using
Service names Service names
Connections
load balancing
ONS ……… ONS
Node1 Noden
2 Application 2 Application
OCI Library 6 OCI Library 6
5 3 7 5 3
8 7
AP 3 ERP 3
7 3
1 1
AP ERP ERP_PRECONNECT
AP =
(DESCRIPTION =(FAILOVER=ON)(LOAD_BALANCE=ON)
(ADDRESS=(PROTOCOL=TCP)(HOST=N1VIP)(PORT=1521))
(ADDRESS=(PROTOCOL=TCP)(HOST=N2VIP)(PORT=1521))
(CONNECT_DATA =
(SERVICE_NAME = AP)
(FAILOVER_MODE =
(TYPE=SESSION)
(METHOD=BASIC)
(RETRIES=180)
(DELAY=5))))
ERP =
(DESCRIPTION =(FAILOVER=ON)(LOAD_BALANCE=ON)
(ADDRESS=(PROTOCOL=TCP)(HOST=N1VIP)(PORT=1521))
(ADDRESS=(PROTOCOL=TCP)(HOST=N2VIP)(PORT=1521))
(CONNECT_DATA = (SERVICE_NAME = ERP)
(FAILOVER_MODE = (BACKUP=ERP_PRECONNECT)
(TYPE=SESSION)(METHOD=PRECONNECT))))
ERP_PRECONNECT =
(DESCRIPTION =(FAILOVER=ON)(LOAD_BALANCE=ON)
(ADDRESS=(PROTOCOL=TCP)(HOST=N1VIP)(PORT=1521))
(ADDRESS=(PROTOCOL=TCP)(HOST=N2VIP)(PORT=1521))
(CONNECT_DATA = (SERVICE_NAME = ERP_PRECONNECT)))
TAF Verification
To determine whether TAF is correctly configured and that connections are associated with a
failover option, you can examine the V$SESSION view. To obtain information about the
connected clients and their TAF status, examine the FAILOVER_TYPE, FAILOVER_METHOD,
FAILED_OVER, and SERVICE_NAME columns. The example includes one query that you
could execute to verify that you have correctly configured TAF.
The above example is based on the previously configured AP and ERP services, and their
corresponding connection descriptors.
The first output in the slide is the result of the execution of the query on the first node after two
SQL*Plus sessions from the first node have connected to the AP and ERP services respectively.
The output shows that the AP connection ended up on the first instance. Because of the load-
balancing algorithm, it can end up on the second instance. Alternatively, the ERP connection
must end up on the first instance because it is the only preferred one.
The second output is the result of the execution of the query on the second node before any
connection failure. Note that there is currently one unused connection established under the
ERP_PROCONNECT service which is automatically started on the ERP available instance.
The third output is the one corresponding to the execution of the query on the second node after
the first instance failure. A second connection has been created automatically for the AP service
connection, and the original ERP connection is now using the preconnected connection.
ALTER SYSTEM 2
1
ENABLE RESTRICTED SESSION;
RAC01 RAC02
CRS 4
ERP 3 ERP
Archived Archived
log files log files
Mirrored Database
disks backups
RECORDS_TOTAL RECORDS_USED
------------- ------------
500 500
Initiate Archiving
To enable archive logging, your database must be mounted, but not open, by an exclusive
instance. If you are using an SPFILE, you must first create SID-specific entries for this
instance; otherwise, build a special-purpose text parameter file. The parameters that you
must set for this exclusive instance include the following:
• CLUSTER_DATABASE: Set to FALSE
• LOG_ARCHIVE_DEST_n: Up to 10 values, depending on your archive strategy
• LOG_ARCHIVE_FORMAT: Include the %t or %T parameter for thread identification
• LOG_ARCHIVE_START: Set to TRUE
You can now enable archiving for your database by following these steps:
1. Shut down all instances.
2. Start up your exclusive instance.
3. Enter the following command: ALTER DATABASE ARCHIVELOG
4. Modify the LOG_ARCHIVE_* parameters for each of the other instances.
5. Shut down the exclusive instance and reset its CLUSTER_DATABASE parameter.
6. Restart all of your instances using the modified parameters.
Flash
recovery
area
Shared NFS
Cluster file system
directory
ASM
User-
managed • For offline backup,
backup shut down all
With- instances
out With
arch- arch- • Can use multiple
iving iving nodes for parallel
• Full offline • Full or backup
backups partial,
only
• Can convert
offline or
• Recovery online
tablespace between
to point of backups backup modes from
last • Recovery any instance
backup to point of
• Provide access to all
failure
threads of archived
redo
Copyright © 2003, Oracle. All rights reserved.
Distribution of Backups
When configuring the backup options for RAC, you have three possible configurations:
• Network backup server: A dedicated backup server performs and manages backups
for the cluster and the cluster database. None of the nodes have local backup
appliances.
• One local drive: One node has access to a local backup appliance and performs and
manages backups for the cluster database. All nodes of the cluster should be on a
cluster file system to be able to read all datafiles, archived redo logs, and SPFILEs. It
is recommended that you do not use the non-cluster file system archiving scheme if
you have backup media on only one local drive.
• Multiple drives: Each node has access to a local backup appliance and can write to its
own local backup media.
In the cluster file system scheme, any node can access all the datafiles, archived redo logs,
and SPFILEs. In the non-cluster file system scheme, you must write the backup script so
that the backup is distributed to the correct drive and path for each node. For example, node
1 can back up the archived redo logs whose path names begin with /arc_dest_1, node 2
can back up the archived redo logs whose path names begin with /arc_dest_2, and node 3
can back up the archived redo logs whose path names begin with /arc_dest_3.
CPU
time
Possibly Scalable
needs SQL application
tuning
Wait
time
RAC-Specific Tuning
Although there are specific tuning areas for RAC, such as interconnect traffic, you get most
benefits by tuning your system like a single-instance system. At least, this must be your
starting point.
Obviously, if you have serialization issues in a single-instance environment, these may be
exacerbated with RAC.
As shown in the slide, you have basically the same tuning tools with RAC as with a single-
instance system. However, certain combinations of specific wait events and statistics are
well-known RAC tuning cases.
In the remaining of this lesson, you see some of those specific combinations, as well as the
RAC-specific information that you can get from the Database Control performance pages,
and Statspack and AWR reports. Finally, you see the RAC-specific information that you can
get from the Automatic Database Diagnostic Monitor (ADDM).
gc buffer busy
Block arrival time
less than buffer pin time
Wait:
gc current block request
1
LMS
Direct send
SGA2 2
LGWR:
LGWR
Log sync
SGA1
Block transfer
LMS
Wait complete: 3
gc current block 2-way
Wait:
gc current block request
1 LMS 2
LMS Resource
Direct
Master
message
SGA2 3
LGWR
SGA1
LMS
Block transfer
Wait complete: 4
gc current block 3-way
Wait:
gc current block request
1 LMS
LMS Resource
Direct
Master
message
SGA2
2
SGA1
3
Grant message
Wait complete:
gc current grant 2-way
Wait:
gc current block request
1
LMS
Block request
SGA2 2
LGWR:
LGWR
Log sync
SGA1
Side channel message
4
Wait complete:
gc current failure
LMS
Block 3
send
TX TM
US HW
TA SQ
Wait events
Index
enq: TX - index
contention block
Split in
gc buffer busy
progress
gc current block busy
gc current split
System statistics
Leaf node splits
Branch node splits
Exchange deadlocks
gcs refuse xid
gcs ast xid
Service ITL waits RAC01 RAC02
1…50000 50001…100000
RAC01 RAC02
Index Changes
Reads
SGA1 SGA2
Undo Undo
Additional
interconnect traffic
Wait events
enq: HW -
contention Heavy
gc current grant inserts
HWM
Heavy
inserts
New extent
RAC01 RAC02
External clients
EM SQL*Plus …
SGA
Efficient V$ DBA_*
in-memory AWR
statistics snapshots
collection MMON
Self-tuning … Self-tuning
ADDM
Internal clients component component
AWR Tables
DBA_HIST_*
AWR Tables
AWR contains two types of tables:
• Metadata tables: Are used to control, process, and describe AWR tables. For example,
the Oracle database uses metadata tables to determine when to perform snapshots, and
what to capture to the disk. Also, metadata tables contain the mapping between the
snapshot_id and the corresponding wall clock time.
• Historical statistic tables: Store historical statistical information about the Oracle
database. Each snapshot is a capture of the in-memory database statistics data at a
certain point in time.
All names of AWR tables are prefixed with WRx$, where x specifies the kind of table:
• WRM$ tables store metadata information for AWR.
• WRH$ tables store historical data or snapshots.
You can use dictionary views to query AWR data. Any view related to historical
information in AWR has the DBA_HIST_suffix.
AWR uses partitioning for efficient querying and purging of data. The snapshot tables are
organized into the following categories: File Statistics, General System Statistics,
Concurrency Statistics, Instance Tuning Statistics, SQL Statistics, Segment Statistics, Undo
Statistics, Time-Model Statistics, Recovery Statistics, and RAC Statistics.
MMON Coordinator
In-memory
statistics
SYSAUX
SGA (Inst1)
AWR tables
… 6:00 a.m.
9:00 a.m. 7:00 a.m.
8:00 a.m.
MMON 9:00 a.m.
In-memory
statistics
SGA (Instn)
Statspack AWR
schema schema
Migration
60 minutes
MMON
In-memory
statistics
Snapshots
SGA
ADDM ADDM
results
EM AWR
ADDM
results
… …
System wait
Concurrency Parse latches …
I/O waits
…
Nonproblem areas
Hot objects
ADDM
Instance LMS
contentions congestions
Top SQL
ADDM Recommendations
On the Performance Finding Details page, you are given recommendations to solve the
corresponding issue. Recommendations are divided into many categories such as Schema,
SQL Tuning, DB configuration, and so on. The Benefit(%) column gives you the
maximum reduction in database elapse time if the recommendation is implemented.
The ADDM considers a variety of changes to a system, and its recommendations can
include:
• Hardware changes: Adding CPUs or changing the I/O subsystem configuration
• Database configuration: Changing initialization parameter settings
• Schema changes: Hash-partitioning a table or index, or using automatic segment space
management
• Application changes: Using the cache option for sequences or using bind variables
• Using other advisors: Running the SQL Tuning Advisor on high-load SQL or running
the Segment Advisor on hot objects
Objectives
The goal of this lesson is to provide an overview of the various high-availability
architectures that you can implement with RAC. It is out of the scope of this lesson to give
you detailed information on how to set up these architectures. For more information, refer
to the corresponding documentation or courses.
Tape
Controllers
Network
Power
Network
Data Guard
Rolling upgrades
System
changes
Dynamic provisioning
Planned
down time
Data
Online redefinition
changes
Nodes RAC
Component
Instances failure RAC
Software failure
Human
error
Data Data Guard
Environment
Clients
Oracle Oracle
Application Application
Server Server
WAN Traffic
Manager
Primary Secondary
site Data Guard site
ARCn
ARCn LGWR RFS
Flash
Primary recovery
database Standby area
Online redo
redo files
files Standby
Flash
recovery database
area LGWR RFS
Apply
ARCn
ARCn
Primary instance B Standby apply instance D
*.DG_BROKER_CONFIG_FILE1=+DG1/RACDB/dr1config.dat
*.DG_BROKER_CONFIG_FILE2=+DG1/RACDB/dr2config.dat
RAC01 RAC02
Shared storage
Clients 1 Clients 2
Oracle
A B Patch patch
upgrades
Operating
Initial RAC configuration Clients on A , patch B system
upgrades
4 Clients Clients 3
Hardware
upgrades
Patch
Clients
DWDM DWDM
device device
DB DB
Copy Copy
Public network
Clients
DB DB DB DB
copy copy copy copy
Mid-tier S1
partition 1 S0 S1 S2 RAC01
Mid-tier S0
non-DT S0 S1 S2 RAC02
Mid-tier S2
partition 2 S0 S1 S2 RAC03
Production Test
cluster cluster
RAC RAC
database database