ASM Best

Known Methods
 Configuring Oracle on All Flash

INDEXES1
SYSTEM

ROLLBAC
Software

K CTRL 2
CTRL 1

DATA1
CTRL3
Oracle

ARCHIVE
INDEXES2

REDO1
REDO2
DATA2

TOOLS
TEMP

The Nine Disk Solution from "Oracle 8 DBA Handbook"

(author: Kevin Loney, OraclePress)
Chapter 4: Physical database layouts, Page 99

2
Configuring Oracle on All Flash

Having separate LUNs for different database I/O types not needed.

Good practice to still separate:

– OCR/Vote (if using RAC)
– DATA
– REDO
– Archive (FRA)

3
Configuring Oracle on All Flash

Best Practice for Oracle on All Flash:

Implement Balance Zoning Best Practice (8, 16 Paths Max)

Multi-pathing – Load-Balancing, Fail-over, Failback; i.e., Round-Robin, Queue-Length, Min IO 1

Having a small number of LUNs (4-10) is fine. 4 minimum.

Align by 8K when ever possible (block sizes, etc)

4
Partition Alignment still matters!
Reads and writes not aligned with 4K
physical sectors. Cause: incorrect
8K write operation partitioning leading to
512 512 512 512 512 512 512 512
improper alignment
512 512 512 512 512 512 512
512 512 512 512 512 Misaligned
512 512 512

4K Physical Sector 4K Physical Sector 4K Physical Sector

512 512 512 512 512 512 512 512 512 512 512 512

512 512 512 512 512 512 512 512 512 512 512 512

• Misaligned reads and writes are split across three 4KB physical sectors negatively
impacting performance: three I/O requests for misaligned instead of the optimal aligned
two I/O requests.

• For writes this is very important where each operation performs two I/O
transfers: read and modify (write)
5
Solution 1: Unpartitioned Devices

Don’t Partition LUNs

Metalink Note: 452924.1

• Logical unit numbers (LUNs)—Using hardware RAID
functionality to create LUNs is a recommended approach.
• Storage hardware RAID 0+1 or RAID5, and other RAID
configurations, can be provided to ASM as ASM disks.

6
Solution 2: Partition with 1MB Offset
Required if using ASMLib or ASM Filter Devices

[root@localhost ~]# parted /dev/sdc mklabel gpt ; parted

/dev/sdc mkpart primary 2048s 100%

Information: Don't forget to update /etc/fstab, if necessary

Information: Don't forget to update /etc/fstab, if necessary

7
ASM AU Sizes
 AU (Allocation Unit) sizes range from 1MB to 64MB
 Smaller sizes are good for OLTP
 Larger sizes are good for OLAP/DW
 AU can be set by Disk Group, but not changed after
 Exadata uses a 4MB AU
Smaller AU, needs more memory for metadata Larger AU, needs less memory for metadata

4MB 8MB 16MB 32MB 64MB

1MB 2MB

8
ASM Data Protection
 Redundancy is provided through mirroring
 Normal redundancy is a two-way mirror
 High redundancy is a three-way mirror
 External redundancy is no mirroring, handled by array instead

Use External Redundancy from

your flash array

9
ASM Data Protection for OCR/VOTE diskgroup
 For Oracle Clusterware files ASM mirroring is a little different
 External redundancy requires a single disk
 Normal redundancy requires 3 devices – 3 voting files - 2 failure
groups
 External redundancy requires 5 devices – 5 voting files – 3
failure groups

OCR OCR OCR

Voting Voting
ConsiderVoting
Voting
Normal Redundancy
Voting
File File File
for OCR/VOTEFilefiles
File

source https://docs.oracle.com/database/121/CWADD/votocr.htm#CWADD91889
10
ASM Diskgroups

Diskgroup Name Files Replicate?

DATA General purpose data diskgroup •CONTROL FILES •SYSAUX YES

•SYSTEM
•DATA •TEMP
•SYSAUX
•INDEX •TEMP
•UNDO
•REDO LOGS
•SYSTEM •UNDO
FRA Flash Recovery Area/ •ARCHIVE LOGS •CONTROL
Depends
FILES
FRA FlashRecovery
Fast RecoveryArea
Area •ARCHIVE LOGS
•BACKUPS •CONTROL FILES
•BACKUPS •REDO LOGS
CRS Cluster •CLUSTER REPOSITORY NO
CRS Cluster •CLUSTER REPOSITORY
•VOTING DISK
•VOTING DISK

REDO Redologs •REDO LOGS YES

REDO Redologs •REDO LOGS

TEMP Temporary •TEMP NO

11
ASM Diskgroups
 Store business critical DBs in separate ASM disk groups
– ASM disk group must be replicated as a whole
– Data striped across equally across the Disk Group
– Can do surgical analysis & monitoring of I/Os & performance

 Avoid creating unnecessary number of ASM disk groups

– Does 2 Disk Groups satisfy requirements?
– Typically, DATA, REDO and FRA minimally
– If TEMP low I/O write activity, combine with DATA
– Configuration based on Backup, IOPS/MBs needed, DR solution

12
ASM Rebalance and IDP
 ASM will still rebalance when ASM disks are added, this is
desirable to balance I/O queues in the OS.

 ASM Compaction Phase is irrelevant, but disabling it requires

setting a hidden parameter – check with MOS to ensure you
are still supported!

 Intelligent Data Placement (IDP) is irrelevant in all-flash –

there is no “outer edge”.
13
Powerpath or MPIO
 If using PowerPath or MPIO multi-pathing software and ASM
 Use the pseudo-device name to prevent errors
 Otherwise ASM will see multiple paths

/dev/sdb

Server Storage
/powermnt /dev/sdc

/dev/sdd

14
Powerpath or MPIO
Edit the file /etc/multipath.conf and add/modify the following parameters:

devices {
device {
vendor XtremIO Capture XtremIO Devices
product XtremApp
path_selector "queue-length 0" (FOR RHEL>=6)
path_selector "round-robin 0" (FOR RHEL<6)
rr_min_io 1 (FOR kernels older than 2.6.31) LB Algorithm
rr_min_io_rq 1 (FOR kernels 2.6.31 and above)
path_grouping_policy multibus
path_checker tur
}
}

15
SLA Based Data
Protection
 RecoverPoint
• SLA Based Protection of Oracle & Other Applications

• Apply protection policies by application

– Maintain or improve application recovery objectives

• Prioritize application failover to ensure business continuity

– Minimize data loss and downtime for mission-critical applications

• Testing and automation is key to ensuring lowest recovery objective

– Simplify disaster recovery drills and improve disaster recovery processes

Bronze Silver Gold Platinum Bronze Silver Gold Platinum

WAN

17
Accelerate Oracle Recovery
• Concurrent local and remote
Virtual Server Virtual Server replication for Oracle Applications and
Databases
Production Server / OS Test/Failover
– DVR-like recovery to any point-in-time
– Compress and de-dupe for 9x savings
in bandwidth costs
Applications – Supports Oracle DataGuard
Middleware
Databases • VMware Effect
– Failover Oracle virtual machines in
minutes non-disruptively
Local Remote – Non-disruptive testing of recovery
plans with less infrastructure

18
RecoverPoint for Oracle & VMware
Improved Operational Recovery and Disaster Recovery
RecoverPoint Family • Dynamic synchronous and
asynchronous replication
CDP: local
protection • Out-of-band replication
CRR: remote • Integrated with VMware vCenter
protection
Server and Site Recovery Manager
CLR: concurrent
local and remote • DVR-like recovery of replicated
protection data to any point in time
Protect Oracle Applications, Middleware
and Databases with single Solution • Support federated and clustered
applications
19
Operational Recovery for Oracle
Daily backup Daily recovery points—from tape or disk
Snapshots More frequent disk-based recovery points
Any point in time
All recovery points
Significant point in time
Database Pre-app Post-app Database Quarterly Any user-
checkpoint patch patch checkpoint close configurable event
Significant
points in time
RecoverPoint
Any point
in time
Snapshot

Daily backup
24 hours
Yesterday Midnight Now

20
Disaster Recovery for Oracle
Ensure application consistency and recovery
Production Site Remote Site • Replicate over long distances
• Send only compressed
Application Source Application
1
Source
CG1
Target
CG1 1 changes over networks –
Application
reduce bandwidth up to 90%
Source Sync
Sync oror Source
Target Application
2 async
async
CG2
replication
replication
CG2 2
• Recover production site
Application Source Source
Target Application and fail back applications
3 CG3 CG3 3
without disruption
Fail
Update
back Recover
Policy-based
FailProduction
over
Resynch
toat
toproduction
remote
remote
replication
sites
failure
site
site
site • Can be deployed with Oracle
Data Guard or independent
21U S E C A S E
RecoverPoint for Oracle
EMC Recover Point can be used to replicate ASM disk groups

DB DB DB DB
ASM ASM ASM ASM

Passive
Active Node
Failed Node Image Access
Passive
Active Mode
Node
Node

23
RecoverPoint Supports Oracle ASM
EMC Recover Point can be used to replicate ASM disk groups
CONSISTENCY GROUPS
replicate all ASM disk groups
DB DB
ASM ASM
REPLICA can be used for dev/test
refresh, reports, backups or DR

DATABASE AND APPLICATION

can be protected this way

REPLICA remains
down until accessed
Active Node ASM DISK GROUPS Passive Node
can be renamed on the replica

24
VPLEX: Distributed Storage Federation Platform

Next Generation of Data Mobility and Access

Local Metro Geo Global

Data Center Synchronous Asynchronous Anywhere
Less than Less than
5 ms 15 ms

25
Extended RAC with EMC VPLEX
Extended Clusters with Continuous Application Availability
100% uptime of ORACLE RAC over
distance
DB DB DB DB
ASM ASM ASM ASM
SIMPLE to configure and manage

ACTIVE
VPLEX Metro
ACTIVE
Use standard Oracle RAC
UNMODIFIED

Rebuilds are AUTOMATIC

Nodes come on-line immediately
Active Failure Domain Active Failure Domain
FULL asset utilization

26
Oracle Storage Compatibility Program
• Oracle Storage Program Change Notice

Over the past few years Oracle has worked with all the leading system and storage vendors to
validate their specialized storage products, under the Oracle Storage Compatibility Program (OSCP),
to ensure these products were compatible for use with the Oracle database. Under the OSCP, Oracle
and its partners worked together to validate specialized storage technology including NFS file
servers, remote mirroring, and snapshot products.

At this time Oracle believes that these three specialized storage technologies are well understood by
the customers, are very mature, and the Oracle technology requirements are well know. As of
January, 2007, Oracle will no longer validate these products. We thank our partners for their
contributions to the OSCP.

http://www.oracle.com/technetwork/database/features/availability/oscp-092136.html
28
My Oracle Support note 1304727.1
Certification Information for Oracle Database on Linux x86-64

This document contains general information pertaining to all releases of the Oracle Database, and information about individual releases

Platform Information

Processor Chip Information:

Linux x86-64, support on AMD64/EM64T and Intel Processor Chips that adhere to the x86-64 architecture with supported Linux releases.

OS Information:

Oracle Unbreakable Enterprise Kernel information:

Unbreakable Enterprise Kernel for Linux is available for x86-64 platforms. It is based on a recent stable mainline Linux kernel, and also includes optimizations developed in collaboration with Oracle
Database, Oracle middleware, and Oracle hardware engineering teams to ensure stability and optimal performance for the most demanding enterprise workloads. New releases of Unbreakable
Enterprise Kernel occur approximately every 18 months.

ZFS - use of ZFS with Oracle Database on Linux x86-64 is not supported.

Virtualization and Partitioning Information:

Customers should review Supported Virtualization and Partitioning Technologies for Oracle Database and RAC Product Releases

Oracle Single Instance Databases on Linux with HP Virtual ConnectFlexFabric (FCoE with 10gE technologies) are supported.

Microsoft Azure and Windows Server with Hyper-V:

Oracle Database 11g Release 2 (11.2.0.4, single instance only) is supported running in an Oracle Linux 6.4 (or later) guest on Microsoft Azure,

No version of Oracle Database is certified to run in a Linux guest on Microsoft Windows Server with Hyper-V.

29
My Oracle Support note 1304727.1 (cont)
• 12.1
OS versions and minimum levels:
Oracle Linux 7 (12.1.0.2 only) with
the following kernels:
Oracle Linux 6.x
Oracle Linux 5 Update 6
Red Hat Enterprise Linux 7 (12.1.0.2
only) with the following kernels:
Red Hat Enterprise Linux 6
Red Hat Enterprise Linux 5 Update 6
SLES 11 SP2: 3.0.13-0.27 or later
30
11.2
OS versions and minimum levels:
Oracle Linux 4 Update 7 and higher
Oracle Linux 5 Update 2 and higher
Oracle Linux 6.x - Database 11.2.0.3 and
higher
Oracle Linux 7 - Database 11.2.0.4 or
higher
Red Hat Enterprise 4 Update 7 and higher
Red Hat Enterprise 5 Update 2 and higher
Red Hat Enterprise Linux 6.x - Database
11.2.0.3 and higher
Red Hat Enterprise Linux 7 - 11.2.0.4
only
My Oracle Support note 417770.1
Purpose

This document outlines Oracle's Linux Support Policy of Linux running under emulated and virtualized environments.

Scope & Application

This document applies to anyone wishing to use Oracle Linux running under a virtualized or emulated environment.

Oracle Linux Support Policies for Virtualization and Emulation

Oracle Linux Support Program provides support for:

Oracle VM Server for x86

The Oracle Unbreakable Enterprise Kernel running on Oracle Linux 5.5 or higher.

Operating system support for RHEL3 (and higher), Oracle Linux 4 (and higher) under the Oracle Linux Support Program on Oracle VM

Operating system support for RHEL3 (and higher), Oracle Linux 4 (and higher) under the Oracle Linux Support Program on VMware vSphere (ESX Server).

Operating system support for RHEL3 (and higher), Oracle Linux 4 (and higher) under the Oracle Linux Support Program on Citrix XenServer Enterprise Edition

Xen components as part of Oracle Linux 5 and RHEL5 under the Oracle Linux Support Program. This does not include Oracle Product support on Xen offerings

31
 Oracle OEL 6 Certified for VMware
• Storage Systems, Volume Management and File
Systems

• 3Par

• Compellent

• EMC VPLEX

• FalconStor

• Fusion-io

Virtualization Platforms and Cloud Providers

• Amazon EC2

• Citrix XenServer

• Rackspace Cloud

• VirtualBox

• VMWare ESX

Source: https://blogs.oracle.com/sergio/entry/the_oracle_enterprise_linux_so
32
My Oracle Support note 249212.1
Scope & Application

For Customers running Oracle products on VMware virtualized environments. No limitation on use or distribution.

Support Status for VMware Virtualized Environments

Oracle has not certified any of its products on VMware virtualized environments. Oracle Support will assist
customers running Oracle products on VMware in the following manner: Oracle will only provide support for
issues that either are known to occur on the native OS, or can be demonstrated not to be as a result of running
on VMware.

If a problem is a known Oracle issue, Oracle support will recommend the appropriate solution on the native OS.
If that solution does not work in the VMware virtualized environment, the customer will be referred to VMware
for support. When the customer can demonstrate that the Oracle solution does not work when running on the
native OS, Oracle will resume support, including logging a bug with Oracle Development for investigation if
required.

If the problem is determined not to be a known Oracle issue, we will refer the customer to VMware for support.
When the customer can demonstrate that the issue occurs when running on the native OS, Oracle will resume
support, including logging a bug with Oracle Development for investigation if required.

NOTE: Oracle has not certified any of its products on VMware. For Oracle RAC, Oracle will only accept
Service Requests as described in this note on Oracle RAC 11.2.0.2 and later releases.

33
VMware’s Oracle Support Policy
Expanded Support

VMware is committed to the success of its customers in deploying simplified, cost-effective, and better
information technology services. To further this, we recently announced expanded support for Oracle
Database technical issues with the VMware vSphere platform. This expanded technical support is driven by
our VMware customers’ choice to deploy increasing amounts of their Oracle Database software with
VMware products.
This expanded support is targeted at Oracle Database usage “above and below” vSphere, where the Oracle
database is:
– used as a data store for VMware products
– run within a virtual machine on vSphere/ESX

VMware Oracle Support provides customers the following new advantages as part of the existing Support
and Subscription contract at no additional charge:
– Total ownership of Oracle Database technical issues reported to VMware Support
– Access to a team of Oracle DBA resources within VMware Support to troubleshoot related to Oracle Databases used as a data store
or run within a VM
– Performance tuning and Best Known Methods related to Oracle Database used as a data store or run within a VM
– Faster resolution of technical issues in VMware environments via a TSANet collaborative support arrangement between VMware
Support and Oracle Support

Source: http://www.vmware.com/support/policies/oracle-support.html

34
VMware’s Oracle Support Policy
Total Ownership

VMware Support will accept accountability for any Oracle-related issue reported by a customer. By
being accountable, VMware Support will drive the issue to resolution regardless of which vendor
(VMware, Oracle, or others) is responsible for the resolution. In most cases, reported issues can be
resolved via configuration changes, bug fixes, or feature enhancements by one of the involved
vendors.
In the rare situation that another vendor is unable or unwilling to provide a satisfactory technical
resolution, VMware Support will immediately notify the customer, assist in escalation and explore
other potential technical workarounds with the customer.
VMware will also assist its customers with technical issues for other Oracle software products,
besides the Oracle Database and provide similar escalation assistance if needed.
Besides technical assistance, VMware Support will advocate on the customer’s behalf to:
– Provide any relevant evidence that virtualization does not play a part in the Oracle product technical problem
– Engage Oracle Support in resolving the customer’s technical issue, escalating management attention as appropriate

Source: http://www.vmware.com/support/policies/oracle-support.html
35