h8029 Maximize Oracle Rac Symmetrix Fast VP WP

White Paper
EMC GLOBAL SOLUTIONS

Abstract
This white paper describes an automated storage tiering solution for
Oracle RAC 11g databases residing on Symmetrix VMAX storage.
EMC
Symmetrix
FAST VP automatically monitors and moves data

between thin storage tiers at the sub-LUN level, placing the right data
on the right tier at the right time, without user intervention.

February 2011

MAXIMIZE OPERATIONAL EFFICIENCY FOR
ORACLE RAC ENVIRONMENTS WITH EMC
SYMMETRIX FAST VP (AUTOMATED TIERING)
An Architectural Overview

2 Maximize Operational Efficiency for Oracle RAC Environments
with EMC Symmetrix FAST VP (Automated Tiering)An Architectural Overview

Copyright 2011 EMC Corporation. All Rights Reserved.

EMC believes the information in this publication is accurate of its
publication date. The information is subject to change without notice.

The information in this publication is provided as is. EMC
Corporation makes no representations or warranties of any kind with
respect to the information in this publication, and specifically
disclaims implied warranties of merchantability or fitness for a
particular purpose.

Use, copying, and distribution of any EMC software described in this
publication requires an applicable software license.

For the most up-to-date listing of EMC product names, see EMC
Corporation Trademarks on EMC.com.

VMware and ESX are registered trademarks or trademarks of VMware,
Inc. in the United States and/or other jurisdictions. All other
trademarks used herein are the property of their respective owners.

Brocade may have substituted components of the original
environment in this document with hardware of a similar (or higher)
specification to the original equipment used in the EMC Proven
Solution. The content contained in this document originated from a
validated EMC Proven Solution. The modification introduced by
Brocade may have caused changes in performance, functionality, or
scalability of the original solution. Please refer to
www.EMC.com/solutions for further information on validated EMC
Proven Solutions.

Part Number: h8029

Table of Contents
Executive summary .............................................................................................................. 5
Introduction to EMC Symmetrix VMAX with Enginuity 5875 ................................................. 5
Business case .................................................................................................................... 5
Solution overview ............................................................................................................... 6
Key results ......................................................................................................................... 7
Introduction ........................................................................................................................ 8
Purpose .............................................................................................................................. 8
Scope ................................................................................................................................. 8
Audience ............................................................................................................................ 8
Terminology ....................................................................................................................... 8
Key technology components .............................................................................................. 10
Overview .......................................................................................................................... 10
EMC Symmetrix VMAX with Enginuity 5875 ....................................................................... 10
EMC Symmetrix Management Console .............................................................................. 10
EMC Symmetrix Performance Analyzer .............................................................................. 10
EMC Virtual Provisioning .................................................................................................. 11
EMC Virtual LUN VP Mobility ............................................................................................. 11
EMC Symmetrix FAST VP ................................................................................................... 11
EMC PowerPath ................................................................................................................ 11
Oracle Database 11g R2 Enterprise Edition ....................................................................... 11
Brocade infrastructure ...................................................................................................... 12
EMC Virtual Provisioning .................................................................................................... 13
Virtual Provisioning overview ............................................................................................ 13
Overprovisioning .............................................................................................................. 13
Monitoring and administering virtual pools ...................................................................... 14
Growing and rebalancing virtual pools .............................................................................. 15
EMC Symmetrix FAST VP .................................................................................................... 16
FAST VP overview .............................................................................................................. 16
FAST VP components ........................................................................................................ 16
FAST VP performance measurement and data movement ................................................. 17
FAST VP with an Oracle OLTP workload ............................................................................. 18
FAST VP with Oracle ASM striping ..................................................................................... 19
Solution architecture and design ....................................................................................... 20
Solution architecture ........................................................................................................ 20
Use case profile ................................................................................................................ 21
Hardware and software environment ................................................................................ 21
Database storage layout for thin devices .......................................................................... 22
Storage design considerations for Oracle ......................................................................... 22

Oracle database configuration ........................................................................................... 23
Database schemas ........................................................................................................... 23
Database services ............................................................................................................ 23
SOE tables and indexes .................................................................................................... 23
Configuring EMC Symmetrix FAST VP .................................................................................. 24
Overview of FAST VP configuration .................................................................................... 24
Step 1: Enable FAST controller and set control parameters ............................................... 24
Step 2: Create storage groups .......................................................................................... 25
Step 3: Create storage tiers .............................................................................................. 26
Step 4: Create FAST policies ............................................................................................. 26
Step 5: Associate storage groups with FAST policies ......................................................... 27
Step 6: Configure FAST VP monitoring and move windows ................................................ 28
Solution testing and validation .......................................................................................... 29
FAST VP versus manual tiering .......................................................................................... 29
FAST VP test procedure ..................................................................................................... 30
FAST VP rebalancing across tiers ...................................................................................... 31
FAST VP effects on database transactions per minute ....................................................... 32
FAST VP effects on Oracle service response time .............................................................. 32
FAST VP effects on average host IOPS ............................................................................... 33
Summary of FAST policy effects ........................................................................................ 33
Conclusion ........................................................................................................................ 35
Summary .......................................................................................................................... 35
Findings ........................................................................................................................... 35
References ........................................................................................................................ 37
White papers and technical notes .................................................................................... 37
Product documentation .................................................................................................... 37
Other documentation ....................................................................................................... 37

Executive summary
Introduction to EMC Symmetrix VMAX with Enginuity 5875
EMCs latest Proven Solutions for both desktop and business-critical application
environments make it easier and faster to manage infrastructure and clone applications for
test and development through virtualization. These solutions document best practices that
can drive significant cost savings and better performance for demanding application
workloads, while accelerating the customers journey to the private cloud.
Proven Solutions use new and enhanced features to validate storage efficiency and scale for
organization growth:
EMC
Symmetrix
Fully Automated Storage Tiering for Virtual Pools (FAST VP)

providing better optimization of high-performance Flash drives, and better
responsiveness to changes in data activity through sub-LUN automated storage tiering.
VMware
vStorage APIs for Array Integration (VAAI)enabling storage administrators to

maintain control of the infrastructure, while managing virtual server storage and other
applications sharing the same storage array.
New duplicate TimeFinder
/Snapscreating capacity-efficient replicas for application

testing and development, and maximizing the use of existing storage.
Business case
Today's business needs are driving changes in IT models and infrastructure solutions,
demanding increased operational efficiency and improved quality of service to advance
workforce productivity and enable business transformation.
To meet this challenge, there is an ongoing shift by IT departments to provide more agile
service delivery and to design for the future, while still cost-effectively managing existing
business requirements and service levels. To achieve this, they are taking advantage of
concepts such as resource pooling, virtualization, dynamic and thin provisioning, and
commodity computing, with the goal of delivering IT as a service.
Transitioning to a virtualized infrastructure and pooled architecture can simplify the
partitioning and management of application workloads to deliver the required levels of
availability, performance, and scalability.
To deliver the quality of service required in these environments, it is essential that
infrastructure and tools simplify storage management processes and improve capacity
utilization. EMC Symmetrix VMAX
, Enginuity
5875, and associated management tools have

been developed to be the foundation of this infrastructure and to meet real business needs.

Solution overview
This white paper demonstrates the benefits of using Symmetrix FAST VP to provide
automated storage tiering for an Oracle Real Application Clusters 11g Release 2 database.
The main challenges addressed by this EMC solution are outlined in this section.
Challenge Solution
Optimized
performance tuning
FAST VP offers highly granular sub-LUN movement of data across a
mix of drive technologies and protection levels, based on analysis of
data access patterns and user-defined policies. This enables it to
dynamically adjust use of the underlying storage to meet the
changing performance requirements of application workloads in real
time.
FAST VP ensures predictable service levels and maintains these over
time, reducing the need for manual tuning of the underlying storage.
Optimized storage
capacity utilization
FAST VP identifies highly active data and moves it to a high-
performing tier, such as Enterprise Flash Drives (EFDs), while moving
inactive or less active data to a storage tier more suited to its
performance requirements, such as Fibre Channel (FC) or Serial
Advanced Technology Attachment (SATA).
This ensures the most efficient use of all storage under FAST VP
control.
Simplified
management and
operations
EMC Virtual Provisioning helps reduce the complexity of storage
provisioning as well as the time and effort required. Virtual
Provisioning also increases capacity utilization by allowing
applications to share a common pool of storage.
EMC Symmetrix Management Console (SMC) templates and wizards
provide simple setup and administration, enabling administrators to
quickly provision and optimize Symmetrix VMAX resources.
Flexibility Symmetrix VMAX systems support EFDs, and FC and SATA drives, as
well as an extensive range of RAID types. This enables FAST VP to
distribute application data across different drive technology types to
deliver the right level of service to each application.
Virtual Provisioning enables storage capacity to be added and
removed nondisruptively and on demand.
Virtual LUN VP Mobility provides additional flexibility by enabling
administrators to manually move devices between tiers with different
performance characteristics.

Key results
This solution provides the following benefits:
Increased performance, at a similar cost to an all-FC environment, by using FAST VP to
automatically tier data across a mix of EFDs, and FC and SATA drives. The introduction
of only 4 percent of high-performing EFD devices enabled the Oracle RAC cluster to
handle more transactions per minute and improved database service response times
by 26 percent.
Reduced administration costs as the underlying storage is continually and
automatically being optimized for best performance. Storage, server, and database
administrators can now each spend less time monitoring and optimizing their IT
infrastructure.
FAST VP reduces the time spent monitoring and reconfiguring storage from a repeated
process taking more than 9 hours to complete each time (manual configuration) to a
one-off operation taking 40 minutes or less (FAST VP), eliminating the need for
repetitive analysis and manual retuning by administrators.
Nondisruptive, transparent movement of data to the most effective storage tier.
Improved service-level agreement (SLA) compliance by automatic tuning of storage to
adapt to changing workload patterns.

Introduction
Purpose
This white paper describes an automated tiered storage solution for Oracle Database 11g
Release 2 RAC environments, using Symmetrix VMAX storage with FAST VP.
Scope
The scope of this paper is to:
Present an overview of the concepts and technologies involved in the solution.
Document details of the storage infrastructure for the use case that was implemented
to validate the solution.
Describe how Oracle and FAST VP were configured for the use case.
Present the results of the use case validation and testing.
Identify the key business benefits of automating storage tiering with FAST VP in Oracle
RAC environments.
Audience
This white paper is intended for Oracle database administrators, storage architects, EMC
customers, and field personnel who want to understand how EMC products and solutions can
provide Virtual Provisioning, automatic storage tiering, and advanced management,
monitoring, and reporting facilities for Oracle Database 11g RAC in an online transaction
processing (OLTP) environment.
Terminology
Table 1 defines terms used in this document.
Table 1. Terminology
Term Definition
Data device Virtual Provisioning term for devices (not mapped to the host) that provide
physical storage for thin devices. Data devices must be contained in a
virtual pool before they can be used.
Enginuity The operating environment that provides the intelligence that controls all
components in a VMAX array.
Extent group See Thin device extent.
FAST Fully Automated Storage Tiering. A feature introduced with EMC Enginuity
5874 that provides automated storage tiering at the LUN level.
FAST policy FAST policies manage data placement and movement across storage
types to achieve service levels for one or more storage groups. They
specify a set of tier usage rules that defines how an applications devices
will be most effectively distributed across storage tiers.

Term Definition
FAST VP Fully Automated Storage Tiering for Virtual Pools. A feature of EMC
Enginuity 5875 that provides automatic storage tiering at the sub-LUN
level. VP denotes virtual pools, which are Virtual Provisioning thin pools.
SMAS Symmetrix Management Application Server. A combined product installer
for SMC and SPA.
SMC Symmetrix Management Console. A browser-based interface for managing
EMC Symmetrix storage.
SPA Symmetrix Performance Analyzer. A server-based application that
monitors workload activity for Symmetrix arrays.
Storage group A user-defined logical grouping of devices for common management by
FAST VP. A storage group is associated with a FAST policy, which
determines how the storage groups devices are allocated across tiers.
Storage tier A user-defined set of one or more virtual pools containing data devices of
the same technology and protection type.
SYMCLI Symmetrix Solutions Enabler command line interface.
Thin device (TDev) A cache-only device that is presented to a host. TDevs are pointers to units
of physical storage contained in Virtual Provisioning thin pools.
Thin device extent A thin device extent is one Symmetrix track group (12 tracks or 768 KB).
An extent group is a group of contiguous thin device extents. FAST VP
relocates data at the extent group level.
Virtual pool A Virtual Provisioning thin pool, consisting of a collection of data devices
that provides storage capacity for the thin devices that are bound to the
pool. Virtual pools provide the storage tiers used by FAST VP.
Virtual Provisioning EMCs implementation of thin provisioning.

Key technology components
Overview
The solution uses the following EMC hardware and software components:
EMC Symmetrix VMAX with Enginuity 5875
EMC Symmetrix Management Console (SMC)
EMC Symmetrix Performance Analyzer (SPA)
EMC Virtual Provisioning
EMC Virtual LUN VP Mobility
EMC Symmetrix FAST VP
EMC PowerPath

The solution uses the following Brocade and Oracle technologies:
Brocade 5100 switches
Oracle Database 11g R2 Enterprise Edition, Oracle Grid Infrastructure, and Oracle RAC
EMC Symmetrix VMAX with Enginuity 5875
EMC Symmetrix VMAX storage arrays provide high performance and scalability for demanding
enterprise storage environments. Built on the strategy of simple, intelligent, modular storage,
the Symmetrix VMAX incorporates a highly scalable Virtual Matrix Architecture that enables it
to grow seamlessly and cost-effectively from an entry-level configuration into the worlds
largest storage system. The VMAX supports EFDs, and FC and SATA drives within a single
array, as well as an extensive range of RAID types.
The EMC Enginuity operating environment provides the intelligence that controls all
components in a VMAX array. EMC Enginuity 5875 introduces FAST VP, which enables
customers to improve efficiency and security, increase cost savings, and ensure predictable
performance from their Symmetrix VMAX arrays.
EMC Symmetrix Management Console
The Symmetrix Management Console (SMC) is a powerful, browser-based interface that
simplifies management of EMC Symmetrix storage, from device creation to advanced
Symmetrix features such as FAST, FAST VP, Virtual Provisioning, Auto-Provisioning Groups,
replication configuration, and monitoring.
EMC Symmetrix Performance Analyzer
Symmetrix Performance Analyzer (SPA) is a server-based application that provides a single
tool to monitor real-time workload activity for a number of Symmetrix arrays. It is integrated
with SMC and shares SMC resources while providing diagnostic, performance, and planning
information with easy-to-use graphical data representations.

Virtual Provisioning is EMCs implementation of thin provisioning. It is a key component of the
solution described in this white paper. A detailed overview is provided in the EMC Virtual
Provisioning section of the paper.
EMC Virtual LUN VP Mobility
Virtual LUN VP Mobility enables administrators to manually move thin LUNs between pools
VP denotes virtual pools, which are Virtual Provisioning thin pools.
This new feature of Enginuity 5875 includes the ability to re-gather a thin volumes many thin
device extents from multiple virtual pools and move them all to a single pool, regardless of
the underlying disk technology or RAID protection. Movements are completely transparent to
the environment and have no host or application impact.
EMC Fully Automated Storage Tiering for Virtual Pools (FAST VP) is a feature of EMC Enginuity
5875 that provides automatic storage tiering at the sub-LUN level. Virtual pools are Virtual
Provisioning thin pools.
FAST VP is a key component of the solution described in this white paper. A detailed overview
is provided in the EMC Symmetrix FAST VP section of the paper.
EMC PowerPath
EMC PowerPath is server-resident software that enhances performance and application
availability by supporting multiple I/O paths to logical devices and intelligently distributing
I/O requests across all available paths. PowerPath also provides automatic failover in the
event of a hardware failure by automatically detecting the path failure and redirecting I/O to
another path.
Oracle Database 11g R2 Enterprise Edition
This white paper presents a storage solution for consolidated Oracle Database 11g R2 OLTP
environments. The solution takes advantage of features of Oracle Database 11g R2 such as
RAC and Automatic Storage Management (ASM).
In Oracle Database 11g R2, Oracle ASM and Oracle Clusterware have been integrated into a
single set of binaries and named Oracle Grid Infrastructure. This now provides all the cluster
and storage services required to run an Oracle RAC database. Oracle ASM has also been
extended to include support for Oracle Cluster Registry (OCR) and voting files to be placed
within ASM disk groups.

Brocade infrastructure
The Brocade 5100 Switch combines 1, 2, 4, and 8 gigabits per second (Gb/s) technology in
configurations of 24, 32, or 40 ports in an efficient 2U design. The combination of port
density, performance, and pay-as-you-grow scalability enables higher throughput and
greater storage utilization while reducing complexity for virtualized data centers. The
evolutionary design makes it very efficient with regard to power consumption, cooling, and
rack densityenabling medium- and large-scale server and storage consolidation for greater
cost savings, better price/performance, and manageability.
Brocade SAN switches feature a non-blocking architecture with ports concurrently active at 8
Gb/s (full duplex) with no oversubscription. This level of performance and connectivity is
ideal for expanding virtual server environments. In addition, enhanced Inter-Switch Link (ISL)
Trunking can supply up to 64 Gb/s of balanced data throughput in a single logical link.
The Brocade 5100 offers Top Talkers (part of Advanced Performance Monitoring) and
Adaptive Networking services, a suite of tools including Ingress Rate Limiting, Traffic
Isolation, and Quality of Service (QoS). These advanced capabilities help optimize fabric
utilization and allocate ample bandwidth for mission-critical Oracle applications. Providing
maximum flexibility, the Brocade 5100 has Integrated Routing (IR) capabilities to connect
switches in different fabrics. Also, the Virtual Fabrics feature enables the partitioning of a
physical SAN into Logical Fabrics. This provides fabric isolation by application, business
group, customer, or traffic typewithout sacrificing performance, scalability, security, or
reliability.
The Brocade Network Advisor streamlines management of the storage network and provides
fabric-wide monitoring capabilities.

Virtual Provisioning overview
EMC Virtual Provisioning is EMCs implementation of thin provisioning and is designed to
simplify storage management, improve capacity utilization, and enhance performance. Virtual
Provisioning provides for the separation of physical storage devices from the storage devices
as perceived by host systems. This enables nondisruptive provisioning and more efficient
storage utilization.
Virtual Provisioning introduces the following concepts and components:
Thin devices (TDevs) are devices that do not have storage allocated to them when they
are created. Thin devices can be created with an inflated capacity, because the actual
storage space for data written to them is provided by data devices (see below). To a
host operating system, thin devices look like regular devices with their configured
capacity, and the host interacts with them in the same way as with regular devices.
Data devices are special devices (not mapped to the host) that provide physical
storage for thin devices. Data devices must be contained in a virtual pool before they
can be used.
A virtual pool is a collection of data devices that provides storage capacity for the thin
devices that are bound to the pool. All data devices in a given virtual pool share the
same RAID protection type and are of the same drive technology.
Virtual Provisioning automatically stripes data across all data devices in a virtual pool,
balancing the workload across physical storage devices. To ensure even striping of
data, it is recommended that all data devices in a virtual pool are the same size.
Virtual pools provide the storage tiers used by FAST VP.
Overprovisioning
Virtual Provisioning makes it possible to provision storage for applications without providing
the physical storage up front. This means that administrators can assign enough storage to
last the lifetime of the application without needing to purchase all the physical storage in
advance.
This approach has the following benefits:
Initial acquisition costs can be reduced, because storage is added only as required.
There are fewer disruptions to the application to add or change storage devices.
Figure 1 illustrates a virtually provisioned environment and the manner in which storage is
associated with thin devices in the pools.


Figure 1. Storage allocation in Virtual Provisioning environments
The environment consists of three thin devices and two virtual pools. Devices 1 and 2 are
bound to RAID5_Pool; device 3 is bound to RAID1_Pool. In both pools, the extent groups are
evenly distributed across all data devices.
Oracle creates large files that are striped across multiple devices by Oracle ASM. Virtual
Provisioning complements Oracle ASM and ensures that each extent group is completely
filled before moving on to the next one. This ensures that data is striped evenly across the
virtual pool.
Monitoring and administering virtual pools
It is important to have processes in place to monitor virtual pools in order to ensure that
sufficient capacity is always available. Administrators can configure utilization thresholds
and alerts for virtual pools using EMC tools such as Symmetrix Management Console,
Symmetrix Performance Analyzer, and Solutions Enabler.

Growing and rebalancing virtual pools
As capacity demands increase, data devices can be added to existing virtual pools to meet
requirements this is an online process. After adding new data devices to a virtual pool,
administrators can use the automated pool rebalancing feature of Virtual Provisioning to
redistribute data evenly across all enabled data devices in the pool. Figure 2 shows the effect
of rebalancing a virtual pool.

Figure 2. Rebalancing virtual pools
In the first example, the pool has three new data devices that have not been rebalanced. New
writes will be striped across the new devices using the same stripe width as before the
devices were added.
The second example shows the pool after a rebalancing operation. The data is now spread
more evenly across all devices in the virtual pool and space has been freed up on the existing
devices to accept new writes. The stripe width has automatically been increased to
encompass the new devices.
Rebalancing virtual pools following the addition of new data devices can have very positive
effects on performance for applications such as Oracle Database. For further information, see
the EMC white paper Implementing Virtual Provisioning on EMC Symmetrix VMAX with Oracle
Database 10g and 11gApplied Technology.
Pool with newly-added devices
after automated rebalance Pool with newly-added devices

FAST VP overview
FAST VP provides support for sub-LUN data movement in thin provisioned environments. It
combines the advantages of Virtual Provisioning with automatic storage tiering at the sub-
LUN level to optimize performance and cost while radically simplifying storage management
and increasing storage efficiency.
FAST VP uses intelligent algorithms to continuously analyze devices at the sub-LUN level. This
enables it to identify and relocate the specific parts of a LUN that are most active and would
benefit from being moved to higher-performing storage such as EFD. It also identifies the
least active parts of a LUN and relocates that data to higher-capacity, more cost-effective
storage such as SATA, without altering performance.
Data movement between tiers is based on performance measurement and user-defined
policies, and is executed automatically and nondisruptively by FAST VP.
This section provides an overview of FAST VP features and functionality. The Configuring EMC
Symmetrix FAST VP section of the white paper outlines the main steps for configuring FAST VP
on Symmetrix VMAX and the settings defined for the solution use case.
FAST VP components
FAST VP configuration involves three types of componentsstorage groups, FAST policies,
and storage tiersas shown in Figure 3.

Figure 3. FAST VP components
A storage group is a logical grouping of storage devices used for common
management. A storage group is associated with a FAST policy, which determines how
the storage groups devices are allocated across tiers.
A FAST policy is a set of tier usage rules that is applied to associated storage groups. A
FAST policy can specify up to three tiers and assigns an upper usage limit for each tier.
These limits determine how much data from a storage group can reside on each tier
included in the policy.

Administrators can set high-performance policies that use more Flash drive capacity
for critical applications, and cost-optimized policies that use more SATA drive capacity
for less-critical applications.
A storage tier is made up of one or more virtual pools. To be a member of a tier, a
virtual pool must contain only data devices that match the technology type and RAID
protection type of the tier.
In summary, by simply pooling storage resources, defining a policy, and applying it to the
application, FAST VP will automatically and dynamically move application data to the tier that
best suits the level of service required.
FAST VP performance measurement and data movement
With FAST, entire devices are promoted or demoted between tiers based on overall device
performance. FAST VP works at the sub-LUN level, introducing finer granularities of both
performance measurement and data movement, and can spread the data from a single thin
device across multiple tiers.
The metrics collected at the sub-LUN level for thin devices under FAST VP control contain
measurements that enable FAST VP to make separate data movement requests for every
7,680 KB unit of storage that makes up the thin device. This unit of storage consists of 10
contiguous thin device extents and is known as an extent group.
Two types of moves are performed by FAST VP algorithms:
Compliance movement: Initially, FAST VP distributes data across the different tiers in
order to enforce compliance with the FAST policy with which the data is associated.
Performance movement: When compliance with the policy is achieved, FAST VP
continues moving data between tiers to optimize performance, while maintaining
compliance with the policy.
Figure 4 shows a thin device bound to a single virtual pool residing on FC storage with no
FAST policy applied.

Figure 4. Thin device storage before applying a FAST policy

A FAST policy is then associated with the thin devices storage group, with the policy
incorporating a mix of FC, SATA, and EFD tiers. The effect of this is shown in Figure 5.

Figure 5. Thin device storage after applying a FAST policy
FAST VPs automatic analysis identifies the busiest extent groups and moves them to the
RAID5_EFD_POOL in the highest-performing EFD tier. It also identifies inactive extent groups
and moves them to the RAID6_SATA_POOL in the SATA tier.
This results in the thin devices data being distributed across multiple thin pools. Because
the most active data is residing on the highest-performing storage devices, application
response times are unaffected.
Automatic analysis and re-tiering free database and storage administrators from the
repetitive tasks of performance analysis and tuning. FAST VP continuously tunes the storage
resources to ensure that the right data is placed on the right tier at the right time.
The Configuring EMC Symmetrix FAST VP section of the white paper outlines the main steps
for configuring FAST VP on Symmetrix VMAX and the settings defined for the solution use
case.
FAST VP with an Oracle OLTP workload
FAST VP is an enabling technology for workloads with small, random I/O and relatively small
working sets that fit into the higher-performing tiers of a FAST policy. Oracle OLTP databases
tend to be highly random in nature, with small working sets compared to the total database
size. Additionally, OLTP databases have inherent locality of reference with varied I/O
patterns, for the following reasons:
The relative importance of data changes from object to object. Some tables tend to be
accessed more than others.

The number of IOPS per gigabytes size of an object, also known as object intensity,
changes quite significantly. A good example is a database index compared with a
database table. The relative IOPS received by a database block occupied by an index
object can be very high compared to the IOPS received by a database block consumed
by a table object.
Note Oracle redo logs have a very predictable sequential write workload, and this type of activity
does not benefit significantly from up-tiering to EFD. It is recommended that these logs be
excluded from any FAST policy, or else pinned to their existing tier so that FAST VP will not
include them in its analysis.
FAST VP with Oracle ASM striping
When using FAST VP, there is no need to match the Logical Volume Manager (LVM) stripe
depth with the Virtual Provisioning thin device extent.
Because Oracle typically accesses data either by random single-block read/write operations
(usually 8 KB in size) or by sequentially reading large portions of data, FAST VP movements
have no impact on the LVM striping or on data access.

Solution architecture and design
Solution architecture
EMC solution use cases are designed to reflect and validate real-world deployments. Figure 6
depicts the physical architecture of the use case developed for the solution described in this
white paper.

Figure 6. Solution architecture
The use case details an implementation of FAST VP in an Oracle Database 11g R2 RAC
environment. Storage is allocated to Oracle ASM from virtual pools. The underlying physical
storage is provided by a Symmetrix VMAX array running Enginuity 5875. The array provides a
mix of EFDs, and FC and SATA drives, with FAST VP automatically relocating data across
storage tiers based on performance and predefined FAST policies.
For the use case, two identical database schemas were loaded onto the Oracle RAC cluster.
Workload was generated using Swingbench, and a performance baseline was taken to
validate the performance characteristics of the environment. Testing was then performed with
the goal of emulating a tier-1 database workload that exceeds the performance requirements
of most organizations.
A storage group was defined and all devices belonging to the schemas and to the +DATA ASM
disk group were added to it. A FAST policy was then associated with the storage group, and
the performance characteristics before and after applying the policy were compared. The
effects of applying the FAST policy are documented in the Solution testing and validation
section of this white paper.

Use case profile
Table 2 details the use case profile.
Table 2. Use case profile
Profile characteristic Details
Database schema size 960 GB each
Database profile OLTP
Benchmark profile Swingbench Order Entry (TPC-C-like) benchmark
User scaling Database scale to keep host CPU utilization in the range of 70%
Hardware and software environment
Table 3 details the hardware and software environment for the use case.
Table 3. Use case hardware and software environment
Purpose Quantity Configuration
Storage array 1 Symmetrix VMAX with:
2 x VMAX 64 GB engines
8 Gb FC connectivity
128 x 300 GB, 15k FC drives
28 x 2 TB, 7,200 rpm SATA drives
8 x 200 GB EFDs
Enginuity 5875
Infrastructure management
host
1 VMware ESX
4.1 server with:

2 x quad-core CPU, 96 GB RAM
Microsoft Windows Server 2008
Symmetrix Management Console 7.2
Symmetrix Performance Analyzer 2.0.1.4
Solutions Enabler 7.2
Swingbench 2.4
PowerPath 5.3.1
Oracle RAC database servers 2 Linux server with:
4 x eight-core CPUs, 128 GB RAM
2 x quad-port 1 Gb NICs
Red Hat Enterprise Linux 5.5
Oracle Database 11g Release 2 Enterprise
Edition (with Oracle RAC and Oracle Grid
Infrastructure)
FC switches 2 Brocade 5100 switches, 40 ports per switch

Ethernet switches 2 Gigabit Ethernet switches
Host bus adapters 6 8 Gb dual-port HBAs (2 per server)
Database storage layout for thin devices
Table 4 details the initial database storage layout for the thin devices.
Table 4. Initial database storage layout for thin devices
ASM disk group Virtual pool RAID protection
TDev hyper
size
No. of
TDevs
Provisioned
capacity
+DATA
(system, sysaux, and undo)
RAID5_FC RAID 5 (3+1) 64 GB 2 128 GB
+SOE1 RAID5_FC RAID 5 (3+1) 64 GB 15 960 GB
+SOE2 RAID5_FC RAID 5 (3+1) 64 GB 15 960 GB
+REDO RAID1_FC RAID 1 64 GB 1 64 GB
+CRS RAID1_FC RAID 1 5 GB 5 25 GB
+FRA RAID6_SATA RAID 6 (6+2) 64 GB 4 256 GB
+TEMP RAID5_FC RAID 5 (3+1) 64 GB 1 64 GB
Storage design considerations for Oracle
The Oracle database for the use case has separate ASM disk groups for data, redo log, CRS,
FRA, and temp files. The database schemas created for the use case, SOE1 and SOE2, were
stored in their own ASM disk groups.
The +DATA, +SOE1, and +SOE2 ASM disk groups were laid out on thin provisioned devices
that were bound to a virtual pool consisting of FC devices only, using RAID 5 3+1 protection.
The thin devices for the +TEMP ASM disk group were also bound to this pool.
The +CRS and +REDO thin devices were also fully provisioned and were bound to virtual pool
RAID1_FC. The +FRA devices were bound to virtual pool RAID6_SATA.
Note For the use case, there is a one-to-one mapping between a schema, a storage group, and an
ASM disk group, and all three components have the same name.

Oracle database configuration
Database schemas
Two identical instances of the Swingbench Order Entry - PL/SQL (SOE) schema were used to
deliver the OLTP workloads required by the solution: schemas SOE1 and SOE2. To test the
effects of FAST VP, a Swingbench Order Entry workload was generated and run against
schema SOE1. Schema SOE2 was left idle to provide a more realistic skew in the dataset.
Database services
Database services are entry points to an Oracle database that enable the management and
balancing of workloads across the cluster. They are a logical grouping of similar user sessions
that can be dynamically assigned to and between individual or multiple Oracle RAC nodes
and instances.
For the use case, each of the test schemas has a corresponding database service mapped to
it. As well as offering load balancing and failover across the cluster, this enables monitoring
of Oracle I/O to be mapped to each individual service and hence schema.
Table 5. Oracle database services
Schema Service RAC node/instance Swingbench sessions
SOE1 SOE1.emcweb.ie tce-ora04/dnb11 400
SOE2 SOE2.emcweb.ie tce-ora05/dnb12 0
SOE tables and indexes
The Swingbench SOE schema models a traditional OLTP database. Tables and indexes reside
in separate tablespaces. The two SOE schemas used in this solution are identical and each
contains the tables and indexes shown in Table 6.
Table 6. Schema tables and indexes
Table name Index
CUSTOMERS CUSTOMERS_PK (UNIQUE), CUST_ACCOUNT_MANAGER_IX,
CUST_EMAIL_IX, CUST_LNAME_IX, CUST_UPPER_NAME_IX
INVENTORIES INVENTORY_PK (UNIQUE), INV_PRODUCT_IX, INV_WAREHOUSE_IX
ORDERS ORDER_PK (UNIQUE), ORD_CUSTOMER_IX, ORD_ORDER_DATE_IX,
ORD_SALES_REP_IX, ORD_STATUS_IX
ORDER_ITEMS ORDER_ITEMS_PK (UNIQUE), ITEM_ORDER_IX, ITEM_PRODUCT_IX
PRODUCT_DESCRIPTIONS PRD_DESC_PK (UNIQUE), PROD_NAME_IX
PRODUCT_INFORMATION PRODUCT_INFORMATION_PK (UNIQUE), PROD_SUPPLIER_IX
WAREHOUSES WAREHOUSES_PK (UNIQUE)
LOGON

Configuring EMC Symmetrix FAST VP
Overview of FAST VP configuration
This section outlines the main steps for configuring FAST VP on a Symmetrix VMAX array.
1. Enable FAST controller and set control parameters.
2. Create the storage groups.
3. Create the storage tiers.
4. Create the FAST policies.
5. Associate each storage group with the relevant FAST policy.
6. Configure performance and move time windows.
Configuration steps can be carried out using the SMC FAST Configuration Wizard, SMC menu
commands, and the SYMCLI.
Step 1: Enable FAST controller and set control parameters
The first step in configuring FAST VP is to enable the FAST controller and to set the following
FAST control parameters:
Movement Mode: Either automatic or Off.
Relocation Rate: This controls how aggressive FAST VP will be in its data movements.
The lower the value, the more aggressive FAST VP will be. The minimum value is 1, the
maximum value is 10, and the default value is 5. After testing several values, 8 was
found to be the best setting for this use case.
This setting affects the amount of data that will be moved at any given time, and the
priority given to moving the data between pools it does not affect the speed of the
data movement.
Reserved Capacity Limit: A percentage of the capacity of each virtual pool that is
reserved for non-FAST activities. If the free space in a given virtual pool (as a
percentage of pool-enabled capacity) falls below this value, the FAST controller does
not move any more data into that pool.
Workload Analysis Period: The amount of workload sampling to maintain for analysis,
in units of time (hours, days, or weeks). The use case uses a value of two hours.
Initial Period: The minimum amount of workload sampling to be completed before
analyzing the samples for the first time. The use case uses a value of two hours.
For further information on control parameters used by FAST VP, see the EMC document
Implementing Fully Automated Storage Tiering for Virtual Pools (FAST VP) for EMC Symmetrix
VMAX Series ArraysTechnical Notes.

Step 2: Create storage groups
Creating a storage group involves specifying the following attributes:
The storage group name, which uniquely identifies the storage group.
The device source type, which specifies the source of the devices to be added to the
storage group.
The Symmetrix option allows the user to select specific devices from all those
available on the array.
After the source type has been specified, one or more of the available devices can then be
selected for inclusion in the storage group. It is best practice to label all devices when they
are created so that they can be easily identified for selection, as shown in Figure 7.

Figure 7. Creating a storage group
For the use case, a single storage group was created: OracleDataASM. This storage group
contains all the devices to be monitored by FAST VP.
Note A storage group should always include all of the devices in an ASM disk group that is under
FAST VP control.
Device labels make
devices easy to identify

Step 3: Create storage tiers
Creating a storage tier involves specifying the following attributes:
The tier name, which uniquely identifies the storage tier
The disk technology on which the tier will reside
The RAID protection type for the tier
The name(s) of the virtual pool(s) that belong to the tier
The use case defines three tiers. Figure 8 shows the SMC FAST Configuration Wizard window
for creating these tiers. The wizard proposes a set of tiers based on the virtual pools that
reside on the Symmetrix VMAX array. For the use case, three virtual pools reside on the array.
The default tiers proposed by the wizard, and based on these pools, were accepted.
Users can also define tiers manually, and can rename tiers, as required.

Figure 8. Creating storage tiers
Note When possible, it is best practice to configure each FAST VP tier with a single virtual pool, or
else to ensure that all pools within a tier have the same overall performance capabilities. For
example, if a tier contains one pool spread over 128 drives and another pool spread over 16
drives, the number of spindles making up the underlying storage will be different for each
pool. This could result in unbalanced use of each pools performance capabilities.
Step 4: Create FAST policies
Creating a FAST policy involves specifying the following attributes:
The policy name, which uniquely identifies the policy
The name of each tier to be added to the policy
The upper limit (percent) of each tier that an associated storage group can occupy
under the specified policy
Figure 9 shows the creation of the FAST policy for the use case, using the FAST Configuration
Wizard.


Figure 9. Creating a FAST policy
This policy was set up to test whether, by adding a small amount of EFD storage, FAST VP is
able to effectively balance the workload across the policy tiers while maintaining or
improving levels of service.
The percentages applied ensure that the cost per GB of provisioned storage remains the
same, while 67 percent of the high-performing FC storage initially provisioned can be
reclaimed for use elsewhere.
Step 5: Associate storage groups with FAST policies
To apply a FAST policy to a schema, the schemas storage group must be associated with the
FAST policy. Figure 10 shows the association of policy OracleRAC with storage group
OracleDataASM.

Figure 10. Associating a FAST policy with a storage group

Step 6: Configure FAST VP monitoring and move windows
Time windows are used by FAST VP to specify when data can be collected for performance
analysis and when data movements can be executed. FAST VP shares monitoring windows
with FAST and Symmetrix Optimizer. However, FAST VP requires a separate data movement
window.
Performance time windows define the days and times when FAST VP performs analysis. Data
movement windows define the days and times when FAST VP moves data between tiers.
It is recommended that both monitoring and data movement windows are configured to be
always open, so that FAST VP can use the most recent analysis and metrics to optimize data
placement.
Figure 11 shows the definition of a monitoring window for the use case.

Figure 11. Creating time windows
Note If you require different analysis and movement periods, check with your local EMC Account
Team to ensure that sufficient EFD resources are available to match your workload
requirements.

Solution testing and validation
FAST VP versus manual tiering
Traditionally tiering has been a manual and time-consuming process. FAST VP eliminates the
repetitive tasks associated with manual tiering, and significantly reduces the time required to
configure and manage a tiered storage environment.
As shown in Figure 13, manual tiering involves a repeated process taking 9 hours or more to
complete each time. In contrast, configuring FAST VP is a one-off process taking 40 minutes
or less, and the underlying storage is then continually and automatically tuned based on the
FAST policy applied and the performance needs of the application.

Figure 12. Manual tiering versus FAST VP
Note Times stated for configuring FAST VP are conservative measurements, not based on using the
SMC FAST Configuration Wizard. For details on configuring FAST VP, see the Configuring EMC
Symmetrix FAST VP section of this white paper.

Before configuring any FAST policy, administrators need to understand the overall workload
characteristics of their application. To assist this process, you can use SPA snapshots and
trend views to profile your storage groups. Your local EMC Account Team can also help with
understanding workloads before applying any FAST policy.
For further information, see Implementing Fully Automated Storage Tiering for Virtual Pools
(FAST VP) for EMC Symmetrix VMAX Series ArraysTechnical Notes.
FAST VP test procedure
In order to test the effects of FAST VP, a Swingbench Order Entry workload was generated and
run against schema SOE1. Schema SOE2 was left idle to provide a more realistic skew in the
dataset.
Initially, all storage allocated for the two schemas resided on FC devices.
Workload was generated against schema SOE1 for two hours to ensure that it was
running a steady transaction rate.
FAST monitoring and move windows were opened to enable FAST VP to monitor and
move data.
The performance of the storage and database was monitored, and the average
database service response times, host I/Os per second (IOPS), and database
transactions per minute were recorded.
Once the storage group was compliant with the applied FAST policy, the load was
continued to ensure that the performance levels observed were sustained.
The results of these tests are detailed in the following sections.
Note Swingbench does not enable tailoring of the workload to access new data more frequently
than old data. Data was therefore accessed randomly and evenly. The idle schema (SOE2) was
used to represent a more real-world environment where some objects are less active and not
often accessed.
In the test results, a transaction refers to the metric value recorded by the Automatic Workload
Repository within the Oracle RAC database and not the user-defined transactions recorded in
the Swingbench workload.

FAST VP rebalancing across tiers
In Figure 13, the Solutions Enabler CLI shows how FAST VP has rebalanced the storage group
data across the three tiers based on the applied policy.

Figure 13. Storage tier rebalancing by FAST policy
Figure 14 shows the sub-LUN tiering of devices 010F and 0111 after the FAST policy was
applied and the FAST VP algorithms have optimized data placement.

Figure 14. Sub-LUN tiering of devices 010F and 0111 after the FAST policy was applied
Amount of storage in
each tier after FAST VP
The majority of tracks for device
0111 have downtiered to SATA

Device 010F belongs to the highly-active dataset for schema SOE1 and is now spread across
three FAST VP tiers. Device 0111 belongs to the idle data set for schema SOE2 and is now
spread across two tiers. With further rebalancing, all tracks on device 0111 will migrate to the
RAID6_SATA tier if they remain inactive.
FAST VP effects on database transactions per minute
The Swingbench instance was configured with 400 users, delivering over 142,000
transactions per minute, with the host CPU running at 60-70 percent utilization. Figure 14
shows the database transactions per minute before and after applying the FAST policy.

Figure 15. Database transactions per minute before and after the FAST policy was applied
The chart shows that the number of transactions processed increased by 12.7 percent when
FAST VP was monitoring and optimizing the storage to meet the changing demands of the
workload.
FAST VP effects on Oracle service response time
Figure 16 shows the average Oracle service response times before and after the FAST policy
was applied.

Figure 16. Average Oracle service response times

After the FAST policy was applied and FAST VP had optimized data placement, the average
service response time for the database improved by 26 percent. This is a result of FAST VP
identifying the busiest data and relocating it to EFDs, which are more responsive than FC
drives.
Note The service response time is the total time taken for a database request to traverse through
the entire application stack.
FAST VP effects on average host IOPS
Figure 17 shows the average host IOPS before and after the FAST policy was applied. An
increase of 8.3 percent was observed after FAST VP optimization.

Figure 17. Average host IOPS before and after the FAST policy was applied
Summary of FAST policy effects
The OracleRAC FAST policy defines the following upper limits of each FAST tier that storage
group OracleDataASM could occupy:
RAID 5 (3+1) EFD 4% RAID 5 (3+1) FC 33% RAID 6 (6+2) SATA 63%
The benefits of applying the policy include:
With an allocation of only 4 percent EFD storage, FAST VP was able to effectively
balance the load on the devices in order to deliver more IOPS with a lower average
response time than the FC-only environment, as shown in Figure 18:
A 26 percent improvement in database service response times
A 12.7 percent increase in the number of transactions processed per minute
An 8.3 percent increase in the number of IOPS performed at the storage level
More efficient use of the higher-performing EFDs, as only active data now resides
there. The 4 percent usage limit for EFDs equates to 81 GB, which is less than half of a
single 200 GB EFD.
A reduction in the amount of data now residing on FC storage, freeing up capacity on
these high-performing devices for use by other applications.

No additional cost per GB to achieve the performance improvements observed (see
Figure 18).
OPEX savings as storage is always automatically and optimally tuned to meet the
demands and access patterns of the application, within the tier usage limits specified
by the user-defined FAST policy. No manual storage tuning is necessary.

Figure 18. Key performance indicators and cost per GB before and after FAST VP

Conclusion
Summary
This white paper describes an automated tiered storage solution for Oracle Database 11g
RAC, enabled by EMC Symmetrix VMAX storage with FAST VP.
FAST VP tunes storage from underlying Virtual Provisioning thin pools, moving data at the
sub-LUN level between pools with different performance characteristics. It automatically
identifies and moves data based on its performance needs, ensuring that the storage is
always tuned to match the current application performance requirements, increasing storage
efficiency, and reducing energy and space requirements.
Symmetrix customers now have a new option for moving from an all-FC drive configuration to
a tiered configuration with a mix of two or more disk technologies (for example, SATA, FC, and
EFD). FAST VP provides automated movement of data at the sub-LUN level, enabling
customers to create policies that guarantee service levels for the life of an application and
that ensure maximum efficiency from their storage resources.
Findings
The key findings of the testing performed for the solution include:
Simplified management
Administrators can configure a FAST VP environment in minutes.
Automatic storage tuning by FAST VP radically reduces the time required to manage
a tiered storage environment.
Set and forget
FAST VP automatically tunes the underlying storage to meet the changing
requirements of the database, without any manual storage tuning.
Nondisruptive data movement
FAST VP data movements are nondisruptive and transparent.
Dynamic performance scaling
By adding only 4 percent EFD storage, a 12.7 percent increase in database
transactions per minute and a 26 percent improvement in database service
response times was achieved without any manual host or application tuning.
More efficient use of higher-performing storage devices
FAST VP automatically identifies inactive or seldom active data and relocates it to
lower-cost SATA devices, freeing capacity on higher-performing tiers for use by
other applications.
Introducing a small percentage of EFD storage enables predictable high levels of
performance that are continually tuned to meet application needs.

Reduced total cost of ownership
By automatically monitoring and re-tiering data, FAST VP significantly reduces the
cost and complexity of storage management.
Re-tiering inactive data to energy-efficient SATA reduces power consumption.
Introducing SATA devices into the storage environment, without negatively
effecting performance, helps contain storage footprint as applications grow.

References
White papers and technical notes
For additional information, see the EMC white papers and technical notes listed below.
Maximize Operational Efficiency for Oracle RAC with EMC Symmetrix FAST VP
(Automated Tiering) and VMware vSphereAn Architectural Overview
New Features in EMC Enginuity 5875 for Open Systems Environments
Implementing Fully Automated Storage Tiering for Virtual Pools (FAST VP) for EMC
Symmetrix VMAX Series ArraysTechnical Notes
Implementing Virtual Provisioning on EMC Symmetrix VMAX with Oracle Database 10g
and 11gApplied Technology
Best Practices for Fast, Simple Capacity Allocation with EMC Symmetrix Virtual
ProvisioningTechnical Notes
EMC Symmetrix VMAX Virtual Provisioning Space Reclamation and Application
ConsiderationsApplied Technology
EMC Symmetrix Virtual ProvisioningApplied Technology
Implementing EMC Symmetrix DMX-4 Flash Drives with Oracle DatabasesApplied
Technology
FAST VP Theory and Practices for Planning and PerformanceTechnical Notes
Product documentation
For additional information, see the following EMC product document.
EMC Solutions Enabler Symmetrix Array Controls CLI Version 7.2 Product Guide
Other documentation
For additional information, see the documents listed below.
Oracle Grid Infrastructure Installation Guide 11g Release 2 (11.2) for Linux
Oracle Real Application Clusters Installation Guide 11g Release 2 (11.2) for Linux
Oracle Database Installation Guide 11g Release 2 (11.2) for Linux
Oracle Database Storage Administrator's Guide 11g Release 2 (11.2)
Oracle Real Application Clusters Administration and Deployment Guide 11g Release 2
(11.2)
Oracle Clusterware Administration and Deployment Guide 11g Release 2 (11.2)

h8029 Maximize Oracle Rac Symmetrix Fast VP WP

Hochgeladen von

Dokumentinformationen

Originaltitel

Copyright

Verfügbare Formate

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

Verfügbare Formate

h8029 Maximize Oracle Rac Symmetrix Fast VP WP

Hochgeladen von

Copyright:

Verfügbare Formate

White Paper

EMC GLOBAL SOLUTIONS

FAST VP automatically monitors and moves data

Fully Automated Storage Tiering for Virtual Pools (FAST VP)

vStorage APIs for Array Integration (VAAI)enabling storage administrators to

/Snapscreating capacity-efficient replicas for application

5875, and associated management tools have

4.1 server with:

Das könnte Ihnen auch gefallen