H11402 Bidirectional Automated DR SAP WP

White Paper
EMC TRANSFORMS IT FOR SAP: FULLY

AUTOMATED DISASTER RECOVERY
EMC Symmetrix VMAX 10K, EMC VNX5300, EMC RecoverPoint,
VMware vCenter Site Recovery Manager
Minimize SAP business processes downtime with a quick and
automated recovery
Simplify and accelerate nondisruptive disaster recovery testing
Increase ROI by leveraging existing landscape investments in DR
EMC Solutions Group

Abstract
This white paper demonstrates how EMC RecoverPoint can be integrated
seamlessly with VMware vCenter Site Recovery Manager to perform
seamless recovery and storage migration between heterogeneous storage
systems such as EMC VNX and EMC Symmetrix VMAX in SAP environments.
This solution increases Return On Investment (ROI) by using existing
investments in implementing a disaster recovery solution.
January 2013
Copyright 2013 EMC Corporation. All Rights Reserved.

EMC believes the information in this publication is accurate as of its
publication date. The information is subject to change without notice.
The information in this publication is provided as is. EMC Corporation makes
no representations or warranties of any kind with respect to the information in
this publication, and specifically disclaims implied warranties of
merchantability or fitness for a particular purpose.
Use, copying, and distribution of any EMC software described in this
publication requires an applicable software license.
For the most up-to-date listing of EMC product names, see EMC Corporation
Trademarks on EMC.com.
All trademarks used herein are the property of their respective owners.
Part Number H11402
EMC Transforms IT for SAP: Fully Automated Disaster Recovery

EMC Symmetrix VMAX 10K, EMC VNX5300, EMC RecoverPoint, VMware vCenter Site Recovery
Manager
Table of contents
Executive summary............................................................................................................................... 5
Business case .................................................................................................................................. 5
Solution overview ............................................................................................................................ 6
Key results ....................................................................................................................................... 7
Introduction.......................................................................................................................................... 8
Purpose ........................................................................................................................................... 8
Scope .............................................................................................................................................. 8
Audience ......................................................................................................................................... 8
Terminology ..................................................................................................................................... 8
Technology introduction..................................................................................................................... 10
Overview ........................................................................................................................................ 10
EMC RecoverPoint .......................................................................................................................... 10
EMC Symmetrix VMAX 10K ............................................................................................................. 10
Virtual Provisioning ................................................................................................................... 10
EMC VNX5300 ................................................................................................................................ 11
EMC Unisphere .............................................................................................................................. 11
VMware vCenter Site Recovery Manager ......................................................................................... 11
SAP ERP 6.0 ................................................................................................................................... 12
Basics of RecoverPoint replication ..................................................................................................... 13
Replication modes ......................................................................................................................... 13
Main components .......................................................................................................................... 13
EMC RecoverPoint appliance ..................................................................................................... 14
Write splitters ............................................................................................................................ 14
Replication sets and consistency groups ................................................................................... 14
Storage Replication Adapter ...................................................................................................... 14
Journal volumes ........................................................................................................................ 14
Initial replication ............................................................................................................................ 15
Solution architecture .......................................................................................................................... 16
Overview ........................................................................................................................................ 16
Physical design .............................................................................................................................. 16
Hardware resources ....................................................................................................................... 17
Software resources ........................................................................................................................ 18
Virtual machine resources.............................................................................................................. 19
SAP application environment ......................................................................................................... 20

Manager
Solution configuration ........................................................................................................................ 21

Overview ........................................................................................................................................ 21
Configuring replication ................................................................................................................... 21
Enabling bi-directional replication ............................................................................................. 21
Customization specific to SAP ........................................................................................................ 22
Heterogeneous test environment............................................................................................... 22
Replication mode ...................................................................................................................... 23
Startup priorities ....................................................................................................................... 24
Network settings ....................................................................................................................... 24
/etc/hosts file ........................................................................................................................... 24
SAP shared folders .................................................................................................................... 25
SAP startup sequence ............................................................................................................... 25
SAP user logon .......................................................................................................................... 26
Load-generation environment.................................................................................................... 27
Bi-directional recovery testing ................................................................................................... 27
System integrity and quality control .......................................................................................... 27
Test and validation ............................................................................................................................. 29
Overview ........................................................................................................................................ 29
Test scenario 1: Nondisruptive failover testing ............................................................................... 29
Test scenario 2: Disaster recovery .................................................................................................. 32
Test scenario 3: Failback ................................................................................................................ 34
Test scenario 4: Point-in-time recovery ........................................................................................... 37
Conclusion ......................................................................................................................................... 42
Summary ....................................................................................................................................... 42
Findings ......................................................................................................................................... 42
References.......................................................................................................................................... 44
White papers ................................................................................................................................. 44
Product documentation.................................................................................................................. 44
SAP notes ...................................................................................................................................... 44
Appendix A: RecoverPoint replication modes ..................................................................................... 45
Continuous data protection............................................................................................................ 45
Continuous remote replication ....................................................................................................... 46
Concurrent local and remote replication......................................................................................... 47
Appendix B: Setting up replication ..................................................................................................... 49
Preparing VMAX 10K and VNX5300 for RecoverPoint connectivity .................................................. 49
Configuring RecoverPoint ............................................................................................................... 52
Configuring SRM ............................................................................................................................ 54

Manager
Executive summary
Business case
For most businesses, the growth in eCommerce and focus on consumers experience
has led to more opportunity and competition. Simply stated, businesses have
expanded beyond traditional geographic boundaries and markets, interacting with
consumers directly. Capitalizing on this opportunity requires:
Responsiveness: Consumers have choices. If they are not satisfied with the
service level they receive, they go elsewhere to find what they need, leading to
severe productivity loss.
Market and customer intelligence: Availability of data and information is critical

to any business for understanding customer behaviors and preferences to
maintain competitive advantages.
Supply, demand, and order fulfillment management: What may seem to be a

simple transaction to an eCommerce consumer in reality necessitates the
seamless availability of a federated landscape of dependent business
applications.
Ultimately, these business requirements translate into the challenges for SAP IT
including:
Scalability: Providing increased performance to handle increasing transactional

processing volumes while maintaining access to increasing amounts of data.
For example, customers who have started out with smaller legacy systems for
SAP environments can opt to scale outward to more powerful storage solutions
like EMC Symmetrix VMAX .
Disaster Recovery: How to respond to increasing pressures to mitigate planned

and unplanned downtimes as the growth in production operations gives rise to
the need for new levels of protection against disasters or logical corruption.
Additionally, from the standpoint of business continuity, Information Security
Management Systems (ISMS) standards require regular testing of backup and
recovery solutions to ensure effectiveness in the event of a disaster.
Unfortunately, regular failover testing is rather difficult for businesses with very
low downtime tolerance.
Traditional disaster recovery (DR) solutions, such as an offsite tape restore, may no
longer be adequate to address increasing recovery time objectives (RTO) and recovery
point objectives (RPO). Storage replication solutions can satisfy these requirements.
However, maintaining an idle data center for disaster recovery may not be suitable for
some organizations either. These challenges demand a solution that offers effective,
affordable, and efficient disaster recovery protection for these critical SAP business
functions, without wasting any previous investments.

Manager
Solution overview
This solution provides customers with a strategy that enables DR, using VMAX and
VNX series interchangeably as production and DR storage for SAP applications.
The solution team designed the architecture for central SAP installations, distributed
implementations (database, central services, and application are on separate virtual
machines), and heterogeneous EMC storage solutions such as VNX and VMAX
families.
We implemented this using a Symmetrix VMAX 10K storage array together with an
existing VNX5300 storage array to form a bi-directional disaster recovery solution. We
achieved this by replicating both storage devices to each other with EMC
RecoverPoint and designing automated recovery plans using VMware vCenter Site
Recovery Manager (SRM).
This solution demonstrates the following capabilities:
Fast and seamless failover

This solution allows quick and fully automated failover on multiple systemsin
the correct order, in a matter of minutes, and with a few clicks of a button. This
allows enterprises to be resilient against disasters and to regain productivity
sooner.
Automated failback
This solution facilitates automated reversal of the data centers roles after
failover. After the original production site is operational, failback is just a
matter of performing another failover.
Point-in-time (PIT) recovery without database restore

The solution can perform PIT recovery without the need to restore and roll
forward the entire databases just before a logical corruption occurs.
Nondisruptive failover testing

This solution allows online testing by restoring a snapshot and encapsulating it
in a private network during the test. This procedure effectively reduces the need
for disruptive testing. Information security audit requirements are now easier to
manage.

Manager
Key results
This solution significantly improves business agility and scales out for many common
needs for SAP production and nonproduction systems:
The bi-directional replication strategy enabled by EMC RecoverPoint allows two

sites to mutually replicate and serve as production sites and recovery sites of
each other. This strategy reuses the existing storage systemsVNX in this case,
to effectively increase ROI.
The integration of EMC RecoverPoint and VMware vCenter SRM provides a

seamless end-to-end recovery strategy, which requires minimal downtime for
failover and failback, and point-in-time recovery operations.
This solution provides an environment for performing regular recovery testing

without any downtime, to be compliant with the security audit requirements.
Customers can leverage this solution to migrate their SAP environments

between heterogeneous system environments.

Manager
Introduction
Purpose
This white paper demonstrates how EMC RecoverPoint can be integrated seamlessly
with VMware vCenter SRM to perform seamless recovery and storage migration
between EMC VNX and EMC Symmetrix VMAX systems.
It describes the benefits in the following scenarios:
Scope
Audience
Terminology
After migrating business data from VNX to Symmetrix VMAX 10K, relocating the
existing VNX5300 from the production site to the recovery site
Configuring two active production data centers to mutually protect each other
To achieve the stated purpose, the document covers the following:
Components used and the key concepts involved in implementation
The reference architecture of the bi-directional disaster recovery system using

EMC Symmetrix VMAX 10K and VNX5300
Installation and configuration procedures for EMC RecoverPoint and VMware

Site Recovery Manager
Use-case functionality test results
This white paper is intended for SAP administrators, storage administrators, IT

architects, and disaster recovery managers who:
Have virtual SAP environments on existing storage and plans to migrate to a

VMAX storage family, while keeping good use of the original storage after
migration.
Have existing virtual SAP environments on Symmetrix VMAX storage solutions

and aim to find more economic strategies in implementing disaster recovery
solutions.
Table 1 includes the terminology used in this white paper.

Table 1.
Terminology
Term
Definition
Asynchronous replication
Process of committing data writes when the data has

been sent out for replication without waiting for
acknowledgement from the receiving side.
Asynchronous replication provides crash-consistent
protection and recovery to specific points-in-time, with
a small RPO.
CDP
Continuous data protection.
CLR
Concurrent local and remote replication.

Manager
Term
Definition
Consistency group
RecoverPoint configuration that logically groups

replication sets that must be consistent with one
another. The consistency groups ensure that updates to
the production volumes are written to the replicas in a
consistent write order and that the replicas can always
be used to continue working from or to restore the
production source.
CRR
Continuous remote replication.
Deduplication
A specialized data compression technique for

eliminating duplicate copies of repeating data.
Gatekeepers
Small, three-cylinder devices (approximately 3 MB)

used by the solutions enabler to send control
commands to the Symmetrix VMAX array.
Journal volume
Timestamped application writes stored in RecoverPoint

journals for later recovery to selected points-in-time.
KPI
Key performance indicator.
PIT
Point-in-time; usually refers to a system state that

occurred at a specific date and time in the past.
Replica volume
Volume that contains the replicated image from a

production site.
Replication set
RecoverPoint configuration that defines an association

between a production volume and the local and/or
remote volumes to which it is replicating.
Repository volume
Volume that stores configuration information about

RecoverPoint appliances (RPAs) and consistency
groups. This volume enables a surviving RPA to take
over the replication activities of a failed RPA from the
same cluster.
RPA
RecoverPoint appliance.
RPO
Recovery point objective.
RTO
Recovery time objective.
Synchronous replication
Process of committing data writes only after the

receiving side acknowledges the data. This ensures that
writes between two sites are always consistent.

Manager
Technology introduction
Overview
This solution uses the key components described in Table 2.

Table 2.
Key components
Component
Role
EMC RecoverPoint
Storage replication device
EMC Symmetrix VMAX 10K
Production site storage
EMC VNX5300
Recovery site storage
EMC Unisphere
Storage management application
VMware vCenter Site Recovery Manager
DR failover coordination
SAP ERP 6.0
Enterprise application software
Note
While Symmetrix VMAX and VNX models were used in the solution testing,
this solution is generally applicable to other models as well.
EMC RecoverPoint
EMC RecoverPoint is an advanced enterprise-class disaster recovery solution

designed with the performance, reliability, and flexibility required for enterprise
applications in heterogeneous storage and server environments. It provides bidirectional local and remote data replication, without distance limits.
EMC Symmetrix
VMAX 10K
The Symmetrix VMAX 10K system is built on the highly scalable EMC Virtual Matrix
Architecture, which enables the storage to grow seamlessly and cost-effectively from
an entry-level, single-bay, single-engine configuration to a six-bay, four-engine
system with 512 GB cache memory, 1,080 drives, and up to 1.5 PB usable capacity.
Built for simplicity and ease-of-use, Symmetrix VMAX 10K is 100 percent virtually
provisioned, with storage tiering automatically managed by EMC Fully Automated
Storage Tiering for Virtual Pools (FAST VP). Symmetrix VMAX 10K supports Flash,
Serial Attached SCSI (SAS), and Near-Line SAS (NL-SAS) drives with RAID 1, 5, and 6
protection options.
Symmetrix VMAX 10K systems are preconfigured and ready for same-day installation
and startup. They support all EMC Symmetrix monitoring and management tools,
including the latest enhanced Symmetrix Management Console, which provides
simpler installation and management with smart wizards.
The integrated RecoverPoint splitter for Symmetrix VMAX 10K enables local and
remote replication for flexible and efficient RPOs and RTOs.
Virtual Provisioning
EMC Virtual Provisioning simplifies storage configuration and management,
improves capacity utilization, and enhances performance by enabling the creation of
thin devices. Thin devices present more capacity to applications than what is
physically allocated in the storage array.

Manager
10
Physical storage that supplies disk space to the thin devices comes from a thin pool.
The thin pool is composed of data devices that provide the actual physical storage.
You can add or remove data devices to grow or shrink thin pools nondisruptively.
Virtual Provisioning supports local and remote replication with RecoverPoint on VMAX
10K.
Virtual Provisioning is described in detail in EMC Solutions Enabler Symmetrix Array
Controls CLI Version 7.3 Product Guide.

EMC VNX5300
EMC VNX5300 storage array is a member of the VNX series storage platforms, which
delivers innovation and enterprise capabilities for file, block, and object storage in a
scalable, affordable, and easy-to-use solution. Designed to meet the highperformance, high-scalability requirements of midsize and large enterprises, the VNX
series enables enterprises to grow, share, and cost-effectively manage multiprotocol
environments.
The VNX series is powered by quad-core processors, making it two to three times
faster than its predecessor. The VNX processors support the demands of advanced
storage capabilities such as Virtual Provisioning, compression, and deduplication.
VNX arrays incorporate the RecoverPoint splitter, which supports unified file and
block replication for local data protection and disaster recovery.
EMC Unisphere
EMC Unisphere is the central management platform for EMC Symmetrix VMAX and
VNX series, providing a single combined view of file and block systems, with all
features and functions available through a common interface. Unisphere is optimized
for virtual applications and provides industry-leading VMware integration. It
automatically discovers virtual machines and VMware ESX servers and provides
end-to-end, virtual-to-physical mapping.
VMware vCenter
Site Recovery
Manager
VMware vCenter SRM is a disaster recovery framework that integrates with EMC
RecoverPoint to automate recovery of VMware datastores so that it becomes as
simple as pressing a button.
SRM is an extension to VMware vCenter that enables integration with array-based
replication, discovery, and management of replicated datastores, and automated
migration of inventory from one vCenter to another. SRM does not replicate any data,
but, instead, leverages an external replication solution such as RecoverPoint. SRM
servers coordinate the operations of the replicated storage arrays and vCenter servers
on the production and recovery sites so that, as virtual machines on the production
site are shut down, virtual machines on the recovery site start up and resume the
responsibility of providing the same services, using the data replicated from the
production site.
Migration of protected inventory and services from one site to the other is controlled
by a recovery plan that specifies the order in which virtual machines are shut down
and started up, the computing resources that are allocated, and the networks they
can access. Integrated with RecoverPoint, SRM also enables testing of recovery plans,
using a temporary copy of the replicated data, in a way that does not disrupt ongoing
operation on either site.

Manager
11
SAP ERP 6.0
SAP ERP 6.0, powered by the SAP NetWeaver technology platform, is a world-class,
fully integrated enterprise resource planning (ERP) application that fulfills the core
business needs of midsize and large enterprises across all industries and market
sectors. SAP ERP 6.0 delivers a comprehensive set of integrated, cross-functional
business processes and can serve as a solid business-process platform that supports
continued growth, innovation, and operational excellence.

Manager
12
Basics of RecoverPoint replication

Replication modes
RecoverPoint offers different modes of replication, as shown in Figure 1.
Continuous Data
Protection (CDP)
PRD
Continuous Remote
Replication (CRR)
PRD
Concurrent Local
and Remote
Replication (CLR)
Local
replica
Remote
replica
Local
replica
Remote
replica
PRD
*PRD Production volume

Figure 1.
RecoverPoint data replication options
Types of data replication in RecoverPoint include:
Continuous data protection (CDP) for local replication
Continuous remote replication (CRR) for remote protection
Concurrent local and remote replication (CLR) for combined local and remote
protection
Appendix A: RecoverPoint replication modes provides more details.

Main components
The following components enable this technology:
RecoverPoint appliance (RPA)
Write splitters
Replication sets and consistency groups
RecoverPoint Storage Replication Adapter (SRA)
Journals

Manager
13
EMC RecoverPoint appliance

RecoverPoint is available as a physical appliance. This enables support of large
amounts of information stored across heterogeneous environments. This out-of-band
approach enables RecoverPoint to deliver continuous replication. RPAs run the
RecoverPoint software and manage all aspects of data replication. For local
replication, a cluster configuration of two or more active RPAs is deployed. This
supports immediate switchover to another appliance if one of the RPA nodes in a
cluster goes down. For remote replication, a RecoverPoint cluster is deployed at both
sites.
RPAs use powerful deduplication, compression, and bandwidth reduction
technologies to minimize the use of bandwidth and dramatically reduce the time lag
between writing data to storage at the source and target sites.
Write splitters
RecoverPoint monitors writes using a technology called write splitters, which ensure
that all host writes to a protected volumes are tracked and a copy is sent to local
RecoverPoint appliances. RecoverPoint supports three types of write splitters in
general: array write splitters, intelligent fabric-based write splitters, and host-based
write splitters.
In this solution, RecoverPoint uses an array-based write splitter that operates in each
front-end adapter of the Symmetrix VMAX 10K array, as well as inside the storage
processors of an EMC VNX storage array.
Replication sets and consistency groups
Regardless of the mode chosen, RecoverPoint allows you to define an association
between a production volume and the local and/or remote volumes to which it is
replicating, called a replication set.
These replication sets, which must be consistent with one another, are logically
grouped into consistency groups. The consistency groups ensure that updates to the
production volumes are written to the replicas in a consistent write order and that the
replicas can always be used to continue working from or to restore the production
source.
Storage Replication Adapter
The EMC RecoverPoint SRA for VMware vCenter Site Recovery Manager provides
integration of SRM with RecoverPoint. The RecoverPoint SRA supports discovery of
arrays attached to RecoverPoint and of consistency groups that are enabled for
management by SRM. It also supports SRM functions such as failover and failover
testing by mapping SRM recovery plans to the appropriate RecoverPoint actions.
Journal volumes
Journal volumes hold data that is waiting to be distributed to target replica volumes
and retain copies of the data previously distributed to the target volumes. Journal
volumes facilitate operational recovery to any point-in-time, as long as the relevant
journal history is retained.

Manager
14
Each consistency group has its own set of journal volumes, which allows for differing
retention periods across the consistency groups. Each consistency group has two or
three journal volumes assigned to it:
Initial replication
Local journal volumes, if configured
Remote journal volumes, if configured
Production journal volume, which stores information about the replication

process (called marking information) that is used to synchronize the replication
volumes at the two sites much more efficiently, when required
Regardless of the mode chosen, RecoverPoint allows you to define an association

between a production volume and the local and/or remote volumes to which it is
replicating to, called a replication set.
These replication sets can be logically grouped into consistency groups. The
consistency groups ensure that updates to the production volumes are written to the
replicas in a consistent write order and that the replicas can always be used to
continue working from or to restore the production source.
Each consistency group is in one of the following states:
Replicating: When the consistency group is enabled, the splitter is replicating

to the RPAs, and the RPAs are transferring to the replica journal/journals.
Marking: If the RPAs are unable to transfer to the replica journal, the location of
the changes is stored in the RPAs as well as on the production journal volume.
When contact with the remote site is reestablished, the remote replica is
synchronized, but only at those locations that were marked as having changed.
No marking/no replication: The splitter does not write to the RPAs. This can be
caused by a manually disabled consistency group or by a disaster on the
production site (no RPAs are available).

Manager
15
Solution architecture
Overview
This solution used a Symmetrix VMAX 10K storage array with an existing VNX5300
storage array to form a bi-directional disaster recovery environment. In this solution,
we replicated both storage devices to each other with EMC RecoverPoint and
designed automated recovery plans using SRM.
Physical design
The technical design adopted in this solution demonstrates an effective disaster

recovery technique using CLR technology. In our test scenario, the production
systems VM2 and VM3 were initially in a VNX storage solution on site C. Later, VM2
was migrated to VMAX storage on site A.
Figure 2 depicts the final architecture of the solution environment after migration.
Figure 2.
Solution architecture

Manager
16
We used two data centers to host SAP systems VM2 and VM3:
1.
We used two data centers:
Site A: Contained VMAX 10K; production site of VM2 and recovery site of
VM3.
Site C: Contained VNX5300; production site of VM3 and recovery site of

VM2.
2.
We configured the two pairs of RPAs contained in each data center (RPA1 and
RPA2 on site A, RPA3 and RPA4 on site C) as a cluster.
3.
We partnered each RPA cluster with an SRM server.
4.
We configured consistency groups on the RecoverPoint Management

application so that the datastores of each SAP system can be replicated
locally and remotely.
5.
We installed two SAP systems:
VM2 SAP IDES ERP, Oracle, Suse Linux, distributed SAP installation
VM2 was initially installed on the VNX5300 array and then migrated to the
VMAX 10K array. This DR solution is also used in migrating SAP systems in
VNX to VMAX storage.
VM3 SAP IDES ERP, SQL Server, Windows, central SAP system installation
VM3 was independently installed on VMAX, and then migrated to the VNX
array. This use case demonstrates bi-directional recovery.
6.
Hardware
resources
We integrated SRM with RPA and configured it to allow mutual failovers

between the two sites.
Table 3 details the hardware resources for the solution.

Table 3.
Solution hardware environment
Hardware
Quantity
Configuration
Storage (Production
site A)
EMC Symmetrix VMAX 10K with:

Single engine
42 GB cache memory
66 x 400 GB, 15K FC drives, plus
spares
13 x 200 GB Flash drives, plus spares
34 x 2TB SATA drives, plus spares
8 Gb FC connectivity
1 Gb network connectivity
Enginuity 5876.159.102

Manager
17
Hardware
Quantity
Configuration
Storage (site C)
EMC VNX5300 with:

58 x 600 GB, SAS drives, plus spares
9 x 200 GB Flash drives, plus spares
30 x 2 TB SATA drives, plus spares
8 Gb FC connectivity
1 Gb network connectivity
VNX Operating Environment
Block software version: 05.31
VMware ESXi
server
2 x quad-core CPUs, 2.93 GHz,

64 GB RAM
4 x 8 Gb FC HBA
2 x 1 Gb Ethernet
4 x 10 cores CPUs, 2.00 GHz

400 GB RAM
4 x 8 Gb FC HBA
2 x 1 Gb Ethernet
2 x quad-core CPUs, 1.86 GHz

16 GB RAM
2 x 4 Gb FC HBA
2 x 1 Gb Ethernet
Network switch
1 Gb Ethernet switches
FC switch
8 Gb/s FC switches, 2 per site
RecoverPoint
appliance
Gen 4, 2 RPAs per site
Software resources Table 4 details the software resources for the solution.
Table 4.
Solution software environment
Software
Version
Purpose
EMC Unisphere for VMAX
1.0.0.1018
Symmetrix VMAX configuration and

management tool
EMC Unisphere for VNX
1.1
VNX management software
EMC Solutions Enabler
7.4
Symmetrix VMAX management

software
EMC RecoverPoint
3.5
EMC replication software, installed

on each of the 4 RPAs
VMware vSphere
5.0.0 B623860
Hypervisor hosting all virtual

machines

Manager
18
Virtual machine
resources
Software
Version
Purpose
VMware vCenter
5.0.0 B755629
Management of VMware vSphere
VMware vCenter Site

Recovery Manager
5.0.1 B633117
Managing failover and failback of

virtual machines
EMC RecoverPoint adapter

for VMware vCenter Site
Recovery Manager
2.0
EMC software for integrating

RecoverPoint and SRM
SUSE Linux Enterprise

Server 11
2.6.32.24
Server operating system for SAP

VM2
Microsoft Windows Server
2008 R2 *64
Server operating system for SAP

VM3
Microsoft SQL Server
SQL Server 2008

R2
Database for SRM
Table 5 details the virtual machine resources for the solution.

Table 5.
Virtual machine environment
Role
Purpose
Quantity
Configuration
SAP VM2 ABAP SAP Central

Services server (ASCS)
Critical production
2 vCPUs, 8 GB RAM, SLES 11, 2

vNICs
SAP VM2 database server
Critical production

vNIC
SAP VM2 Application servers
Critical production

vNICs
SAP VM3 Central System
Non-critical production
16 vCPUs, 24 GB RAM, Windows

2008 R2, 2 vNICs
SRM site A application
SRM management system

2008 R2, 1 vNIC
SRM site A database

2008 R2, 1 vNIC
SRM site C application

2008 R2, 1 vNIC
SRM site C database

2008 R2, 1 vNIC
Load runner controller
Simulate user activity

2003 R2, 1 vNIC
Load runner generator
Simulate user activity

2003 R2, 1 vNIC

Manager
19
SAP application
environment
Table 6 details the SAP application environment for the solution.

Table 6.
SAP application environment
Specifications
VM2
VM3
Number of
datastores, type
2, VMFS
1, VMFS
Operating system
Suse Linux Enterprise Server

(SLES) 11
Windows 2008 R2
Database
application
Oracle 11g R2 database

server
SQL Server 2008 R2
Size
1 TB SAP data, 1.4 TB

allocated in 2 datastores
630 GB SAP data, 1 TB

allocated in 1 datastore
SAP application
SAP ERP 6.0 EhP 4 ABAP

IDES
SAP ERP 6.0 EhP 5 ABAP

IDES
Installation type
Distributed
Central

Manager
20
Solution configuration
Overview
Configuring
replication
Building this solution involves two major steps:
Replication configuration
SAP-specific customization
The storage configuration procedure is as follows:

1.
2.
3.
Prepare VMAX 10K and VNX5300 for RecoverPoint connectivity.

a.
Create LUNs and replica volumes.
b.
Provision RPA repository and gatekeepers.
c.
Tag all VMAX 10K devices to be protected by RecoverPoint.
Configure RecoverPoint.
a.
Install RecoverPoint.
b.
Configure RecoverPoint access to the splitters on the VMAX 10K and VNX
arrays.
c.
Create consistency groups.
d.
Enable bi-directional replication.
Configure SRM.
a.
Installing SRM virtual machines and SRA for integration to RPA.
b.
Establish connections between protected and recovery sites.
c.
Configure RecoverPoint array managers.
d.
Configure protection groups.
e.
Configure inventory mappings.
f.
Customize virtual machine recovery options.
g.
Customize recovery site IP addresses.
h.
Create recovery plan.
Appendix B: Setting up replication provides more detailed explanations of these

steps. For a detailed discussion of all the steps, refer to Site Recovery Manager
Administration vCenter Site Recovery Manager 5.1.
Enabling bi-directional replication
Adding a second consistency group with the roles of the data centers reversed (that is,
production volumes on site C are replicated to replica volumes on site A) effectively
enables bi-directional replication.
You can use single set of paired SRM (and RecoverPoint) sites to protect virtual
machines in both directions. Each site can serve as a production and recovery site
simultaneously, but for a different set of virtual machines. Bi-directional operation is
Manager
21
not limited to array-based replication. However, whenever you use an array basedreplication as in this solution, each of the arrays LUNs replicates in one direction
only. Two LUNs in the same array can replicate in different directions from each other.
Customization
specific to SAP
Some SRM customizations are valuable in automating recovery in an SAP

environment. A distributed SAP system requires the following steps during the
recovery startup:
1.
Change IP addresses of the SAP instances.
2.
Update hosts files to reflect internal IP settings.
3.
Mount the /sapmnt/<SID>/SYS/exe, /sapmnt/<SID>/SYS/global, and

/sapmnt/<SID>/SYS/profile folders of the ABAP Central Services instance to
the SAP application and database servers.
4.
Start the applications in this order:

a.
SAP ABAP Central Services instance
b.
Oracle database instance
c.
All SAP ABAP Application instances
This solution used the flexibility of the SRM recovery plan by inserting operating
system commands that perform the previous tasks during a failover. This fully
automates the startup process until all SAP systems are ready for login.
To achieve the desired level of automation, the test environment was prepared as
described in the following sections.
Heterogeneous test environment
We considered diversity when we built the test environment for a more extensive
scope and more credible results. This includes SAP releases, types of databases and
operating systems. We also tested the solution across components with non-uniform
time zones.

Manager
22
Replication mode
We set up CLR in both VM2 in the consistency group SiteA_PRD and VM3 in SiteC_TST,
as shown in Figure 3 and Figure 4.
Figure 3.
RecoverPoint topology showing VM2 in CLR mode
Figure 4.
RecoverPoint topology showing VM3 in CLR mode

Manager
23
Startup priorities
SRM allows virtual machines to be started up in a particular order by assigning a
priority. Priority 1 virtual machines are started up first, followed by priority 2 virtual
machines, and so on. We changed the priorities of the virtual machines as follows:
Priority 1ASCS; to make the SAP shared folders available for mounting
Priority 2Database instance; to ensure that the database is started before the
SAP application instances are started
Priority 3All application instances
Figure 5 shows the priority assignment.
Figure 5.
Virtual machine startup priorities
Network settings
We configured two static IP addresses for each virtual machine:
An internal 192.168.x.x IP address
A service IP for log in (SAP virtual machines only)
You can achieve this by configuring two network interface cards (NICs) for each virtual
machine or by creating a script that binds the internal IP addresses as virtual. We
used the former in this solution.
/etc/hosts file
SRM updates only the entry of the virtual machine's own hostname. Ensure that the
internal IP addresses are used before and after recovery. Steps are as follows:
1.
Create a hosts.i file as a template that maps the hostnames to the internal IP
addresses.
2.
Insert a script as a post-startup recovery step to reset the hosts files with
contents from the hosts.i file to keep consistency.

Manager
24
SAP shared folders

You can insert a mount command as a post-startup recovery step on every application
instance or add the mount point entries in the /etc/fstab file. This solution used the
latter technique to mount the /sapmnt/<SID>/SYS/exe, /sapmnt/<SID>/SYS/global,
and /sapmnt/<SID>/SYS/profile folders of the ASCS instance to the SAP application
servers.
SAP startup sequence
We inserted the following scripts as a post-startup recovery step on the recovered
virtual machine, with a 5-minute timeout. These scripts were assigned as follows:
On all virtual machines: change the host script:

cp /etc/hosts.i /etc/hosts
On SAPVM2DB (Priority 2): assign the startdb script:

mount -a
su - oravm2 -c "lsnrctl start"
su - vm2adm -c "startdb"
ON SAP virtual machines: assign the startsap script:

su - vm2adm -c "startsap"
Figure 6 shows the scripts that were applied to the automated application startup
after powering on.
Figure 6.
Scripts to automate application startup in post-power on steps

Manager
25
SAP user logon

All SAP instances communicate with internal IPs. Thus, the hosts file only included
internal IP addresses, as shown in Figure 7.
Figure 7.
Hosts file
For SAP logon, we created two entries; one is for logging in to the production site A,
the other for the recovery site C, as shown in Figure 8. Note that full IP addresses will
not be shown in Figure 8 and Figure 9.
Figure 8.
SAP logon
We created two logon groups using transaction code SMLG, which corresponds to the
two sets of IP addresses. Users can just switch the logon group used in the SAP GUI
after the failover, as shown in Figure 9.

Manager
26
Figure 9.
Logon groups created in VM2
Load-generation environment
We used LoadRunner for every test in this solution to run successive updates in SAP
tables and to simulate a real-world environment. The timestamps on the table entries
inserted by LoadRunner served as counterchecks of the timestamps recorded in SRM
or RecoverPoint, but at the SAP level. Timestamps from LoadRunner updates provide
more accurate application-level RTO and RPO key performance indicators (KPIs) for
testing. The tests were considered successful when the results from
SRM/RecoverPoint and SAP were consistent.
Bi-directional recovery testing
We performed the recovery tests on both VM2 and VM3 systems to validate bidirectional recovery. However, because the procedure was the same, for brevity this
white paper shows only the screenshots of the more complex VM2 system.
System integrity and quality control
We took timestamps carefully to ensure consistency among all components. We
performed SAP checks before and after testing for each scenario according to the
best practices to ensure that the failed-over system was healthy technically and
functionally.

Manager
27
We performed checks on the following transactions/commands:
SICK System installation check
SM51 Instance overview
SM50 Work process overview, per instance
SM21 System logs
ST22 ABAP dumps
DB02 Database management
SM12 Display and delete locks
SM13 Administrate (sic) Update Records
dfPh1 For Linux file systems
Note
These controls were observed, but the raw data and screenshots are not
shown in this paper.
df Ph displays file systems, but in a POSIX human-readable format.

Manager
28
Test and validation

Overview
In the context of operational efficiency and effectiveness, this section aims to test
and demonstrate the following capabilities:
Nondisruptive failover testing

This solution allows online testing by restoring a snapshot and encapsulating it
in a private network during the test. This procedure effectively reduces the need
for disruptive testing.
Disaster recovery
This solution allows the client to quickly automate the entire failover operation
with a few clicks of a button.
Failback testing
Because site A may be geographically closer to users, or applications running
previously on it are presumably more resource intensive, failback to the original
site is recommended. This operation reinstates failed-over systems on VNX to
their original servers as soon as possible. This design allows for rapid failback
to reduce downtime for the operation.
PIT recovery
The solution aims to show how easy it is to perform PIT recovery, eradicating
the need to restore and roll forward entire databases just before the logical
corruption occurs.
Test scenario 1:
Nondisruptive
failover testing
Test scenario
An organization originally has a central site C that contains SAP production systems
VM2 and VM3 on VNX5300. A second auxiliary site A that contains VMAX 10K was
built. VM2 will be failed over/migrated from site C (VNX5300) to site A (VMAX 10K)
using this solution.
To prepare the migration, we simulated disaster recovery failover on VM2 and VM3 on
site C while both were in production.
Test objectives
The test objectives were:
Performing failover testing on the recovery site without shutting down any
virtual SAP system
Testing bi-directional recovery
Achieving an RTO of 30 minutes

Manager
29
Test procedure
The test procedure was as follows:
1.
Ran load on VM2.
2.
Took timestamps and performed test recovery on VM2 and VM3. During
testing, we did a health check on both systems to verify integrity.
3.
Logged in to the SAP GUI, took the timestamp, and performed an SAP health
check.
4.
Checked the availability logs of VM2 and VM3 on site C to see if the
production systems had gone down during testing.
Findings
Table 7 shows the findings of this test.
Table 7.
Nondisruptive failover test findings
Success criteria
Findings
Result
VM2 and VM3 stay online
Availability logs showed that both systems

were available during failover
Passed
Test failover is 100% successful
Logs completed without errors
Passed
Test VM2 is operational
System health status validated availability

and integrity after completion
Passed
Duration of test recovery falls

within RTO
Timestamp in SRM showed recovery time

of 00:13:47
Passed
To summarize this test:
Initiation of the testing required only three steps. Additional manual steps were
not needed until completion, as shown in Figure 10.

Manager
30
1
1
12
3
1
Figure 10.
Test recovery execution
The recovery was successful in SRM, as shown in Figure 11.
Figure 11.
Final status of recovery plan in SRM after testing
Conclusion
A fully consistent bi-directional SAP DR was easily made available for DR testing in
less than 30 minutes while primary sites were available for production users with
continued DR replication.
Manager
31
Test scenario 2:
Disaster recovery
Test scenario
After a successful simulation in scenario 1, the same production SAP system VM2 on
site C was failed over to site A for planned migration/disaster recovery. To simulate
the disaster, we terminated the database processes and triggered disaster recovery
from site C to site A.
Test objectives
Performing failover on the recovery site
Test procedure
1.
Ran load on VM2.
2.
Took the timestamp and performed planned migration/disaster recovery on

VM2.
3.
Logged in to the SAP GUI, took the timestamp, and performed an SAP health
check.
4.
Checked the availability log of VM2 on the protected site.
Findings
Table 8.
Disruptive failover test findings
Success criteria
Findings
Result
Test failover is 100% successful
Passed
Recovered VM2 is operational

Passed
Duration of failover falls within

RTO
Timestamp in SRM showed recovery time

of 00:15:06
Passed
Initiation of the testing required only four steps. Additional manual steps were
not needed until completion, as shown in Figure 12.

Manager
32
41
31
Figure 12.
Recovery execution using VMware SRM

Manager
33
The recovery was successful in SRM and validated in SAP, as shown in

Figure 13.
Figure 13.
Final status of recovery plan executed in VMware SRM
Conclusion
The actual failover required only four steps. A fully consistent SAP DR was available in
less than 30 minutes. Manual intervention was not needed after the recovery was
initiated. This technique can be used to migrate systems from VNX to VMAX or vice
versa.
Test scenario 3:
Failback
Test scenario
VM2 was on site A after recovery in scenario 2. Assume that site C is now fully
operational after it has crashed. We then re-established protection between the two
sites, but with the roles reversed for VM2: Site A was the protected site, replicating to
recovery site C. VM3 remained unchanged.
Test objectives
Performing failover on the recovery site
Test procedure
1.
Issued reprotection on VM2 to reverse the direction of protection.
2.
Took the timestamp and recovered VM2 to the original site.
3.
Logged on to the SAP GUI, took the timestamp, and performed an SAP health
check.
Manager
34
4.
Checked the VM2 availability log in the protected site.
5.
Failed over to site C again to verify if the reprotection was effective.
Findings
Table 9.
Failback test findings
Success criteria
Findings
Result
Reprotection is 100%
successful
Passed
Failover is 100% successful
Passed
Recovered VM2 is operational

Passed
Duration of failback falls within

RTO
The timestamp in SRM showed:
Passed
Reprotection time of 00:01:48

Recovery time of 00:16:48
Initiation of reprotection required just four steps. Additional manual steps were
not needed until completion, as shown in Figure 14. Then another four steps
were needed for the failover.
1
12
31
41
Figure 14.
Reprotection procedure

Manager
35
Figure 15 shows how reprotection reversed the direction of replication so that further
post-processing was no longer necessary.
Figure 15.
SRM reprotection log showing successful reprotection in a reverse direction
Figure 16 compares the replication status in the RecoverPoint Manager application

after reprotection was triggered. The replication source and target volumes were
automatically changed by SRM as well. Arrows were added for reference.

Manager
36
After failover reprotection
After failback reprotection
Figure 16.
RecoverPoint VM2 topologies after failover and failback
Conclusion
Reprotection needed only four steps and the actual failback needed another four
steps. All DR changes to databases and file systems were reinstated to the original
production site in less than 30 minutes.
Test scenario 4:
Point-in-time
recovery
Test scenario
The system was corrupted because of an application logic error after a transport. We
performed a point-in-time (PIT) recovery to restore the system before the transport.
Test objectives
Performing PIT recovery without using a database restore

Manager
37
Test procedure
1.
Populated the table VBAK and captured a timestamped screenshot of the final
entries.
2.
Created a RecoverPoint bookmark. The screenshot taken previously

represented the system state of the bookmark, as shown in Figure 25.
3.
Populated the table VBAK further and captured a timestamped screenshot of

the final entries. These entries represented the erroneous data that caused
the PIT restore.
4.
Used RecoverPoint to restore VM2 to a bookmark:
5.
a.
Powered off VM2 in vCenter.
b.
Recovered production.
c.
Enabled image access of production.
d.
Resumed production.
e.
Powered on VM2 in vCenter.
Logged on to the SAP GUI, displayed the table VBAK, and captured a
timestamped screenshot of the final entries.

Manager
38
Findings
Table 10.
Point-in-time recovery test findings
Success criteria
Findings
Result
VM2 is operational after the PIT

restore
SAP log in and health check verified

application integrity
Passed
PIT restore is successful
Timestamps were consistent through the

screenshots but the erroneous data was
no longer found
Passed
Duration of restore falls within

RTO
The entire process completed in 00:25:00
Passed
Figure 17 shows the bookmark S1_T1 created in RecoverPoint.
Figure 17.
RecoverPoint journal bookmark
Figure 18 shows that the final bookmarked entry was Sales Doc 3451802 inserted at
23:12:15. Figure 18 shows the System Status window on the right-hand side.
Figure 18.
Table VBAK status during RecoverPoint bookmarking

Manager
39
Figure 19 shows that user activity resumed after bookmarking and the table was
populated further. We captured this image approximately 6 minutes after capturing
Figure 18.
Figure 19.
VBAK table after RecoverPoint bookmarking
Figure 20 shows the RecoverPoint status in the middle of the restore process. We
captured this image 10 minutes after capturing the previous image.
Figure 20.
RecoverPoint in the middle of the restore process

Manager
40
Figure 21 shows the status after the PIT recovery. Note that the result of the PIT
recovery is correct and consistent, even if the time zone of the Management GUI
(Figure 20, Windows time) is 8 hours ahead of the RecoverPoint server time and 12
hours ahead of the SAP GUI time. The latest entry was the same sales document, that
is, Sales Doc 3451802.
Figure 21.
VBAK status reverted to when the bookmark was created
Conclusion
Data recovery from a corruption can occur in minutes without the need of a full
database restore or file system recovery.

Manager
41
Conclusion
Summary
Findings
This solution offers effective, affordable, and efficient disaster recovery protection for
critical business functions without wasting previous investments. This strategy
significantly improves business agility and scales out for common SAP production
system and test system needs, providing:
Bi-directional replication of a central SAP environment and distributed SAP

environments between heterogeneous arrays by using the integrated
RecoverPoint splitters for Symmetrix VMAX 10K and VNX across two sites.
Rapid and automated failover for planned and unplanned maintenance,

enabled by integration of RecoverPoint and SRM.
Nondisruptive disaster recovery testing using RecoverPoint and SRM.
Point-in-time database recovery in a virtualized environment from bookmarked

RecoverPoint images.
Redeemed storage investment by reusing VNX as DR storage after migration to

Symmetrix VMAX.
The key findings of this proven solution are as follows:
This solution is able to manage the varying DR complexities of SAP

environments.
Initiating the failover testing requires only three steps. A fully consistent bidirectional SAP DR is easily made available for DR testing in less than 30
minutes, while the production systems remain completely undisrupted.
Additional manual steps are not required until completion.
The actual failover requires only four steps to initiate. A fully consistent SAP DR
is available in less than 30 minutes. Manual intervention is not required after
the recovery is initiated. This technique can be used for VNX/VMAX migrations.
Initiation of reprotection requires only four steps. The actual failback requires
another four steps. All DR changes to databases and file systems are reinstated
to the original production site in less than 30 minutes.
Data recovery from a corruption can occur in minutes without the need of a full
database or file system recovery using RecoverPoint point-in-time recovery.

Manager
42
The solution implements control measures to protect against human errors and
technical limitations:
We performed the tests using super user accounts for convenience, but the
solution is capable of role-based access control (RBAC) to manage
configuration and administrative activities in a more granular sense, if needed.
For more information, refer to Site Recovery Manager AdministrationvCenter
Site Recovery Manager 5.1.
While the entire failover process is automated, the trigger itself is manual. This
ensures that a failover can be initiated only after proper processes are followed,
executive approvals have been granted, and a state of disaster (or resumption
of normal operations) has been declared.
The failover operation can be triggered with one or two steps. However, we put
confirmation prompts in place to prevent inadvertent executions.

Manager
43
References
White papers
Product
documentation
SAP notes
For additional information, see the following white papers:
Business Continuity and Disaster Recovery for Oracle 11g Enabled by EMC
Symmetrix VMAX 10K, EMC RecoverPoint, and VMware vCenter Site Recovery
Manager
EMC Symmetrix VMAX RecoverPoint Splitter and Enhancements in Enginuity

5876
For additional information, see the following product documents:
EMC Solutions Enabler Symmetrix Array Controls CLI Version 7.3 Product Guide
Site Recovery Manager Installation and ConfigurationvCenter Site Recovery

Manager 5.1
Site Recovery Manager AdministrationvCenter Site Recovery Manager 5.1
For additional information, see the following SAP notes:
Note 923610: Memory parameter recommendations for banking systems
Note 1171650: Automated Oracle DB parameter check
Note 1431798: Oracle 11.2.0: Database Parameter Settings

Manager
44
Appendix A: RecoverPoint replication modes

Continuous data
protection
CDP continuously captures and stores data modifications locally, enabling local
recovery from any PIT with no data loss. CDP supports both synchronous and
asynchronous replications. Figure 22 illustrates the CDP workflow.
Figure 22.
CDP workflow
The CDP workflow is follows:

1.
The application server issues a write operation to a LUN protected by

RecoverPoint. The write is intercepted by the RecoverPoint array splitter.
2.
The splitter splits (duplicates and distributes the write request) to both the
protected volume and RPA.
3.
(3a) RPA acknowledges back to the splitter when receiving the write. (3b) At
the same time, RPA moves the data into the local journal volume, along with
the timestamp and any bookmarks for the write operation.
4.
After the write request has been secured in the journal, it is then applied to
the target replica volumes. The write order is preserved during distribution.

Manager
45
Continuous remote Continuous remote replication (CRR) supports synchronous and asynchronous
replication between remote sites and a wide area network (WAN). CRR supports
replication
synchronous replication when the remote sites are connected through Fibre Channel
(FC) and provides zero RPO. Asynchronous replication provides crash-consistent
protection and recovery to specific points-in-time, with minimal RPO.
Figure 23 illustrates the CRR workflow.
Figure 23.
CRR workflow
The CRR workflow is follows:

1.

2.
The splitter duplicates the write and sends it simultaneously to the protection
volume and to its local RPA.
3.
The local RPA receives the write operation:
3a. In asynchronous mode, acknowledgment is sent back from the local

RPA to the splitter.
3b. In synchronous mode, acknowledgement is instead delayed until the

write is received by the remote RPA. This request is bundled with other
writes, deduplicated (that is, redundant blocks are removed), sequenced,
Manager
46
and timestamped. The bundle is then compressed and transmitted with a

checksum for delivering to the remote RPA cluster.
3c. If the receiving RecoverPoint is not reachable, the write request is
temporarily stored on the production journal volume until the remote
RecoverPoint is accessible once again. The write entries are moved from the
production journal to the target journal and normal replication is resumed.
Concurrent local
and remote
replication
4.
When the package is received on a remote site, the receiving RPA verifies the
checksum first to ensure integrity and then decompresses the data.
5.
The remote RPA writes the data to the journal volume on the remote site.
6.
The write is then distributed to the target replica volumes through an RPA FC
connection after the data has been stored safely in the journal. The write
order is preserved during distribution.
CLR is a combination of CRR and CDP and provides concurrent local and remote data
replication. Figure 24 illustrates the CLR workflow.
Figure 24.
CLR workflow

Manager
47
The CLR workflow steps are as follows:

1.

2.
The splitter duplicates the write and sends it simultaneously to the protection
volume, and to its local RPA.
3.
The local RPA receives the write operation:
3a. In asynchronous mode, acknowledgment is sent back from the local

RPA to the splitter.
3b. The write request is then sent to the local journal and applied to the
local replica volume. This completes the CDP sequence.
3c. In synchronous mode, acknowledgement is instead delayed until the

write is received by the remote RPA. This request is bundled with other
writes, deduplicated (that is, redundant blocks are removed), sequenced,
and timestamped. The bundle is then compressed and transmitted with a
checksum for delivery to the remote RPA cluster.
3d. If the receiving RecoverPoint is unreachable, the write request is

temporarily stored on the production journal volume until the remote
RecoverPoint is accessible once again. The write entries are moved from the
production journal to the target journal and normal replication is resumed.
4.
When the package is received on a remote site, the receiving RPA verifies the
checksum first to ensure integrity, and then decompresses the data.
5.
The remote RPA writes the data to the journal volume on the remote site.
6.
The write is distributed to the target replica volumes through an RPA FC

connection after the data has been stored safely in the journal. The write
order is preserved during distribution. This completes the CRR sequence.
CDP is normally used for PIT recovery where business data or application logic has
been severely compromised. On the other hand, CRR is normally used for disaster
recovery. CLR thus implies that two replicas are made for every volume protected
(local and remote) to protect from both scenarios.

Manager
48
Appendix B: Setting up replication

Preparing VMAX
10K and VNX5300
for RecoverPoint
connectivity
Configuring the RecoverPoint splitter on the storage array requires the following
volumes to be provisioned beforehand.
Step 1: Creating LUNs and replica volumes
Create the following LUNs for RecoverPoint usage:
One repository volume (at least 3 GB) for each RPA cluster.
One replica volume each in the protected and recovery site for every
consistency group.
Three journal volumes (production journal, local journal, and remote journal
volumes) for every site.
Eight unique gatekeeper volumes each for RPA1 and RPA2.
Figure 25 shows how we created the gatekeeper volumes using Unisphere for the
VMAX GUI.
Figure 25.
Provisioning gatekeeper volumes using Unisphere
Consider the following during capacity planning and risk assessment:
The size of a replica volume must be equal to or greater than the size of the
protected volume. For example, if the protected volume is 500 GB, CDP or CRR
consumes an additional 500 GB and CLR consumes at least 1,000 GB,
excluding the journal volumes. This means a CLR consumes twice as much as a
CDP or CRR alone.
The journal size depends on the RPO. Longer RPO requires larger journal sizes.
Different consistency groups can be created in the same configuration.

Production systems can be assigned to a consistency group in CLR mode while
others can be assigned to a group in either CDP or CRR mode.

Manager
49
SAP systems falling under the same consistency group will failover as one
bundle. Assigning a separate consistency group to each SAP system is highly
recommended.
The size of the journal volumes should reflect the RPO. You need to determine
the expected peak change in the environment to calculate the minimum size of
the journal volume.
The formula is:

(
) (
Note
Target side log size is 20 percent (0.2) according to the best practice and an
additional 5 percent is allocated (1.05) for internal use.
For example, to calculate the minimum journal size to satisfy a 48-hour rollback
requirement (172,800 seconds) with an estimated 10 Mb new data writes per second:
(
) (
(
)
)
Step 2: Provisioning RPA repository and gatekeepers

To provision RPA repository and gatekeepers, create masking views in VMAX and
storage groups in VNX to present these volumes to the RPA cluster.
We created the following masking views for this purpose:
Recoverpoint_view: This presents the repository, journal, and replica volumes

to all nodes in the RPA cluster.
RPA1_GK_view: This presents the eight gatekeepers to RPA1.
RPA2_GK_view: This presents the eight gatekeepers to RPA2.
Step 3: Tagging all VMAX 10K devices to be protected by RecoverPoint

As of RecoverPoint 3.4 (SP1 and later), production and replica LUNs must be tagged
for RecoverPoint use in VMAX 10K splitter configurations. Otherwise, the devices will
not be accessible to the splitter. Note that this step is not required for VNX storage.

Manager
50
Figure 26 shows how we tagged devices through Unisphere for VMAX.
Figure 26.
Tagging devices using RecoverPoint for VMAX
Alternatively, the following command can be executed from the Solution Enabler for
every device to be tagged:
symconfigure sid <Symmetrix ID> -cmd set dev <device list or
device range> attribute=RCVRPNT_TAG; commit
Figure 27 shows the output of this command.
Figure 27.
Tagging a RecoverPoint device from Solution Enabler
To verify the device tag status, input the following command:

symdev sid <Symmetrix ID> show <device list or device range>

Manager
51
Figure 28 shows the status with the device tagged as RecoverPoint.
Figure 28.
Verifying the RecoverPoint tag status
It is a best practice to tag also the RecoverPoint gatekeepers, journal volumes, and
repository volumes.
Configuring
RecoverPoint
Step 1: Installing RecoverPoint

As shown in Figure 29, the RecoverPoint Installer wizard guides you through the
entire setup procedure including setting up the RPA cluster. Perform the steps
described in the section Preparing VMAX 10K and VNX5300 for RecoverPoint
connectivity before running the wizard.
Figure 29.
RecoverPoint Installer wizard
Step 2: Configuring RecoverPoint access to the splitters on the VMAX 10K and VNX
arrays
You need to configure the VMAX 10K and VNX splitters in RecoverPoint to bridge
the replication. We did this by using the New Splitter wizard in RecoverPoint through
Splitter > Add New Splitter, as shown in Figure 30.

Manager
52
Figure 30.
Adding a new splitter in RecoverPoint
Step 3: Creating a consistency group

After successfully installing RecoverPoint, you can configure the consistency groups
for the replication setup.
The New Consistency Group wizard guides you during the entire setup. You can find
the wizard in the RecoverPoint interface by right-clicking Consistency Groups and
selecting Add Group, as shown in Figure 31.
Figure 31.
Adding a consistency group in RecoverPoint

Manager
53
Consider the following before launching the wizard:
Configuring SRM
Replication policy settings for the consistency group
Which site should be considered as the production site and which should
be considered as the recovery site?
Which type of replication should be used (CDP/CRR/CLR)?
A one-to-one mapping of the production volumes to corresponding remote

replica volumes
Which volumes are allocated for local and remote journal
Installation prerequisites
Before installing SRM, ensure that the following prerequisites are met:
Each site must contain a vCenter server with at least one vSphere data center
being managed.
The recovery site must support array-based replication with the protected site.
At least one virtual machine must be located on a datastore that is replicated

by RecoverPoint on the protected site.
A reliable IP network is required to connect the two sites.
The recovery site must have access to the same public and private networks as
the protected site, though not necessarily having access to the same range of
network addresses.
VMware tools must be installed on all virtual machines.
Step 1: Installing SRM virtual machines and the SRA for integration to RPA
Perform the following steps:
1.
Install a database server on each site for SRM use. The databases will store
the recovery plans and inventory information.
2.
Install an SRM server on both sites.

Manager
54
3.
Install a RecoverPoint SRA on the SRM server on both sites to enable

integration.
4.
Install the SRM plug-in on at least one vSphere client. SRM configurations are
transparent to both sites, but it is highly recommended to install the SRM
plug-in on both sites.
For detailed steps, refer to VMware vCenter Site Recovery Manager Installation and
ConfigurationCenter Site Recovery Manager 5.1.
Step 2: Establishing connections between protected and recovery sites
Before configuring SRM, change the consistency groups management mode to
Group is managed by SRM, RecoverPoint can only monitor. You can change this on
the Policy tab after selecting the consistency group, as shown in Figure 32.
Figure 32.
Policy tab in a RecoverPoint consistency group
When this is completed, the two SRM servers must be connected in SRM to establish
reciprocity. You can do this through the Connect to Remote Site wizard, by specifying
the IP/port details and authenticating credentials of the other (remote) SRM server,
as shown in Figure 33.

Manager
55
Figure 33.
Connecting the remote site in SRM
The new SRM interface contains an easy-to-follow guide tab that explains the steps
throughout the configuration process.
Step 3: Configuring RecoverPoint array managers
After the protected and recovery sites have been paired, configure the RecoverPoint
array managers (through the SRAs installed earlier) for SRM to discover replicated
devices, compute datastore groups, and initiate storage operations. To do this, use
the Add Array Manager wizard, as shown in Figure 34.

Manager
56
Figure 34.
Adding storage arrays in SRM
Typically, you need to perform this step only once after the sites are connected.
Additional reconfiguration is not needed unless there is a change in connection
information or credentials.
After this step, all configurations can be done on just one SRM and will automatically
be synchronized to the other SRM server.
Step 4: Configuring protection groups
You can create a protection group after configuring the array managers.
A protection group is used to organize virtual machines that should be failed over
together, such as in this solution where the SAP database, SAP Central Services
instance, and three application instances are failed over as one entire set.
You can create many protection groups, but all virtual machines assigned to a
protection group must belong to the same datastore group (consistency group).
The Create Protection Group wizard automatically detects the datastores protected by
RecoverPoint and the virtual machines inside it. You need to specify which virtual
machines among the list must be added to a protection group, as shown in Figure 35.

Manager
57
Figure 35.
Creating a protection group in SRM
SRM then creates placeholder virtual machines automatically. Placeholders reserve

the resources on the recovery site and act as a shadow standby before a recovery
operation. The placeholders cannot be started independently of SRM and are
removed altogether if the protection group is deleted.
Figure 36 shows the actual virtual machines and the placeholders on the remote sites.
Site C
Site A
Figure 36.
Placeholders at each site
In this solution, we created a separate protection group for VM2 and VM3 respectively,
because both systems belong to different datastores and are cross-replicating toward
each others site.
Step 5: Configuring inventory mappings and placeholder virtual machines
As shown in Figure 37, the Resource mappings tab allows you to define which
resources on the recovery site will serve as the counterparts to the ones on the
protected site, such as network, resource pool, and virtual machines. SRM cannot

Manager
58
protect a virtual machine unless it has valid inventory mappings for key virtual
machine resources.
SRM derives the resource assignments of a newly created placeholder from the
inventory mappings established on the protected site.
EMC recommends that you group virtual machines into folders for easier
management.
Note
A counterpart folder must exist in the recovery vCenter server as well.
1
3
4
2
Figure 37.
Configuring resource mapping and placeholder in SRM
Step 6: Customizing virtual machine recovery operations

Especially in an SAP landscape, it is customary to start up applications according to
the business priority (for example, SAP Production > Third Party Production > SAP
Development and ERP > HR > BI), and in a particular order (such as DB server > SAP
Central Services > App instances).
These specifics can be customized to suit the recovery requirements of an
organization to ensure that even complex recovery plans can be fully automated.
This solution exploits this feature so that the SAP virtual machines are started up in
the correct order, and a script is executed within the recovery sequence to start up the
database, SCS, and all application instances, ready for login on handover.

Manager
59
Step 7: Customizing recovery site IP addresses

One of the complicating factors in a typical failover is the readjustment of IP settings
to adapt to the infrastructure differences after recovery. You may need to readjust IP
settings on multiple virtual machines.
SRM can also manage this tedious task. You can either do this individually by using
the SRM plug-in, or by using a bulk IP customization utility (dr-ip-customizer.exe),
which is especially useful for first-time configurations.
This utility generates a comma-separated value (CSV) file containing the IP settings of
all virtual machines configured for SRM failover. You can generate and edit the file as
follows:
1.
Log in to the command prompt or console of the vCenter server on the

recovery site.
2.
By using the dr-ip-customizer utility in the bin folder of the SRM installation
path, generate the existing IP table and specify the file name and location
where it will be saved.
C:\Program Files (x86)\VMware\VMware vCenter Site Recovery
Manager\bin>dr-ip-customizer.exe cfg ..\config\vmwaredr.xml cmd generate o <csv filename to generate> vc
<VMware vCenter server hostname>
3.
Edit the CSV file and customize the existing IP settings according to the IP
settings on the recovery site. Figure 38 shows the CSV file opens in a typical
spreadsheet application. Note that full IP addresses are not shown in this
figure.
Figure 38.
4.
CSV file where IP addresses can be changed
Run the dr-ip-customizer utility again to upload the edited CSV.

C:\Program Files (x86)\VMware\VMware vCenter Site Recovery
Manager\bin>dr-ip-customizer.exe cfg ..\config\vmwaredr.xml cmd apply csv <path to the csv file> vc <VMware
vCenter server hostname>

Manager
60
Step 8: Creating a recovery plan

A recovery plan is like an automated runbook. It controls every step of the recovery
process, including the order in which SRM powers on and powers off virtual machines,
the network addresses that recovered virtual machines use, and so on.
When a recovery plan is created, it prescribes a series of recovery steps that SRM
automates depending on the function executed. Figure 39 shows these functions.
Figure 39.
Functions in SRM GUI
The following list explains each of the functions:
Test: Online simulated recovery; SRM performs all recovery steps inside a test
network so that the test can run without disrupting the virtual machines in the
production environment.
Cleanup: A function to undo the result of test recovery (such as unmounting

recovered snapshots and returning placeholders) and to reset the recovery plan
to a ready state.
Recovery: The actual migration of virtual machines in the recovery plan to the
recovery site. There are two types that differ only in error tolerance:
Planned migration: If errors occur during migration, the execution is

stopped and the errors must be resolved before resuming.
Disaster recovery: If errors occur, SRM attempts to continue execution.
Reprotect: An automated function that completely reverses the direction of

protection after a failover. The original protected site becomes the new recovery
site and the original recovery site becomes the new protected site, provided
that the original site is operational.
Cancel: Function to terminate an ongoing test at any time or an actual recovery

during certain steps.

Manager
61
Figure 40 shows an SRM recovery plan. These steps change depending on the
function.
Figure 40.
SRM recovery plan in progress
Recovery plans are flexible and customizable. You can make readjustments to the
workflow as needed, such as:
Specifying steps that are run or skipped during test recoveries
Customizing timeouts to allow some time for a step to complete before the next
step
Creating message prompts to force the plan to pause until the user
acknowledges
Specifying the recovery priority of a virtual machine
Adding custom recovery steps for the SRM server or the recovered virtual
machine using scripts

Manager
62

H11402 Bidirectional Automated DR SAP WP

Hochgeladen von

Dokumentinformationen

Originaltitel

Copyright

Verfügbare Formate

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

Verfügbare Formate

H11402 Bidirectional Automated DR SAP WP

Hochgeladen von

Copyright:

Verfügbare Formate

White Paper

EMC TRANSFORMS IT FOR SAP: FULLY

EMC Solutions Group

Copyright 2013 EMC Corporation. All Rights Reserved.

EMC Transforms IT for SAP: Fully Automated Disaster Recovery

EMC Transforms IT for SAP: Fully Automated Disaster Recovery

Solution configuration ........................................................................................................................ 21

EMC Transforms IT for SAP: Fully Automated Disaster Recovery

Market and customer intelligence: Availability of data and information is critical

Supply, demand, and order fulfillment management: What may seem to be a

Scalability: Providing increased performance to handle increasing transactional

Disaster Recovery: How to respond to increasing pressures to mitigate planned

EMC Transforms IT for SAP: Fully Automated Disaster Recovery

Fast and seamless failover

Point-in-time (PIT) recovery without database restore

Nondisruptive failover testing

EMC Transforms IT for SAP: Fully Automated Disaster Recovery

The bi-directional replication strategy enabled by EMC RecoverPoint allows two

The integration of EMC RecoverPoint and VMware vCenter SRM provides a

This solution provides an environment for performing regular recovery testing

Customers can leverage this solution to migrate their SAP environments

EMC Transforms IT for SAP: Fully Automated Disaster Recovery

To achieve the stated purpose, the document covers the following:

Components used and the key concepts involved in implementation

The reference architecture of the bi-directional disaster recovery system using

Installation and configuration procedures for EMC RecoverPoint and VMware

Use-case functionality test results

This white paper is intended for SAP administrators, storage administrators, IT

Have virtual SAP environments on existing storage and plans to migrate to a

Have existing virtual SAP environments on Symmetrix VMAX storage solutions

Table 1 includes the terminology used in this white paper.

Process of committing data writes when the data has

Continuous data protection.

Concurrent local and remote replication.

EMC Transforms IT for SAP: Fully Automated Disaster Recovery

RecoverPoint configuration that logically groups

Continuous remote replication.

A specialized data compression technique for

Small, three-cylinder devices (approximately 3 MB)

Timestamped application writes stored in RecoverPoint

Key performance indicator.

Point-in-time; usually refers to a system state that

Volume that contains the replicated image from a

RecoverPoint configuration that defines an association

Volume that stores configuration information about

Recovery point objective.

Recovery time objective.

Process of committing data writes only after the

EMC Transforms IT for SAP: Fully Automated Disaster Recovery

This solution uses the key components described in Table 2.

Storage replication device

EMC Symmetrix VMAX 10K

Production site storage

Recovery site storage

Storage management application

VMware vCenter Site Recovery Manager

SAP ERP 6.0

Enterprise application software

EMC RecoverPoint is an advanced enterprise-class disaster recovery solution

EMC Transforms IT for SAP: Fully Automated Disaster Recovery

Controls CLI Version 7.3 Product Guide.

EMC Transforms IT for SAP: Fully Automated Disaster Recovery

SAP ERP 6.0