Double-Take RecoverNow User Guide

D o u b l e - Ta k e R e c o v e r N o w
Ver s i o n 4 . 0 S P 0 1
User Guide
July, 2012
Double-Take RecoverNow Version 4.0 SP01 User Guide
Copyright Vision Solutions , Inc. 20032012
All rights reserved.
The information in this document is subject to change without notice and is furnished under a license agreement. This
document is proprietary to Vision Solutions, Inc., and may be used only as authorized in our license agreement. No portion of
this manual may be copied or otherwise reproduced without the express written consent of Vision Solutions, Inc.
Vision Solutions provides no expressed or implied warranty with this manual.
The following are trademarks or registered trademarks of their respective organizations or companies:
Vision Solutions is a registered trademark and ORION Solutions, Integrator, Director, Data Manager, Vision Suite,
ECS/400, OMS/400, ODS/400, SAM/400, Double-Take GeoCluster, Double-Take RecoverNow, Double-Take SHARE,
RecoverNow and iTERA HA are trademarks of Vision Solutions, Inc.
DB2, IBM, i5/OS, iSeries, System i, System i5, Informix, AIX 5L, System p, System x, and System z, and WebSphere
International Business Machines Corporation.
Adobe and Acrobat ReaderAdobe Systems, Inc.
Double-Take, GeoCluster, and NSINSI Software, Inc.
HP-UXHewlett-Packard Company.
TeradataTeradata Corporation.
IntelIntel Corporation.
Java, all Java-based trademarks, and Solaris-Oracle Corporation.
LinuxLinus Torvalds.
Microsoft and WindowsMicrosoft Corporation.
Mozilla and FirefoxMozilla Foundation.
NetscapeNetscape Communications Corporation.
OracleOracle Corporation.
Red HatRed Hat, Inc.
SybaseSybase, Inc.
Symantec and NetBackupSymantec Corporation.
UNIX and UNIXWarethe Open Group.
All other brands and product names are trademarks or registered trademarks of their respective owners.
If you need assistance, please contact Vision Solutions SCP Certified CustomerCare team at:
CustomerCare
Vision Solutions, Inc.
Telephone: 1.800.337.8214 or 1.949.724.5465
Email: support@visionsolutions.com
Web Site: www.visionsolutions.com/Support/Contact-CustomerCare.aspx
Contents
Chapter 1Overview of Data Replication Concepts . . . . . . . . . . . . . . . . . . . . 15

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
Roles in the System Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
Process and Design Specifics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
RecoverNow Replication Group and Contexts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
RecoverNow File Container . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
RecoverNow Datatap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
RecoverNow Journal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
RecoverNow Agents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
LCA Agent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
ABA Agent. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
AA Agent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
RA Agent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
Replication. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
Journal Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
Production Journal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
Recovery Journal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
Recovery Log Sizing. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
Log File Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
Log Size Estimate. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
Number of Logs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
RecoverNow Snapshots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
Recovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
Vision Solutions Portal (VSP) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
RecoverNow with VSP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
RecoverNow with the Command Line Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
Double-Take RecoverNow v4.0.01.00 User Guide
iii
Contents
Chapter 2Planning your Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .29

Allocating Space for RecoverNow Logs and Journals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
Guidelines for Production Journal Size . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
Production Journal Size Estimate 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
Production Journal Size Best Estimate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
Guidelines for Recovery Journal Size . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
Recovery Journal Size Estimate 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
Guidelines for Selecting Volumes to Be Protected . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
Guidelines for Snapshot Journal Size . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
Guidelines for Log Size . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
Determining Storage Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
Using Event Markers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
Event Marker File and the rtmark command. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
Print Available event marks for rollback . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
Using the rn_shutdown script . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
Application Information Check List . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
Database Information Check List . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
Domain Name Server Check List . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
Network Information Check List . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
Storage Information Check List . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
General Information Checklist . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
Chapter 3Using the Sizing Tool to Calculate LFC Size. . . . . . . . . . . . . . . . .43

Overview of LFC Sizing. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
System Requirements for the Sizing Tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
Installing the Sizing Tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
Running the Sizing Tool from the RecoverNow Sizing Tool GUI . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
Introduction Tab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
Config and Run Tab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
View and Analyze Log Tab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
View Chart Tab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
Running the Sizing Tool Script from the Command Line. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
sztool script Command Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
iv
Contents
Chapter 4Supported Configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

Local CDP Solution Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
Local CDP Solution with WAN Replicated Configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
WAN Replication with No Local CDP Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
Local Bi-directional Replication Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
Remote Bi-directional Replication Configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
Chapter 5Installation Procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
System Requirements for RecoverNow. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
Operating System Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
Disk Space and Memory Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
Archiving Products Supported . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
VSP and RecoverNow Portal Application System Requirements. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
RecoverNow Installation Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
RecoverNow Installation Wizard Prerequisites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
Before You Install . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
Request License Keys from AIX or Windows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
AIX Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
Install Double-Take RecoverNow, the Vision Solutions Portal and the Double-Take RecoverNow Portal
Application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
Install only the Vision Solutions Portal and the Double-Take RecoverNow Portal Application . . . . . 77
User Roles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
Reinstall RecoverNow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
Upgrade RecoverNow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
Use smit to Install RecoverNow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
Step 1. Request a License . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
Step 2: Install RecoverNow. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
User Roles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
Log Files. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
Run the validate_license command . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
License Expiration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
Uninstall RecoverNow on AIX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
Use smit to Uninstall RecoverNow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
Install the Vision Solutions Portal and Double-Take RecoverNow Portal Application on Windows . . . . 118
Uninstall the Vision Solutions Portal and Double-Take RecoverNow Portal Application Uninstallation Wizard on Windows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
Post-Installation Tasks. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126
Contents
Logging in to Vision Solutions Portal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127

Increase Your AIX File Limit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
Enable Automatic Startup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
Chapter 6Configuring Replication Groups . . . . . . . . . . . . . . . . . . . . . . . . .131

Using VSP to Configure Replication Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132
Create and Configure a New Replication Group. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132
Configuration Initialization Progress (New Configuration). . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150
Change a Replication Group Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151
Configuration Initialization Progress (Change Configuration) . . . . . . . . . . . . . . . . . . . . . . . . . . 161
Rename a Replication Group. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164
Delete a Replication Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166
Using Earlier Versions of RecoverNow to Configure Replication Groups . . . . . . . . . . . . . . . . . 168
Replication Group Configuration Window . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169
Using the Command line to Perform Configuration Tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169
Initialize a Context. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169
Send partially filled containers automatically . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170
Configurations created prior to RecoverNow Version 4.0. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170
Verify your Configuration using the sccfgchk Command. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171
Support for LVM commands when the RecoverNow drivers are loaded. . . . . . . . . . . . . . . . . . . . . . . . . . 172
RecoverNow Default Ports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174
Adding specific port assignments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174
Activating syslog Debug. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175
Chapter 7Starting and Stopping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .177

Using VSP to Start and Stop RecoverNow. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177
Using VSP to Start RecoverNow. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177
Using VSP to Stop RecoverNow. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178
Using the command line to Start and Stop RecoverNow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179
Using the command line to Start RecoverNow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179
Monitoring the Log Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180
Using the Command line to Stop RecoverNow. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180
Chapter 8Creating Snapshots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .183

Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183
Using Snapshots to Protect Your Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183
Using VSP to Create a Snapshot. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184
Using the Command Line to Create a Snapshot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186
Creating Snapshots Based on the Current Redo Log. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187
Creating Snapshots Based on a Specific Date and Time. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189
vi
Contents
Verify that the DATEMSK environment variable is defined . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189

Establish the date format for the DATEMSK environment variable . . . . . . . . . . . . . . . . . . . . . . 189
Create Snapshots Based on a Specific Date and Time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190
Creating a Snapshot Based on a Specific LFCID. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191
Chapter 9Administrative Tasks. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193

Verify the configuration using the sccfgchk command. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193
Updating the RecoverNow Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194
Verifying Available Free Space. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194
Extending a RecoverNow-Protected Filesystem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194
Extending a Protected File System Beyond the State Map Limit . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195
Increasing the Snapshot Journal Space on the Recovery Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197
Setting up JFS Log Isolation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198
Removing a Filesystem from a RecoverNow Protected jfslog . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199
Setting Up Error Notification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199
ODM Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201
Error Notify Messages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201
Marking State Maps Dirty and Synchronizing the PVS or one LV in the PVS . . . . . . . . . . . . . . . . . . . . . 204
Alternative Method for Performing Initial Synchronization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205
Using IBM Power Systems Live Partition Mobility with RecoverNow . . . . . . . . . . . . . . . . . . . . . . . . . . . 206
Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 206
Automatically Registering the es_migrate script . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 206
Manually Registering the es_migrate script . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207
Before you Migrate a Partition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208
Migrating a Partition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209
Chapter 10Working with RecoverNow Applications . . . . . . . . . . . . . . . . . 211

Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211
Database Recovery Operations Scenarios . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211
Database Resurrection Concept . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212
Database Repair Concept . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213
Performing a Volume Restore . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213
Step 1: Stop Double-Take RecoverNow on the Production Server . . . . . . . . . . . . . . . . . . . . . . . 214
Step 2: Create New Volumes to Replace Failed Volumes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214
Step 3: Create a Snapshot on the Recovery Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215
Step 4: Validate the Information Provided by the Snapshot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 216
Step 5: Destroy the Snapshot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217
vii
Contents
Step 6: Recreate the Snapshot on the Recovery Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217

Step 7: Copy Volumes on Recovery Server to Replacement Volumes on Production Server. . . 217
Step 8: Restart Double-Take RecoverNow on the Production Server . . . . . . . . . . . . . . . . . . . . . 217
Step 9: rollback the Appropriate Context on the Production Server . . . . . . . . . . . . . . . . . . . . . . 218
Step 10: Mount the Volumes for the Context. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219
Step 11: Destroy the Snapshot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220
Performing a Production Restore . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220
Using VSP for a Production Restore . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220
Using the Command Line to Perform a Production Restore . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225
Chapter 11Working with Archived Data . . . . . . . . . . . . . . . . . . . . . . . . . . .227

Overview of Working with Archived Data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227
Performing a Snapshot-based Backup to Archive Media . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227
Performing a Production Restore from Archived Logs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228
Chapter 12Introduction to Disaster Recovery . . . . . . . . . . . . . . . . . . . . . . .231

Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231
RecoverNow Disaster Recovery Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232
RecoverNow Recovery Process. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232
How RecoverNow Disaster Recovery Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232
RecoverNow Disaster Recovery Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233
Failover Context Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233
Failover Context Specifications. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234
Failover Context and Primary Context Relationship . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234
Setup a Failover Context . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234
Failover Context Naming Conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234
How the Failover Context Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234
rtdr Failover Command . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235
Syntax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235
RecoverNow Disaster Recovery Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236
Failover Process Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237
Preparing Before a Failover. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237
Setting up a Failover Context . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237
Before Performing Failover Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 238
Validating Data Integrity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 238
Verify Restore Point . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 238
Create a Virtual Restore Snapshot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239
viii
Contents
Chapter 13Disaster Recovery Operations . . . . . . . . . . . . . . . . . . . . . . . . . . 241

Using VSP to Perform Failover and Failback . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241
Run a Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241
Run Planned Failover Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242
Run Planned Failover Procedure dialog: Unmount file systems on current production server . . 244
Resume Planned Failover Procedure dialog: Stop replication on current production server . . . . 245
Resume Planned Failover dialog: Failover replication group: Server roles change . . . . . . . . . . . 245
Resume Planned Failover dialog: Start replication on new replicated server. . . . . . . . . . . . . . . . 245
Resume Planned Failover dialog: Start replication on new recovery server. . . . . . . . . . . . . . . . . 245
Resume Planned Failover dialog: Start replication on new production server . . . . . . . . . . . . . . . 245
Run Unplanned Failover Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245
Run Unplanned Failover Procedure dialog: Create snapshot on failover server . . . . . . . . . . . . . 248
Resume Unplanned Failover Procedure dialog: Delete snapshot on failover server . . . . . . . . . . 248
Resume Unplanned Failover Procedure dialog: Rollback failover server . . . . . . . . . . . . . . . . . . 248
Resume Unplanned Failover dialog: Failover replication group: Server roles change. . . . . . . . . 248
Resume Unplanned Failover dialog: Start replication on new replicated server . . . . . . . . . . . . . 249
Resume Unplanned Failover dialog: Start replication on new recovery server . . . . . . . . . . . . . . 249
Resume Unplanned Failover dialog: Start replication on new production server. . . . . . . . . . . . . 249
Run Failback Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 249
Run Failback Procedure dialog: Unmount file systems on current production server . . . . . . . . . 251
Resume Failback Procedure dialog: Stop replication on current production server . . . . . . . . . . . 251
Resume Failback Procedure dialog: Failback configured production server . . . . . . . . . . . . . . . . 252
Resume Failback Procedure dialog: Failback configured recovery server . . . . . . . . . . . . . . . . . . 252
Resume Failback Procedure dialog: Failback configured replicated server . . . . . . . . . . . . . . . . . 252
Using the Command line to Perform Failover and Failback . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252
Failover Process Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253
Preparing Before a Failover . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253
Setting up a Failover Context . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253
Before Performing Failover Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 254
Validating Data Integrity. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 254
Verify Restore Point . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 254
Create a Virtual Restore Snapshot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 255
Failover Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 255
Performing a Failover Restore . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 256
Start the Application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 256
Performing Resync . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 256
ix
Contents
Resync with Region Recovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257

Performing Failback . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 258
Manual Resynchronization Process if production server Data Is Lost. . . . . . . . . . . . . . . . . . . . . 258
Chapter 14CLI Commands. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .261

extend_replica_lv . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 262
Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 262
rtattr. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263
Syntax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263
Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263
Example 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263
Example 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 264
rtdr. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 264
Syntax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 264
Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 265
rtmark . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 268
Syntax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 268
Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 268
rtmnt . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 268
Syntax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 268
Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269
Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269
See Also . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269
rtstart . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269
Syntax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269
Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269
Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 270
See Also . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 270
rtstop . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 270
Syntax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 270
Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 270
Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 270
See Also . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 270
rtumnt . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271
Syntax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271
Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271
Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271
See Also . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271
sclist . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271
Syntax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271
x
Contents
Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273
See Also . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273
scconfig . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 274
Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 274
Syntax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 274
See Also . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 275
scsetup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 276
Syntax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 276
Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 277
scrt_ra . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 278
Syntax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 278
Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 278
Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279
See Also . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279
scrt_rc . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279
Syntax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279
Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279
Session restore targets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 280
Session Termination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 280
Process Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 280
General Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 281
Procedure Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 281
scrt_vfb . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 281
Syntax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 281
Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 281
Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 282
sccfgd_cron_schedule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 282
Syntax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 282
Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283
Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283
sccfgd_putcfg. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283
Syntax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283
Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283
Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283
sccfgchk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283
Syntax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283
Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283
Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 284
sztool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 284
xi
Contents
Syntax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 284
Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 284
Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285
Appendix AIntegration of PowerHA (HACMP) with RecoverNow . . . . . .287

Configuring a Highly Available RecoverNow/PowerHA Production Server Environment . . . . . . . . . . . 287
RecoverNow Configuration Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 287
Configuring Highly Available Production Servers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 288
RecoverNow/PowerHA Integration requirements. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 291
RecoverNow Start Script . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 292
RecoverNow Shutdown Script . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 292
Managing RecoverNow in a HA Production Server Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . 294
Changing the RecoverNow Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 294
Managing Failover to the Recovery Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 295
Unplanned Failover . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 295
Planned Failover/Resync/Failback . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 297
Configuring a Highly Available RecoverNow/PowerHA Recovery Server Environment. . . . . . . . . . . . . 298
RecoverNow Configuration Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 299
Configuring Highly Available Recovery Servers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 299
RecoverNow/PowerHA Integration requirements: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 302
RecoverNow Start Script . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 303
RecoverNow Shutdown Script . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 303
Managing RecoverNow in a HA Recovery Server Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . 305
Changing the RecoverNow Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 305
Managing Failover to the Recovery Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 305
Unplanned Failover . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 306
Planned Failover/Resync/Failback . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 308
Configuring a RecoverNow/PowerHA Production to Recovery Server Environment. . . . . . . . . . . . . . . . 309
Prerequisites. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 309
Overview of the Failover Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 310
Sequence to manually bring the Resource Groups ONLINE: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 311
Sequence to manually bring the Resource Groups OFFLINE: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 311
Planned Failover Procedure. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 311
Unplanned Failover Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 311
Failback Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 311
PowerHA for AIX Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 312
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 317
xii
Overview of Data Replication

Concepts
Introduction
The Double-Take RecoverNow User Guide describes how to install,
configure, maintain and administer Double-Take RecoverNow (hereafter
referred to as RecoverNow), data replication software. The table below
shows the chapters in the RecoverNow User Guide.
Chapter
Description
Chapter 1, Overview of Data

Replication Concepts on page 15
Use this chapter to learn about the main

concepts related to data replication.
Chapter 2, Planning your Environment

on page 29
Use this chapter to prepare RecoverNow

system for post-installation
configuration.
Chapter 3, Using the Sizing Tool to

Calculate LFC Size on page 43
Use this chapter to learn how to use the

Sizing tool to calculate the LFC size.
Chapter 4, Supported Configurations

on page 55
Use this chapter to learn about the

configurations supported by
RecoverNow.
Chapter 5, Installation Procedures on

page 59
Use the procedures in this chapter to

install RecoverNow.
Chapter 6, Configuring Replication

Groups on page 131
Use the Replication Group Configuration

Wizard to configure replication groups
on your servers.
Chapter 7, Starting and Stopping on

page 177
Use the procedures in this chapter to start

up and shut down both the Production
and the recovery servers.
Chapter 8, Creating Snapshots on

page 183

restore a complete copy of the data on the
production server to any time in the past.
Chapter 9, Administrative Tasks on

page 193

perform data replication and restore
operations.
15
Chapter 1: Overview of Data Replication Concepts
Chapter
Description
Chapter 10, Working with RecoverNow

Applications on page 211

rollback application data on the
production server and restore the data to
an earlier point in time.
Chapter 11, Working with Archived

Data on page 227

create complete copies of the data on
archive media such as tape.
Chapter 12, Introduction to Disaster

Recovery on page 231
Use this chapter to become familiar with

disaster recovery concepts.
Chapter 13, Disaster Recovery

Operations on page 241

perform disaster recovery operations.
Chapter 14, CLI Commands on

page 261
Use the Command Line Interface (CLI)

commands in this chapter to work with
RecoverNow.
Appendix A, Integration of PowerHA

(HACMP) with RecoverNow on
page 287
Use the procedures in this appendix to

configure RecoverNow in a HA
production and recovery server
environment.
Overview
This chapter describes the organization and architecture of RecoverNow, a
continuous data protection system designed for immediate data recovery.
RecoverNow is a software only product for IBM servers running on IBM
AIX 5L, AIX 6.1, and AIX 7.1 operating systems.
RecoverNow provides these features in enterprise application environments
using standard systems hardware, commodity storage components, and
common networking infrastructure.
The RecoverNow data replication solution:
16
Provides asynchronous data replication
Includes rapid data rollback with Any Point-in-Time virtualization
Provides Read/Write data snapshots for off-host processing
Provides functionality with little or no production application impact
Minimizes application Recovery Time Objective (RTO) and Recovery

Point Objective (RPO) within an Optimized Recovery Window (ORW)
Is resilient to network, storage, and system failures and outages
Optimizes for rapid recovery to Any Point-in-Time
Roles in the System Architecture
Allows for administrative double check before change of production

data
Supports both block and file based applications
Maintains application write-order fidelity
Has no proprietary hardware dependencies
Always looks to supplement, not replace, existing enterprise processes

for logical storage management, backup, and high availability
Roles in the System Architecture

A standard RecoverNow system configuration consists of at least two
similar servers: a production server and a recovery server. A production
server is where the production application resides, and the recovery server
is where a data replica is maintained, rapid recoveries of the production data
can be initialized, any point-in-time may be virtualized, and enterprise
processes may be off-loaded from the production server.
You also have the option to configure one or many replicated server(s); this
allows for another data replica which is maintained in a cascaded fashion
from the recovery server. The hardware requirements of the replicated
server are similar to that of the recovery server. For the purposes of the
RecoverNow system architecture, the replicated server role appears exactly
as a recovery server. Functionally they are the same with the exception that
the production servers data is never restored from a replicated server, only
from the recovery server role.
Process and Design Specifics

RecoverNow is designed to capture every write from an application. This
supports application write-order fidelity and ensures that there is no data
loss in the face of localized failures, To minimize the effects of latency on
the production application, data is transferred from production server to the
recovery server asynchronously.
To support the widest array of applications and hardware, the system is
designed to appear as an extension to existing logical volume management
paradigms. At the block level, both block-based and file-based applications
can be protected. Furthermore, this allows for the system and the
administrator to take advantage of familiar tools for tuning and monitoring.
To enable any point-in-time to be virtualized and potentially recovered or
rolled back, the system needs to maintain a history of all of the writes from
the production application in additional storage to the full replica data
maintained on the recovery server. Maintaining this journal of all
application writes on the recovery server allows for additional optimizations
17
without impact to the production server. It also enables the trade off of
storage capacity for time, which is at the core of the RecoverNow system
and the source of its advanced functionality.
RecoverNow Replication Group and Contexts

Configuring RecoverNow replication groups involves defining
RecoverNow Contexts for each replication group. A Context is identified
by its Context ID. There are two types of RecoverNow contexts:
Primary context IDContext ID for the replication group. You

configure the Primary context on the production server.
Failover context IDContext ID of the failover context. You configure

the corresponding Failover context on the recovery and replicated
servers.
A RecoverNow context includes:
The applications and associated files and volumes that you want to
protect.
The production and recovery servers, and optionally, the replicated

server.
Archiving systems.
Related Topics:
Using VSP to Configure Replication Groups on page 132.
Initialize a Context on page 169
RecoverNow File Container

A RecoverNow File Container is a generic term used to describe the storage
object that the RecoverNow Context needs to function correctly. While
there are different specialized types of File Containers, the most commonly
discussed and most necessary to understand is the Log File Container
(LFC). A LFC is the logical unit of data movement around the system.
When an LFC is filled with data writes on the production server it is fully
transferred to the recovery server. Once transferred, it is processed and
applied to the data replica. If there is a replicated server in the system then
the LFC is transferred to the replicated server; if not it is freed for reuse.
You can also send partially filled containers. The Send partially filled
containers automatically option enables you to control the frequency of
shipping containers (LFCs) to the recovery server during low I/O periods.
The Send partially filled containers automatically option can be managed
in two ways:
18
Process and Design Specifics
Use the Replication Group Configuration Wizard. Refer to the New

Replication Group Container Options panel, and the field, Send
partially filled containers automatically on page 170.
Use the command line using the scconfig command. Refer to

scconfig on page 274.
RecoverNow Datatap
The AIX Logical Volume Manager (LVM) maintains the hierarchy of
logical structures that manage disk storage. RecoverNow kernel extensions,
or datataps, reside logically above the LVM layer inside the operating
system kernel. Furthermore, these datataps are logically below the file
system level and handle block level transfers. The datatap receives a buf
structure from the file system layer in the case of a file system write
operation or from the application in the case of a raw Logical Volume write.
Data is then processed and sent onto the LVM (Logical Volume Manager)
layer. For read operations from storage, data passes through untouched.
19
The datatap is loaded on both the production server and the recovery server.
On the production server, the datatap is responsible for splitting data write
operations. Each write results in a write to the intended protected volume as
well as to a redo log.
RecoverNow Journal
Two specific structures are used to contain the redo and undo logs in the
RecoverNow architecture. The After Image Log File Container (AILFC)
and Before Image Log File Container (BILFC) are used to hold these logs.
The entire set of logs is known as a journal to RecoverNow, and on the
recovery server, associated redo logs together with undo logs form the
journal. The journal is often illustrated as a single pool, and these logs are
20
RecoverNow Agents
block storage devices that do not interact with resident file systems or their
cache buffers.
RecoverNow Agents
RecoverNow uses the following agents:
Log Creation Agent (LCA)The primary agent that runs on the

production server, called the Log Creation Agent (LCA).
Assured Backup Agent (ABA)A primary agent that runs on the

recovery server
Archive Agent (AA)A primary agent that runs on the recovery server
Restore Agent (RA)A primary agent that runs on the recovery server
LCA Agent
Shipping logs from the production server to the recovery server is the
responsibility of the LCA. The LCA reads from the journal any redo log
information that has been closed, or sealed, and this information is then
shipped over one or more IP networks to an agent that runs on the recovery
server. Both agents bind and communicate over the same socket. Socket
port addresses can either be default addresses or they can be
programmatically selected.
ABA Agent
On the other side of the socket and running on the recovery server, the ABA
is collecting log information. The ABA receives the redo log information in
the time order it was created on the production server, and then stores this
information in recovery logs. Remember, these are block storage devices
that do not interact with resident file systems. As the ABA receives the data,
it dynamically creates optimized State Map Transactions (SMTX). The
blocks identified are then sorted in ascending device/block order. Block
ordering is a more efficient organization for applying modifications to the
replicated data, or replica, on the recovery server.
Before the modifications are applied to the replica, yet another block
storage device is written to with information that would allow the replica to
step backward in time. This storage device is called the undo log and
appears to be nothing more than a logical volume to the volume manager.
Once the undo log information is saved on disk, the redo log can be applied
to the replica to bring it up to date with the data on the production server.
21
AA Agent
The AA, or Archive Agent, also runs on the recovery server. It is used to
extend RecoverNows rollback capabilities by recording redo and undo logs
to media. The AA currently works with Tivoli Storage Manager (TSM). The
AA uses the TSM API to send archive requests to TSM. When the logs are
archived, they are always spooled in pairs. Depending on the TSM
configuration the data is stored on media. A redo and an undo log are
always together when the AA stores on media. This gives RecoverNow the
ability to restore the data to any point in time. By unwinding the data with a
course grained undo log, then applying fine grained redo log information to
the log, the state of the replicated data can be restored to any point in time.
RA Agent
Restoration is handled by the RA and runs on the recovery server. It does
not, however, run continuously like the other agents. It can
programmatically be executed from the command line or through the GUI.
The RA deals with the following types of restore operations:
Virtual restores to a specific date and time
Production restores to a specific date and time
Virtual restores to a virtual full backup (VFB)
Production restores to a virtual full backup (VFB)
Restoring individual protected volumes
Virtual restores occur on the backup server. Production restores are restores
in which all volumes defined in a context are rolled back together on the
production server.
22
Replication
Replication
RecoverNow runs automatically on a production server, creating a mirrored
copy of protected data on the recovery server. For increased availability, it is
recommended that the recovery server be a remote machine. The following
illustration shows storage data flow during RecoverNow replication:
On the production server, write operations to storage are split by the

datatap. One copy of the data is delivered to the protected volume. The
other copy of data is combined with metadata and stored in the redo log.
The LCA sweeps through the production journal and reads any log that has
been filled, or sealed, then transmits the log file over TCP/IP to the ABA
which stores the log file to disk on the recovery server.
NOTE
Data does not pass through the datatap on the recovery server.
The ABA sweeps through the log files in time order and uses the metadata
reads from the replica to calculate the amount of change required to apply
the working log file and stores this information in the undo log. The ABA
then reads from the redo log and applies the modification in block order to
the replica.
Journal Configuration
RecoverNow uses the following journals:
23
Production Journal on page 24
Recovery Journal on page 24
Production Journal
The production journal holds redo log buffers until the logs are transferred
to the recovery server. Then the logs are available to receive new
application write data. Sizing the journal properly prevents the recovery
server from falling so far behind the production server that dynamic
recovery must occur for the recovery server to catch up. If the journal is too
small, then transfers between the production server and the recovery server
are performed more frequently than is efficient. If the journal is too big,
then the recovery server may fall so far behind the production server that
dynamic recovery must occur.
The appropriate size of the production journal is proportional to the length
of network or recovery server downtime that RecoverNow can sustain
without falling into dynamic recovery, or the amount of data in write
throughput spikes that exceed system bandwidth that RecoverNow can
sustain without falling into dynamic recovery.
Recovery Journal
The recovery journal is on the recovery server, and holds redo and undo
logs, that act as RecoverNows internal rollback window. If you are using
external archive media such as tape, then the size of the journal on the
recovery server is not critical to the ability to restore data. The larger the
recovery journal, the larger the internal rollback window, which implies
faster access to redo and undo logs during production restores in that
window.
Recovery Log Sizing

If you are not using external archive media, then the size of the recovery
journal is critical for data protection. Rollbacks cannot extend beyond the
logs that exist on the recovery server. You must estimate average throughput
and calculate journal area based on the length of the desired average restore
window.
The size of the recovery journal is proportional to write throughput and the
required internal rollback window.
The journal on the recovery server should be at least 256MB. Note that this
is twice the space recommended for the minimum on the production server,
because the recovery server contains both redo logs and undo logs.
Refer to Chapter 2 Planning your Environment on page 29 for
information and formulas to help you size the journals for your system.
24
Log File Configuration
Refer to Chapter 3 Using the Sizing Tool to Calculate LFC Size on

page 43 to automated the process of sizing the journals for your system.
Log File Configuration

All logs are the same size on the production server and the recovery server.
The log size affects the amount of data that RecoverNow processes for each
transaction.
By increasing the size of the logs, processing is reduced and the elimination
of common blocks in the undo logs is more efficient. Decreasing log size
results in a more up-to-date replica on the recovery server, because log
transfers occur more frequently.
When you determine the best log size, keep these conditions in mind:
The journal should contain at least eight logs
Minimum log size is 2MB
Maximum log size is one-half of the available RAM but not greater
than 512 MB
Typical log size is 16MB
To calculate log size, you need an estimate of average write throughput, and
the required processing rate. For the required processing rate, if
RecoverNow processes one log every 60 seconds, the replica will be one
minute behind the production system.
Log Size Estimate

log size = (average write throughput) / (processing rate)
Number of Logs
number of logs = (journal size) / (log size)
Even though the calculation for the number of log files appears trivial, keep
in mind that the number of log files can affect performance. If enough log
files are available on the production server, RecoverNow does not have to
rely on state maps during an outage, because it has not run out of log files to
take in data. A state map contains information about data changes for each
storage device protected by RecoverNow. It can be used to reconstruct data
changes if the underlying data is corrupted or lost. During peak usage, when
an application is writing data faster than the network can transmit, extra log
files enable the system to buffer during these peak periods without having to
rely on state maps, eliminating the risk of a restore blackout window. On the
recovery server, a sufficient number of log files allows activity to be
buffered in the event that the tape drive or library is taken offline.
25
The Chapter 2 Planning your Environment on page 29 contains

information and formulas to help you size the logs for your system.
RecoverNow Snapshots
Snapshots use significantly less space and are more efficient than data
mirrors. A mirror is an up-to-date copy of data for a logical volume. Two or
more complete copies can exist at the same time, although only one copy is
seen or used by an application, so mirrors require double or more the
amount of disk space than the original data.
A snapshot is a view of data at a specific point in time, much like a
photographic image is a snapshot of physical images at a particular point in
time. You can use snapshots to validate data before you save it to permanent
storage, data mine and generate reports, and retrieve specific data items.
Snapshots are stored in a different location than the replica so that the
replica can continue to march along in time. The snapshot, however, is
frozen with respect to the replica. Again, using the analogy of a photograph,
you can now draw on the photograph and it does not effect the original
subject of the photograph. The ability to modify the snapshot is
accomplished by using a copy-on-write log file.
Notice from the above figure that data is passing through the datatap on the
recovery server in the case of reads and writes to snapshot data.
RecoverNow uses a different set of device minor numbers when dealing
with snapshots, so that the datatap knows which log files to access in a
specific order. For example, when a write operation is directed at the
snapshot it is actually written to the copy-on-write (COW) log instead. If
the data has not been modified, then a read operation would come from the
26
Recovery
snapshot. If the data has been modified, then the read would come from the
copy-on-write log. Keep in mind that the snapshot is the representation of
the application data at a specific point in time.
Related Topics:
Using VSP to Create a Snapshot on page 184.
Using the Command Line to Create a Snapshot on page 186.
Recovery
Generally, there are two types of recovery restorations. A production restore
is a rollback in time which takes place in the protected volumes on the
production server. The other type of restore, a virtual restore, is a rollback
in time which is executed over a read-writable virtual image of the protected
volumes which reside on the recovery server.
For a production restore, RecoverNow must have exclusive I/O access to
the protected volumes. The application must be stopped, and the file
systems must be unmounted. RecoverNow is the only process that should
be allowed to write into the protected volumes during a production restore.
The control over the protected volumes and the information stored by the
RecoverNow process allow an undo of data corruption faster than the
corruption occurred.
Production restores are useful for a database crash where the database
will not come up. By recovering an image of the actual production database
to some point in the past directly on the production disk itself, RecoverNow
can rollback a crashed database in minutes rather than hours or days for the
most disastrous operational situation a database can encounter.
In contrast, a virtual restore is useful for database repair. In this case, an
image of the database is rolled back to some point in the past on the
snapshot which resides on the recovery server. Select pieces of the data can
then be extracted and copied into the production database.
Related Topics:
Using VSP to Perform Failover and Failback on page 241.
Using the Command line to Perform Failover and Failback on

page 252.
Vision Solutions Portal (VSP)

Vision Solutions Portal is a server that runs portal applications for Vision
Solutions products. VSP enables multiple products and instances to be
easily monitored and managed from a browser-based graphical interface in
27
a single, unified view. VSP also provides services and portlets for
performing activities common to products.
Any Vision Solutions product that provides a portal application at a
compatible version to VSP can be used. A portal application includes the
graphical interface and supporting functionality for the products use within
VSP.
VSP is quick to configure and easy to customize. Portal connections define
how VSP connects to nodes in your enterprise. When configuring an
instance of a product, you identify the portal connections the instance will
use to connect to, retrieve data from, and perform actions for the product.
RecoverNow with VSP

You use the RecoverNow installation wizard to install RecoverNow, the
portal application and VSP. Refer to the section in Chapter 5, Installation
Procedures on page 59.
To log into VSP, refer to Logging in to Vision Solutions Portal on
page 127.
For detailed information on working with VSP, refer to Getting Started with
Vision Solutions Portal (2.0.05.00_VSP_Getting_Started.pdf) packaged
with RecoverNow for AIX.
RecoverNow with the Command Line Interface

You can also use the command line interface to work with RecoverNow. For
a list of commands and their usage, refer to Chapter 14 CLI Commands
on page 261.
28
Planning your Environment
Use this chapter to prepare your RecoverNow system for its initial
configuration.
This chapter contains:
Allocating Space for RecoverNow Logs and Journals on page 29
Guidelines for Production Journal Size on page 30
Guidelines for Recovery Journal Size on page 32
Guidelines for Selecting Volumes to Be Protected on page 33
Guidelines for Snapshot Journal Size on page 33
Guidelines for Log Size on page 33
Determining Storage Requirements on page 34
Using Event Markers on page 35
Using the rn_shutdown script on page 37
Application Information Check List on page 38
Database Information Check List on page 38
Network Information Check List on page 39
Storage Information Check List on page 39
General Information Checklist on page 41
Allocating Space for RecoverNow Logs and Journals

RecoverNow records write information into log files. Logs are block
storage devices of the same size. Typically, multiple logs are contained in
a Logical Volume. The pool of disk storage that contains the logs is called
a journal. A production journal exists on the production server and a
recovery journal exists on the recovery server.
29
Chapter 2: Planning your Environment
The production journal is the storage that contains all of the logs. A single
log is transferred to the recovery server when that log is filled. For example,
if each LFC is 64MB and there are 100 production LFCs, then the
production journal is 6400MB. When the current LFC is filled with
approximately 64MB of write I/O data (there is some additional metadata),
it will be transferred to the recovery server.
All logs in the production journal are redo logs. They contain information
that moves the disk image of the application forward through time when the
information is applied. This is called rolling forward.
Half of the logs in the recovery journal are redo logs, and half are undo logs.
Undo logs contain information that moves a disk image of the application
back through time when they are applied. This is called rolling back.
The recovery server also contains the snapshot journal. The snapshot
journal is the space on the recovery server where RecoverNow stores
copy-on-write information and write-cache data for snapshots.
The following table shows the variables that are used for estimating journal
sizes and log sizes. You need these estimates in order to configure
RecoverNow:
Concept
Meaning
Throughput
Disk write rate of the application over a business cycle,

typically 24 hours.
Average
throughput
Average write rate of the application during a business cycle
Peak throughput
Maximum write rate of the application during a business cycle.
Peak duration
Average length of time that peak throughput occurs.
Bandwidth
Maximum throughput supported by RecoverNow. It is typically

limited by the network that connects the production and
recovery servers.
NOTE
Use a tool such as iostat to estimate throughput. You can also use
the Sizing tool to estimate throughput. For more information, refer
to Using the Sizing Tool to Calculate LFC Size on page 43.
Guidelines for Production Journal Size

The production journal holds redo log buffers until the logs are transferred
to the recovery server. Then, the logs are available to receive new
application write data.
30
Guidelines for Production Journal Size
The goal of sizing the production journal properly is to prevent the recovery
server from falling so far behind the production server that dynamic
recovery must occur for the recovery server to catch up. If the production
journal is too small, then transfers between the production server and the
recovery server are performed more frequently than is efficient.
The appropriate size of the production journal is proportional to one of the
following:
No-impact downtime: The length of network or recovery server

downtime that RecoverNow can sustain without falling into dynamic
recovery.
The amount of data in throughput spikes that exceed system bandwidth

that RecoverNow can sustain without falling into dynamic recovery
Production Journal Size Estimate 1

If you are unable to use a tool such as iostat to estimate write throughput,
then allow at least 128 MB for the production journal. You can also use the
Sizing tool to estimate write throughput. For more information, refer to
Using the Sizing Tool to Calculate LFC Size on page 43.

This estimate uses the following information:
An estimate for average write throughput
The length of no-impact downtime that is required
Calculate production journal size as follows:

production journal size = (average write throughput) x (no-impact
downtime)

This estimate assumes the following:
Estimates for peak throughput, peak duration, and bandwidth exist.
Peak throughput must be greater than bandwidth.
RecoverNow must not fall into dynamic recovery when write spikes
exceed bandwidth.
Calculate production journal size as follows:

production journal size=[(peak write throughput)- bandwidth]x(peak
duration)
31
Production Journal Size Best Estimate

The best estimate for the size of the production journal is the maximum of
the above estimates 1, 2, and 3.
Guidelines for Recovery Journal Size

The recovery journal holds redo and undo log buffers, which act as
RecoverNows internal rollback window. If you are using external archive
media such as tape, the size of the recovery journal is not critical to the
ability to restore data. However, the larger the recovery journal, the larger
the internal rollback window, which implies faster access to redo and undo
logs during production restores in that window.
If you are not using external archive media, then the size of the recovery
journal is critical for data protection. Rollbacks cannot extend beyond the
logs that exist on the recovery server. You must estimate average throughput
and calculate recovery journal area based on the length of the desired
average restore window.
The size of the recovery journal is proportional to:
Throughput
Required internal rollback window
Recovery Journal Size Estimate 1

The recovery journal should be at least 256 MB. Note that this is twice the
space recommended for the minimum size of the production journal,
because the recovery server contains both redo logs and undo logs.

Estimate for average throughput
The internal rollback window is longer than the business cycle.
Calculate recovery journal size as follows:

recovery journal size = 2 x (average write throughput) x (rollback
window)

32
Estimates for peak throughput and peak duration
Guidelines for Selecting Volumes to Be Protected
The internal rollback window is shorter than the business cycle.

Calculate recovery journal size as follows:
recovery journal size = 2 x (peak write throughput) x (rollback window)
Guidelines for Selecting Volumes to Be Protected

If a filesystem is protected by RecoverNow and uses jfslog or jfs2log, then
that corresponding jfslog and jfs2log must also be protected.
Guidelines for Snapshot Journal Size

The snapshot journal should typically be between 10 and 30 percent of the
total size of the protected volumes. Smaller snapshot journals are
appropriate when snapshots are not kept for a long time and when
production I/O is light. Larger snapshot journals are necessary when you
want to keep snapshots for an extended period of time and when heavy
production I/O activities are required.
IMPORTANT
You can also use the Sizing tool to calculate write journal pool
size. For more information, refer to Using the Sizing Tool to
Calculate LFC Size on page 43.
Guidelines for Log Size

All logs are the same size on the production server and the recovery server.
The log size affects the amount of data that RecoverNow processes for each
transaction.
Increasing log size has the following benefits:
RecoverNow processing is reduced.
Elimination of common blocks in the undo logs is more efficient.
Decreasing log size results in a more up-to-date replica on the recovery

server because log transfers occur more frequently.
When you determine the best log size, keep these conditions in mind:
The production journal should contain at least 8 logs.
Minimum log size is 2 MB.
33
NOTE
Use the following equation to ensure that the space you allocate
for LFCs coincides with the physical partition size of the VG
where the LFC LVs are allocated. This enables you to utilize all
the space in a LV. This is not a requirement, you can elect to not
utilize all the available space in a LV.
y = (number of LFCs / number of LFC LVs) * LFC size
Where y
should be evenly divisible by the physical partition size of the VG
where the LFC LVs are allocated.
Maximum log size is one-half of the available RAM on the recovery

server.
To calculate log size, you need the following information:
Estimate of average write throughput
Required processing rate. For example, if RecoverNow processes one

log every 60 seconds, the recovery server will be one minute behind the
production server.
Calculate log size as follows:

log size = (average write throughput)/(processing rate)
Calculate the number of logs as follows:
number of logs = (journal size)/(log size)
Determining Storage Requirements

Consider the following guidelines to determine storage requirements:
1. Use the following assumptions to decide on the appropriate storage
requirements.
Recovery Window goal = 1 day
Average Aggregate Write I/O per day = 1 GB
Network outage protection = 12 hours
Amount of data and metadata that will fit in an LFC:
amount of data = .95 of LFC size
amount of metadata = .05 of LFC size
34
Using Event Markers
Therefore, a 16MB LFC will hold 15.2 MB of data.

2. Using these assumptions, determine the LFC space.
Production LFC space __1GB x .5 = 500MB_________
Recovery LFC space __2 x (1GB) x 1 = 2 GB_________
3. Using this space, calculate the LFC size and the number of production
and recovery LFCs.
LFC size _16 MB___________________________
Production LFCs __500 /(16 * .95)= 33________________
Recovery LFCs ___2000/(16 * .95) = 132_______________
4. If there is space available to create a separate volume group, create a
volume group that is large enough to hold the LFCs.
Using Event Markers

Event Markers are tags that mark points in time or points in process that are
significant to you for the purposes of recovery. An Event Marker can be
selected as the Recovery Point Objective (RPO) when creating snapshots,
performing a Production Server Rollback or Rollback the failover server
prior to performing an unplanned failover. They are typically needed for
applications which cannot take advantage of RecoverNows Any
Point-In-Time (APIT) data restores along with applications which do not
have live transactional durability on disk.
Event Marker File and the rtmark command

To use event markers, you need to a write an event marker file and a script
that calls the rtmark command. You also need to set up a call in your
application that invokes the rtmark script.
If event markers are used in multiple Context IDs, each Context ID must
have a unique event marker file and scripts that calls rtmark.
The following examples show the parameters you need to use in your
application; an event marker file, and a script that calls rtmark:
#example call in your application
<Context ID> <name> <description>
# example event mark script that calls rtmark and emf_1 file
rm -f /tmp/emf_1
printf "name = ${2}\n" >/tmp/emf_1
printf "description = ${3}\n" >>/tmp/emf_1
sync
35
/usr/scrt/bin/rtmark -C ${1} /tmp/emf_1

#example event marker file /tmp/emf_1
name = <event name>
description = chkpt
Event marker file replication from recovery to

replicated server
Event marks are written to file /usr/scrt/run/c<Context ID>/event_marker
and replicated from the production server to the recovery server. If your
configuration includes a replicated server, the event marker information
needs to be replicated from the recovery server to the replicated server. On
the recovery server, use the rtmark command (shown below) to replicate the
event marker information from the recovery server to the replicated server.
rtmark C <Context ID> -r
NOTE
Event markers are lost on failover and failback because the

rollback window is reset. The production servers event_marker
file is cleared during a failback to the configured production
server. The failover servers event_marker file is cleared during a
failover to the configured failover server.
Print Available event marks for rollback

Use the scrt_ra command with the ve parameters to print the Available
event marks for rollback.
scrt_ra veC<Context ID>
Available event marks for rollback:

-----------------------------------Event Name
Date/Time (Epoch Time)
Description
End_of_day
"Wed Apr 4 15:00:31 2012" (1333566031)
End of day processing
DB_Migration
"Fri Nov 25 08:12:14 2011"(1322226734)
DB Migration completed
36
Using the rn_shutdown script
Using the rn_shutdown script

The purpose of the rn_shutdown script is to stop and unload the drivers for
all Double-Take RecoverNow Context IDs on the system where the AIX
shutdown command is executed.
Steps performed by the rn_shutdown script

The rn_shutdown script performs the following actions:
Kills all processes associated with a mounted filesystem that is

configured in a Double-Take RecoverNow Context ID.
Kills all processes associated with a character device that is configured

in a Double-Take RecoverNow Context ID.
Stops RecoverNow and unloads the drivers.
If rtstop fails for a Context ID it will be marked failed and then continue
to do rtstop on any remaining Context IDs. After all the Context IDs are
processed the exit status will be set to 0, or a value equal to the number
of Context IDs that were previously marked as failed. The reason for failure
will be recorded in the /usr/scrt/log/rn_shutdown.out file. A non 0 exit
will abort the AIX shutdown.
To have /usr/scrt/bin/rn_shutdown execute when the AIX shutdown
command is executed, it must be called from /etc/rc.shutdown.
For example, add the following to /etc/rc.shutdown.
#!/bin/ksh
/usr/scrt/bin/rn_shutdown
if(($? != 0))
then
printf "ERROR: rn_shutdown failed aborting.\n"
exit 1
fi
37
Application Information Check List

Application Data Information
(to be protected by RecoverNow)
Customer Information
What is the location of the data

associated with the application?
Specifically, what bus, spindle, HBA,
or LUN contains the data?
What are the characteristics of the
data?
Are there licensing issues related to
running the application on the
recovery server?
Which applications or services need
to be highly available?
Will the application be running
during RecoverNow installation?
Host name dependency?
IP name dependency?
Database Information Check List

Database
Database product and release

Supported in the high availability
(cluster) environment?
Supported for the operating system
release?
Which volume groups contain the
database or table to be protected?
What is the process for keeping the
data consistent?
38
Domain Name Server Check List
Domain Name Server Check List

Domain Name Server
Is DNS enabled?
Is Network Information Services
(NIS) mounted?
Is NIS exported?
Network Information Check List

Network Information Needed
What kind of network connects the

production server and the recovery
server?
Are there firewall restrictions such as
port limitations between any of the
machines?
Which network connections are
shared and which are dedicated?
What are the IP addresses of the
following:
Administrative Server
production server
recovery server
replicated server
What routers, bridges, and hubs are
part of the system?
Is the network currently established,
or will it be established during
RecoverNow installation?
Storage Information Check List

Storage Information Needed
Is storage direct-attached or SAN?

If SAN, what are the HBA and
firmware levels?
39
Storage Information Needed
Disk IDs (hdisk) and associated

volumes (including the JFS log) and
file systems
Space required for the file systems,
including potential growth (number
and size)
Is there logical volume manager
mirroring?
What volumes are to be protected?
Which volumes store the
application?
Are the application volumes
logically shared?
Are the volumes physically shared?
Is the disk subsystem already
established, or will it be established
during RecoverNow installation?
Tivoli Storage Manager

Information Needed
Release:
Policy domain
Policy set
Management class
Backup copy
Archive copy
Storage pools:
Disk
Tape
Policy domain:
Type
Number
Shared?
40
General Information Checklist
General Information Checklist

Additional Information Needed
Is FTP available on all servers?

If FTP is not available, are
CD-ROM drives available?
Does the application need specific
volume or file system names?
Logical volumes to be assigned to
RecoverNow
File system names and mount points
Number, size, and location of
RecoverNow redo logs. Refer to
Allocating Space for RecoverNow
Logs and Journals on page 29
How much time does the restore
window encompass?
41
42
Using the Sizing Tool to Calculate

LFC Size
Overview of LFC Sizing

RecoverNow uses Logical File Containers (LFC) to transfer data from the
production Server to recovery server. Before you install RecoverNow,
you need to define the number of LFCs for the production and recovery
servers. On the recovery server, you must also define the Write Journal
(WJ) disk space percentage.
You can use the Sizing Tool to calculate configuration values before
RecoverNow is installed. It is also useful to run the tool after
RecoverNow is installed to determine if the number of LFCs or WJ
percentage needs to be adjusted.
This chapter describes:
System Requirements for the Sizing Tool on page 43
Installing the Sizing Tool on page 44
Running the Sizing Tool from the RecoverNow Sizing Tool GUI on
page 44
Running the Sizing Tool Script from the Command Line on

page 51
System Requirements for the Sizing Tool

Keep in mind the following system requirements before you run the
Sizing tool:
Can only be run on AIX.
The sztool_gui requires Java 1.5 or greater.
The sztool_gui requires Xwindows.
43
Chapter 3: Using the Sizing Tool to Calculate LFC Size
Installing the Sizing Tool

To install the Sizing tool refer to the following sections in Chapter 5
Installation Procedures on page 59:
AIX Installation on page 64
Use smit to Install RecoverNow on page 103

NOTE
The RecoverNow Installation Wizard and the smit installation

program provide the sztool command for the command line Sizing
Tool and the sztool_gui command you can use to access the
RecoverNow Sizing Tool GUI.
Running the Sizing Tool from the RecoverNow Sizing

Tool GUI
To access the RecoverNow Sizing Tool GUI, type
/usr/scrt/sztool/sztool_gui from the command line.
The RecoverNow Sizing Tool GUI window displays. The first tab,
Introduction displays, by default.
There are four RecoverNow Sizing Tool GUI tabs:
Introduction Tab on page 44
Config and Run Tab on page 46
View and Analyze Log Tab on page 48
View Chart Tab on page 50
Introduction Tab
The Introduction page describes how you use the sizing tool. For detailed
information, click Help. This button displays the URL to access the Vision
Solutions Support web site. From this site you can download documentation
that describes how you use the sizing tool. In addition, you are provided
with CustomerCare support email and phone numbers. Click Exit to exit the
RecoverNow Sizing Tool GUI.
44
45
Config and Run Tab

You use the Config and Run page to configure LVs. Perform the following
activities to configure LVs:
To select individual LVs, use the check box next to the LV, to select the
LVs that will be protected by RecoverNow.
Click the Select all LVs button to select all LVs.
Click the Run Disk Discovery Again button to re-discover the LVs.
The table below describes the parameters that you can modify:
46
Parameter
Description
Collection Interval
Count
Specifies how many times you want to collect data. The

default value is 24 hours.
Collection Interval
Minutes
Specifies how many minutes to wait between each data

collection interval. The default value is 60 minutes.
Parameter
Description
Lfc Size (MB)
Specifies the size for the RecoverNow LFC. The default value
is 16 MB.
Replication Outage
Hours
Specifies the hours that the production server can not send LFCs
to the replicated server. When this occurs, the LFCs will begin
to backup on the Production Sever, until there are no more LFCs
available. Once RecoverNow runs out of LFCs, it marks the
regions which require synchronization in the state map as dirty.
These dirty regions will automatically be synchronized when
the LFCs become available. CDP functionality will resume as
soon as the resynchronization completes. More LFCs are
required as outage time increases. The default value is 8 hours.
CDP Window
Hours
Specifies how many hours the data can go back in time to

restore data from the recovery server to the production server.
The window size determines the number of LFCs on the
recovery server. The default value is 8 hours.
Snapshot Duration
Hours
Specifies the number of hours you want to keep a snapshot

valid. As the snapshot duration hours increase you need to
increase the Write Journals disk space. The default value is 8
hours.
The Run button becomes active, when you select an LV(s) and specify
values for the LV parameters.
NOTE
Before you click the RUN button, start your application on the
selected LVs, and ensure that your application has a heavy load, so
the tool collects enough data to reflect the activity for a worst case
scenario.
Click the RUN button to:

Save the configured data into /tmp/sztool/sztool.cfg file
Invoke the backend sztool script
Display the Job Progress Window. This window shows the sztool
job status, and enables you to:
Close the window and re-display it by clicking the Get Job Status
button.
Halt the current running background job
Exit from sztool and check results
47
NOTE
You can view /tmp/sztool/sztool.log on the production server,

once the back end sztool job (scripts) is running,
View and Analyze Log Tab

You use the View and Analyze Log page to view the log file output from the
sztool script sizing tool.
48
This page contains three sections:
ResultsThis section shows general information for the sztool script

sizing log file. Click the Display Log button to display the results
derived from the original log file. You can view:
Production server Number of LFCs and the production server disk
space requirements
Recovery server Number of LFCs and the recovery server disk space
requirements
Minimum Write Journal Percentage
Detailed logs from latest runThis section shows a scrollable text area
containing detailed statistics for the sztool script sizing log file. The log
file name is /tmp/sztool/sztool.log Click the Display Log button to
display the results derived from the original log file. The columns show:
Logical Volume
49
IO Count
Kb read
Kb written
Kbps (kilobit per second)
Try different parameters to get results from the already collected data.
You can edit the parameters shown below to see different log file
results.
Lfc Size (MB) Low. Refer to Lfc Size (MB) on page 47.
CDP Window Hours. Refer to CDP Window Hours on page 47.
Replication Outage Hours. Refer to Replication Outage Hours on
page 47.
Snapshot Duration Hours. Refer to Snapshot Duration Hours on
page 47.
To see different log file results:
1. Change the LFC size and CDP Window Hours.
2. Click the Show results with above parameters button.
The sztool script executes against the collected data to display log
file information in the Results section and the Detailed logs from
last run section.
NOTE
This output will not overwrite the /tmp/sztool/sztool.log file

contents.
3. Click the Display Log button if you wish to re-display the results
derived from the original log file.
View Chart Tab

The Chart data represents the time with respect to total disk writes over
time, on all selected LVs. You can view data for individual LVs in the
sztool.log. The unit of writes is in MB.
NOTE
A pdf version of the Chart is automatically saved in

/tmp/sztool/DiskWriteChart.pdf.
50
Running the Sizing Tool Script from the Command

Line
Perform the following steps to run the sztool script from the command line.
NOTE
Before you run the Sizing tool you must have performed the
installation steps described in Installing the Sizing Tool on
page 44.
1. To start the RecoverNow Sizing Tool, type
usr/scrt/sztool/sztool from the command line. The tool
automatically creates a working directory /tmp/sztool, a config file
/tmp/sztool/sztool.cfg and a diskinfo file /tmp/sztool/diskinfo.
The diskinfo file contains a list of all LVs on the system. The user
selects which LVs should be protected by RecoverNow. The working
directory is for storing the config file, log file and tmp files. The
51
configuration file is for user to specify LV names and other run time
parameters.
2. Review the diskinfo file and determine which LVs RecoverNow should
protect.
3. Modify the parameters in the sztool.cfg configuration file shown
below:
LVs_1=testloglv testlv
LVs_2=
LVs_3=
LVs_4=
LVs_5=
LVs_6=
LVs_7=
LVs_8=
LVs_9=
LVs_10=
Collection_Interval_Count=24
Collection_Interval_Minutes=60
Lfc_Size_MB=16
Replication_Outage_Hours=8
CDP_Window_Hours=8
Snapshot_Duration_Hours=8
The table below describes the configuration file parameters that you can
modify:
Parameter
Description
LVs_1=
For example, you can enter LVs_1=LV1 LV2

LV2 LV4, with a space between each LV.
Typically, you place three or five LV names per
line as appropriate. You can add more LV
names for any additional lines. If there are not
any additional LV names, leave the remaining
LVs lines blank.
LVs_2=
.....
LVs_10=
52
Collection_Interval_Count
Specifies how many times you want to collect

data. The default value is 24 hours.
Collection_Interval_Minutes
Specifies how many minutes to wait between

each data collection interval. The default value
is 60 minutes. Set the
Collection_Interval_Count to 24 hours and the
Collection_Interval_Minutes to 60 minutes.
This is referred to as the 24*60 setting. This is
the recommended configuration to use when
you execute the sztool script.
Lfc_Size_MB
Specifies the size for theRecoverNow LFC.
Parameter
Description
Replication_Outage_Hours
Specifies the hours that the production server

can not send LFCs to the replicated server.
When this occurs, the LFCs will begin to
backup on the Production Sever, until there are
no more LFCs available. Once RecoverNow
runs out of LFCs, it marks the regions which
require synchronization in the state map as
dirty. These dirty regions will automatically be
synchronized when the LFCs become available.
CDP functionality will resume as soon as the
resynchronization completes. More LFCs are
required as outage time increases.
CDP_Window_Hours
Specifies how many hours the data can go back

in time to restore data from the recovery server
to the production server. The window size
determines the number of LFCs on the recovery
server.
Snapshot_Duration_Hours
Specifies the number of hours you want to keep

a snapshot valid. As the snapshot duration
hours increase you need to increase the Write
Journals disk space.
4. Start your business application on the selected LVs. The load of the
businesses application should be as close to the worst case scenario to
ensure a meaningful result.
5. Type /usr/scrt/sztool/sztool from the command line to restart
the sztool. The sztool runs in background and you can check the results
in the /tmp/sztool/sztool.log. Check the
/tmp/sztool/sztool.log file to be sure the tool is running, one
collection interval is required before LV I/O data is written to the
/tmp/sztool/sztool.log. Check the
/tmp/sztool/sztool.log after the last collection interval count for
the final result. It is safe to log out from the terminal because sztool
uses nohup. The process will take 24 hours with the default collection
interval count of 24 and collection interval of 60 minutes.
6. When the tool completes, check the log file or the AIX window. At the
bottom of the log file or AIX window, the "<<---------<" lines
indicate the production and recovery server number of LFCs, and the
percentage of Write Journal (WJ). The log file also contains detailed
information for each LV IO statistics for each data collection interval.
The standard log file is called sztool.log. An additional copy of the log
file is also created, sztool.log-MM_DD_YYYY-HH:MM:SS. For
example: sztool.log-02_19_2010-14_22_19, where HH is the 24 hour
interval.
53
sztool script Command Options

The table below describes the sztool script command options.
sztool
script
Command
Options
54
Description
sztool
If issued for the very first time, the working directory, diskinfo
file and sztool.cfg file are generated. You should review the
diskinfo file and then modify sztool.cfg, accordingly. You can
then re-run sztool.
sztool -c
Assigns CDP_Window_Hours a value and run against existing

data log file. Only screen output. Example: sztool -c24.
sztool -d
Generates the /tmp/sztool/diskinfo file. Performed automatically

when you run sztool for the first time.
sztool -g
Generates the sztool.cfg file in /tmp/sztool. Performed

automatically when you run sztool for the first time.
sztool -h
Prints out a help message.
sztool -l
When the log file is created, this command prints out the
calculation results for different LFC sizes based on the existing
log file. For example, sztool -l32 prints out the results when the
LFC size is at 32M. sztool -l16 -l512, prints out all the
calculation results from 32MB to 512MB. You cannot have
spaces between -l and the LFC size number. Only screen output,
there is not any delay or sleep.
sztool -r
Assigns Replication_Out_Hours a value and runs against the

existing data log file. Only screen output. For example: sztool
-r24, or sztool -r24 -l2 -l32.
sztool -s
Assigns Snapshot_Duration_Hours a value and runs against the

existing data log file. Only screen output. Example: sztool -s12.
sztool -x
Executes the sztool and prints the file name and line number of
the statement for debugging purposes. For debugging, use
sztool_main -x to view screen output.
sztool -v
Prints out the sztool version number.
Supported Configurations
RecoverNow supported configurations are determined by the number of

datataps (kernel level software that intercepts application writes) and
recovery servers. Some of the configurations supported by RecoverNow
are presented in this chapter.
Local CDP Solution Configuration on page 55
Local CDP Solution with WAN Replicated Configuration on

page 56
WAN Replication with No Local CDP Configuration on page 56
Local Bi-directional Replication Configuration on page 56
Remote Bi-directional Replication Configuration on page 57
Local CDP Solution Configuration

This configuration involves one or more datataps, one recovery server, as
well as one LAN-based (100Mb Ethernet, GigE) network connection
between the Production and the recovery servers.
Production
Server
LAN
Recovery
Server
Datatap
55
Chapter 4: Supported Configurations
Local CDP Solution with WAN Replicated Configuration

This configuration involves one or more datataps, one recovery server, and
a WAN connection to the replicated server.
LAN
Production
Server
WAN
Recovery
Server
Replicated Server
WAN Replication with No Local CDP Configuration

This configuration involves one or more datataps and one recovery server.
When developing Double-Take RecoverNow solution, the WAN bandwidth
as well as application peak throughput write I/O rates need to be considered.
Production
WAN
Remote Recovery
Server
Server
Local Bi-directional Replication Configuration

This configuration involves one or more datataps and two recovery servers
that are associated through a LAN connection. Two interrelated contexts are
created.
The LCA/ABA agents are used in a bi-directional manner. These agents
need to be configured before the failover/failback operations are used. For
more information about RecoverNow failover/failback operations, refer to
Chapter 12 Introduction to Disaster Recovery on page 231.
56
Remote Bi-directional Replication Configuration
NOTE
The recovery server becomes the failover production server in the

bi-directional configurations. For more information about RecoverNow
failover operations, refer to Chapter 12 Introduction to Disaster
Recovery on page 231.
LAN
Production
Server
Recovery
Server
Remote Bi-directional Replication Configuration

This configuration involves one or more datataps, one recovery server, and one
replicated server that are associated through a WAN connection. Two interrelated
contexts are created.
The LCA/ABA agents are used in a bi-directional manner. These agents need to
be configured before the failover/failback operations are used. For more
information on RecoverNow failover/failback operations, refer to Chapter 12
Introduction to Disaster Recovery on page 231.
LAN
Production
Server
Local Recovery
Server
WAN
Replicated
Server
57
Chapter 4: Supported Configurations
58
Installation Procedures
Overview
This chapter describes RecoverNow, RecoverNow Portal Application and
Vision Solutions Portal (VSP) installation procedures.
Before you begin, ensure that you review support information, system
requirements, and decide on your preferred configuration. One you have
installed the RecoverNow components you can work with VSP. Refer to
Logging in to Vision Solutions Portal on page 127.
VSP and RecoverNow Portal Application System Requirements on

page 61
RecoverNow Installation Wizard Prerequisites on page 62
Request License Keys from AIX or Windows on page 63
AIX Installation on page 64
Install Double-Take RecoverNow, the Vision Solutions Portal and

the Double-Take RecoverNow Portal Application on page 64
User Roles on page 83
Reinstall RecoverNow on page 83
Upgrade RecoverNow on page 92
Use smit to Install RecoverNow on page 103

User Roles on page 108
Uninstall RecoverNow on AIX on page 110
Use smit to Uninstall RecoverNow on page 116
Install the Vision Solutions Portal and Double-Take RecoverNow

Portal Application on Windows on page 118
Uninstall the Vision Solutions Portal and Double-Take RecoverNow

Portal Application Uninstallation Wizard on Windows on page 123
59
Chapter 5: Installation Procedures
Uninstall the Vision Solutions Portal and Double-Take RecoverNow

Portal Application Uninstallation Wizard on Windows on page 123
System Requirements for RecoverNow

RecoverNow requires an IBM System p with the AIX operating systems.
NOTE
RecoverNow supports Internet Protocol version 6 (IPv6).
Operating System Requirements

RecoverNow runs on the IBM AIX 5L, AIX 6.1, and AIX 7.1 operating
systems. Supported versions include:
AIX version 7.1
AIX version 6.1
AIX 5L version 5.3 TL 4 or greater
AIX 5L version 5.2 TL 9 or greater

NOTE
AIX 5L version 5.3 TL5 64-bit kernel requires APAR IY92292. It

is currently available at the IBM Web site.
Disk Space and Memory Requirements

Review the RecoverNow disk space and memory requirements:
RecoverNow requires 1 MB in root and 125 MB in /usr. The 125 MB in

/usr is for the /usr/scrt directory which contains binaries, libraries,
configuration, and logs.
IMPORTANT
The /home/usr/scrt/ directory is created when the scrt user is

created in AIX. Do not copy any files to this directory because
RecoverNow deletes this directory during the de-installation
process.
60
The amount of RAM required for the RecoverNow application depends

on the size of the protected data (StateMap size) and log size. The
maximum log size is one half of RAM.
VSP and RecoverNow Portal Application System Requirements
The disk space requirement for RecoverNow file containers is

approximately 500 MB for a small configuration
The disk space requirement is approximately 70 MB for a small

configuration and diagnostic logs.
RecoverNow requires at least 128 MB for logs on the production server,

and at least 256 MB for logs on the recovery server for LFCs. The LFCs
on the recovery server contain the undo and redo logs.
The calculation for the undo and redo logs is based on the required
recovery window and the network outage protection size. Refer to
Determining Storage Requirements on page 34. If you use the
snapshot journal, ensure that you take into account its size. Refer to
Guidelines for Snapshot Journal Size on page 33.
Archiving Products Supported

RecoverNow supports archiving LFCs and making backups of Replica LVs
using the Tivoli Storage Manager on AIX.
VSP and RecoverNow Portal Application System Requirements

The Vision Solutions Portal (VSP) and Portal Application (PA) have the
following system requirements.
The required disk space to install VSP and the RecoverNow portal
application is 280 MB in /opt.
The required physical RAM is 1 GB for VSP.
Internet Explorer 7, or 9, or Firefox 3.5 or above.

NOTE
VSP supports Internet Protocol version 6 (IPv6).
RecoverNow Installation Components

During installation, you can designate the components you want to install.
RecoverNow components include:
Base RecoverNow
You must install the Base RecoverNow software on each AIX cluster
node directly.
The scrt user and group are created for RecoverNow.
Sizing Tool
61
You can use the Sizing Tool to calculate configuration values before
RecoverNow is installed. However, it is also useful to run the tool after
RecoverNow is installed to determine if the number of LFCs or WJ
percentage needs to be adjusted. Refer to Using the Sizing Tool to
Calculate LFC Size on page 43.
Documentation
Documentation is available as.pdf files.
RecoverNow Installation Wizard Prerequisites

If you use the RecoverNow Installation Wizard the following prerequisites
are required on the AIX target system:
Either ssh and scp or rexec and rcp must be allowed. If ssh fails then
rexec and rcp is used.
If rexec will be used rexecd must be configured in /etc/inetd.conf.
To use rcp the ~root/.rhosts file must have the local host and user name.
Check /etc/services to find the ports used by exec and shell and check
that those ports are not blocked.
There is the option available to send a ping to the AIX system to

determine if it is reachable.
This requires that the echo port is not blocked. This is usually defined
as port 7 in /etc/services
NOTE
If you exit the installation wizard you can view the detailed errors
in the RecoverNow_4.n.n.n_Install.log
Before You Install

The installation process involves two steps:
1. Request a license for each node as described in Request License Keys
from AIX or Windows.
2. Install RecoverNow as described in AIX Installation on page 64.
Each target node must be connected by the network to the
production and recovery server(s). Make sure that host names and IP
addresses of RecoverNow servers are accessible from the
workstation.
62
Request License Keys from AIX or Windows
NOTE
If you are installing GeoCluster and RecoverNow, refer to the

Double-Take Availability Integration Guide.
Request License Keys from AIX or Windows

You can request license keys for RecoverNow from an AIX machine or a
Windows PC:
1. Download the RecoverNow and GeoCluster License Key Wizard from the
Vision Extranet (Click Customer/Partner Login at www.visionsolutions.com)
under the License Keys menu option.
2. Do one of the following:
a. For AIX, run install.bin.
b. For Windows, click Update License Keys on the Double-Take
RecoverNow splash screen.
The Double-Take RecoverNow Welcome screen displays.
3. On the Welcome screen, click Next.
4. On the Specify Node screen, specify a node where you want to apply license
keys. The node can be the name or IP address of the local node or a node in
the network.
5. Click Next.
6. On the Node Login screen, to obtain new license keys for RecoverNow, log in
to the node as root (AIX) or as an administrator (Windows).
7. Click Next.
8. On the Retrieving Installation Information screen, click Next.
9. On the Confirm Node Selection screen, you can add or remove nodes that will
be used by the wizard to install or upgrade. To add a node:
a. Specify a node name or IP address
b. Click Add
c. Click Next. On the Node Login screen, provide the User name and
password for the node.
d. Click Next
63
10. On the License Key Locations screen, specify or browse to the location
of the license key file for RecoverNow obtained from Vision Solutions
for each node.
11. Click Next.
12. On the License Key Check screen, click Next to display the Summary
screen.
13. To specify another node click Previous. To cancel, click Cancel.
AIX Installation
This section describes:
Install Double-Take RecoverNow, the Vision Solutions Portal and the

Double-Take RecoverNow Portal Application on page 64
Install only the Vision Solutions Portal and the Double-Take

RecoverNow Portal Application on page 77
Reinstall RecoverNow on page 83
Upgrade RecoverNow on page 92
Install Double-Take RecoverNow, the Vision Solutions

Portal and the Double-Take RecoverNow Portal
Application
1. Download the RecoverNow installation program from the:
Vision Extranet at Customer/Partner Login at
www.visionsolutions.com. Proceed to step a.
Double-Take RecoverNow installation CD. Proceed to step 2.
a. Click Double-Take RecoverNow for AIX in the left-hand pane.
b. Click Product Download in the right-hand pane.
c. Select and download the RecoverNow product zip file.
a. Click Setup.exe, to display the Double-Take RecoverNow
Installation splash screen. Then, click Install Double-Take
RecoverNow.
b. Execute install.bin, to run the RecoverNow Installation Wizard.
c. Run install.exe to install onto a remote AIX machine.
64
AIX Installation
The Double-Take RecoverNow Installation Wizard runs and

displays the Double-Take RecoverNow Welcome screen.
3. Click Next.
The Terms And Conditions screen displays. Read and accept the terms
of the License Agreement.
4. Click Next.
The Select Product screen displays.
65
5. Select either:
Double-Take RecoverNow and Vision Solutions Portal
(recommended) Installs Double-Take RecoverNow, the portal
application, and optionally, the Vision Solutions Portal to each node.
Proceed to step 6 on page 66.
Vision Solutions Portal and the Double-Take RecoverNow portal
applicationInstalls only the portal application and VSP, if
necessary. Proceed to Install only the Vision Solutions Portal and
the Double-Take RecoverNow Portal Application on page 77.
6. Select Double-Take RecoverNow and Vision Solutions Portal
(recommended).
7. Click Next.
The Install or Upgrade screen displays.
66
AIX Installation
8. Select an install, or upgrade or reinstall option.

Create a new installation Create a new RecoverNow installation.
Upgrade or reinstall an existing installationUpgrade or reinstall an
existing RecoverNow installation. To upgrade RecoverNow proceed
to Upgrade RecoverNow on page 92. To reinstall RecoverNow
proceed to Reinstall RecoverNow on page 83.
IMPORTANT
The following steps detail how to create a new installation

9. Click Next.
The Specify Node screen displays.
67
10. Enter the node namePoints the installer to the target node where you
want to install RecoverNow. Alternatively, you may enter an IP address.
11. Click Test ConnectionA dialog displays indicating whether or not the
connection to the node was successful.
12. Click Next.
The Node Login screen displays.
68
AIX Installation
13. Enter the username and root password.

NOTE
You must log into the node as root.

14. Click Log In to log in to the node.
15. Click Next.
The Retrieving Installation Information screen displays.
16. Click Next.

17. The Specify Nodes screen displays.
69
18. Enter the node name or IP address, and click Add to have RecoverNow
software installed on this node.
19. Click Next.
20. Enter the password, and click Log In.

21. Click Next.
70
AIX Installation
After all the installation information is successfully retrieved, the

RecoverNow Options screen displays.
Depending upon the type of installation you are performing the

RecoverNow Options screen displays the following messages and
options:
71
If you are creating a new installation, the RecoverNow Options

screen displays a disabled checkbox labeled Install, to confirm the
nodes on which you will install RecoverNow. The following steps
describe how to install RecoverNow.
If you are reinstalling, the RecoverNow Options screen displays a
checkbox labeled Reinstall, which enables you to reinstall
RecoverNow on nodes where the same version is already installed.
Proceed to Reinstall RecoverNow on page 83.
If you are upgrading Displays a a checkbox labeled Upgrade,
which enables you to upgrade nodes that have earlier versions of
RecoverNow installed. Proceed to Upgrade RecoverNow on
page 92.
Displays messages for nodes where RecoverNow cannot be installed
or upgraded.
22. Click Next
The Documentation Options screen displays.
23. Select the node for which the documentation will be installed.
24. Click Next.
The License Key Location screen displays.
72
AIX Installation

a. Click Next, if you have a license key files from Vision Solutions.
Proceed to step 26 on page 74.
b. Click Continue Without License Keys.
The Continue Without License Keys screen displays.
Select Complete the installation without license keys to install

RecoverNow without license keys. You can install RecoverNow
73
without license keys but cannot use it until valid license keys are
applied. Proceed to step 26 on page 74.
Select Contact Vision Solutions to get new license keys and
click Next, the Contact Vision Solutions screen displays. Use one
of the following methods to procure license keys.
On the InternetLog in to your account at:
VisionSolutions.com/SupportCentral
EmailCopy and paste the information on the Contact Vision
Solutions screen into your email message. When you contact
Customer Accounting to request a license, you will be asked
to provide the machine ID (uname -m) of your servers along
with the hostname, and your OS. Email your information to
Customer Accounting at support@visionsolutions.com and
request a license key. A license file will be generated and
emailed to you.
Telephone(800) 337-8214
NOTE
Once you procure the license keys from Vision Solutions, click
Next on the License Key Location to continue the installation.
Proceed to step 26 on page 74
26. Click Next.
NOTE
If you are reinstalling RecoverNow, the Product Shutdown

Required screen displays. You must shut down RecoverNow
before you reinstall it.
The License Key Check screen briefly displays (not shown) to validate
your license keys, then the Installing Double-Take RecoverNow screen
displays showing the results of the installation process for the
RecoverNow software.
74
AIX Installation
27. When the RecoverNow installation completes the Install Portal

Application screen displays.
28. Click Next.

When the Portal Application installation completes the Installing Vision
Solutions Portal screen displays.
75
When the installation completes, the Installation Complete screen

displays.
29. After you have installed RecoverNow, the RecoverNow Portal

Application, and the Vision Solutions Portal (VSP) you can log into
VSP. You can select one of the highlighted nodes to launch VSP and log
into VSP. See Logging in to Vision Solutions Portal on page 127.
30. Click Done to exit the installation wizard.
76
AIX Installation
Install only the Vision Solutions Portal and the

Double-Take RecoverNow Portal Application
1. Click Next on the Welcome and Terms and Conditions screens.
The Select Product screen displays.
2. Select Vision Solutions Portal and the Double-Take RecoverNow portal

application.
3. Click Next.
77
6. Click Next.
78
AIX Installation

NOTE

9. Click Next.
10. The Specify Nodes screen displays.
79
software installed on this node.
12. Click Next.
13. Enter the password, and click Log In.

14. Click Next.
80
AIX Installation
After the installation information is successfully retrieved, the Install

Vision Solutions Portal screen displays.
15. Click Next.

The Install Vision Solutions Portal screen displays.
81
When the installation completes, the Summary screen displays.
16. After you have installed the RecoverNow Portal Application, and the
Vision Solutions Portal (VSP) you can log into VSP. You can select one
of the highlighted nodes to launch VSP and log into VSP. See Logging
in to Vision Solutions Portal on page 127.
82
AIX Installation
User Roles
The installation process creates the scrt group in /etc/group, identifying the
category of users allowed to access the portal application.
IMPORTANT
The root user is always allowed access to the portal application.
Reinstall RecoverNow
Before you reinstall RecoverNow:
You must stop your application and RecoverNow on the node(s) where
RecoverNow is being reinstalled.
If you have RecoverNow fixes (epkg files) installed they must be

uninstalled before you can perform the reinstall. RecoverNow fixes
(epkg files) begin with the letters ES.
Use one the following methods to list all installed fixes (epkg files).
a. From the command line: emgr -l
b. From SMIT: use fast path by entering emgr
Use one the following methods to remove installed RecoverNow fixes.
From the command line:
a. From the command line: emgr -rL <LABEL>
To reinstall RecoverNow:
1. In the section, Install Double-Take RecoverNow, the Vision Solutions
Portal and the Double-Take RecoverNow Portal Application on
page 64, perform step 1 through step 6.
2. From the Select Product screen select, Double-Take RecoverNow and
Vision Solutions Portal (recommended) and click Next.
83
3. Click Next.
84
AIX Installation
6. Click Next.

NOTE

9. Click Next.
85
10. Click Next.

The Specify Nodes screen displays.
11. Enter the Node name or IP address and click Add. The node is added to
the list.
12. Click Next
13. Specify the password and click Log In, then click Next
The Retrieving Installation Information screen displays. After the
installation information is successfully retrieved, the RecoverNow
Options screen displays.
86
AIX Installation
14. Click Next to reinstall to each node.

15. Click Next.

License Key Options screen displays.
87
You have two options:

If you select Use existing license keys, the License Key Check
screen displays.
If you select, Specify the location of license keys, the License Key
Location screen displays.
16. After you specify license keys and they are validated, click Next.
The Product Shutdown Required screen displays.
88
AIX Installation
17. You must manually ensure that RecoverNow is shutdown, the wizard
does not perform this task. Click Next.
Once the installation wizard has verified that RecoverNow has been
shutdown the Shutdown Verification Complete screen displays.
18. Click Next to reinstall RecoverNow.
89
19. When the RecoverNow reinstall completes, the Install Portal

Application screen appears.
20. Click Next to reinstall the portal application and the Vision Solutions
portal.
The Shut Down Vision Solutions Portal screen displays.
90
AIX Installation
21. Select Yes, to shut down the Vision Solutions Portal and install the
Vision Solutions Portal and the portal application.
NOTE
You can also decide to skip the Vision Solutions Portal and the
portal application reinstall.
The Installing Vision Solutions Portal screen displays.
When the Vision Solutions Portal and the portal application reinstall
completes, a screen briefly appears stating that the reinstall was a success.
Then the Installation Complete screen displays.
91
22. After you have reinstalled RecoverNow, you can log into VSP. You can
select one of the highlighted nodes to launch VSP and log into VSP. See
Logging in to Vision Solutions Portal on page 127.
Upgrade RecoverNow
Before upgrading RecoverNow to the current version:
You must stop your application and RecoverNow on the node(s) being
upgraded.
If you have RecoverNow fixes (epkg files) installed they must be

uninstalled before you can perform the upgrade. RecoverNow fixes
(epkg files) begin with the letters ES.
Use one the following methods to list all installed fixes (epkg files).
a. From the command line: emgr -l
Use one the following methods to remove installed RecoverNow fixes.
From the command line:
a. From the command line: emgr -rL <LABEL>
To upgrade to the current version of RecoverNow:
92
AIX Installation
1. In the section, Install Double-Take RecoverNow, the Vision Solutions

Portal and the Double-Take RecoverNow Portal Application on
page 64, perform step 1 through step 6.
2. From the Select Product screen select, Double-Take RecoverNow and
Vision Solutions Portal (recommended) and click Next.
3. Click Next.
93
6. Click Next.
94
AIX Installation

NOTE

9. Click Next.
10. Click Next.

11. Enter the Node name or IP address and click Add. The node is added to
the list.
12. Click Next
13. Specify the password and click Log In, then click Next
The Retrieving Installation Information screen displays. After the
installation information is successfully retrieved, the RecoverNow
95
14. Click Next

15. Select the node for which the documentation will be installed.
16. Click Next.
The License Key Options screen displays.
96
AIX Installation
You have two options:

If you select Use existing license keys, the License Key Check
screen displays.
If you select, Specify the location of license keys, the License Key
Location screen displays.
17. After you specify license keys and they are validated, click Next.
97
Once the installation wizard has verified that RecoverNow has been
shut down, the Shutdown Verification Complete screen displays.
19. Click Next

If you have efixes, the Fix Removal Required screen displays. You must
manually remove the efixes, the wizard does not perform this task.
Refer to page 92 for information on how to remove efixes.
98
AIX Installation
20. After you remove the efixes, click Next

The Verify Fix Removal screen displays.
When the fix verification successfully completes the Fix Removal

Verification Complete screen displays.
99
21. Click Next.

The Installing screen displays.
When the installation completes, the Install Portal Application screen

displays.
100
AIX Installation
22. Click Next.

When the Portal Application installation completes the Install Vision
Solutions Portal screen displays.
23. Click Next.

The Install Vision Solutions Portal screen displays.
101
When the Vision Solutions Portal installation completes, the Summary

screen displays.
After you have upgraded the RecoverNow Portal Application, you can log
into VSP. See Logging in to Vision Solutions Portal on page 127.
102
Use smit to Install RecoverNow

To use smitty to install RecoverNow:
Step 1. Request a License on page 103
Step 2: Install RecoverNow on page 103
Step 1. Request a License

1. Identify the Machine ID/Host ID number. From a UNIX prompt, issue
the command:
a. Mount the CD
b. Run bin/AIX/machine_id.bin
2. Send the following information in email to Customer Accounting at:
productrequest@visionsolutions.com
The email should contain the following:
Company Name
Product(s) for which you are requesting a license
Machine ID
Operating system
Operating system version
Node name (hostname)
A license file is generated, attached to an email, and sent back to you.
3. Save the license file in a location of your choice.
Step 2: Install RecoverNow

1. Download the RecoverNow installation program from the Vision
Extranet at Customer/Partner Login at www.visionsolutions.com.
2. Extract and copy the appropriate directory and files from your PC to the
AIX server. Depending upon your OS and bit-level, choose one of the
following directory paths for a 32 or 64 bit kernel:
esFiles/52/32/
esFiles/52/64/
esFiles/53/32/
esFiles/53/64/
esFiles/61/64/
esFiles /71/64
103
3. Log into the AIX system as root.

4. Navigate to the RecoverNow software for AIX.
5. Navigate to the AIX version: 52
53
61
For example cd 53
6. Choose the directory path for a 32 or 64 bit kernel:
esFiles/52/32/
esFiles/52/64/
esFiles/53/32/
esFiles/53/64/
esFiles/61/64/
esFiles /71/64
7. Enter the following command:
smit
The System Management screen displays.
104
8. Select Software Installation and Maintenance.
9. Select Install and Update Software.
105
10. Select Install Software.
11. Select the current directory as the INPUT device/directory and enter a
dot (.).
12. Press OK.

The Software to Install screen displays.
106
13. Click List to the right of SOFTWARE to install.
14. Click to select the components you want to install.

15. After you have made your selections, press OK to initiate smit and
install the selected components.
107
After you have installed RecoverNow, the RecoverNow Portal Application,

and the Vision Solitons Portal (VSP) you can log into VSP. See Logging in
to Vision Solutions Portal on page 127.
User Roles
The installation process creates the scrt group in /etc/group, identifying the
category of users allowed to access the portal application.
IMPORTANT
The root user is always allowed access to the portal application.
Log Files
There are two types of log files:
install.logStores all user inputs in the directory from which the

installer is run if you have copied your inputs to an AIX machine. This
log file also stores details of the installation process.
verifier.logStores the details of verification information on the

remote or local machine in the directory from which the installer is run.
Run the validate_license command

After you apply the license, run the validate license command:
/usr/scrt/bin/validate_license
A license validation message appears:

License validated
or License validation failed
If you receive the License validation failed message ensure that the
information specified in license.inf is correct and that the output of the
hostname command matches the hostname in license.inf. If the problem
persists, email or contact Customer Support. Refer to the readme.txt file for
contact information.
License Expiration
When the license expires for RecoverNow, the application on the
production server is not affected. However, data replication to the recovery
server will be stopped, and you will no longer be able to use the Continuous
Data Protection functionality. You can check the license file for the data
replication component for information about the expiration of the license.
The file is named: /usr/scrt/run/node_license.properties.
108
NOTE
RecoverNow verifies the contents of the license file. Do not alter

this file.
109
Uninstall RecoverNow on AIX

To uninstall Double-Take RecoverNow:
IMPORTANT
You must shutdown RecoverNow and unload the drivers before

you run the uninstall program. Use the following command:
rtstop -FC <Context ID>
1. Download the Double-Take RecoverNow installation program from

the:
www.visionsolutions.com.
Double-Take RecoverNow installation CD.
a. Click Uninstall Double-Take RecoverNow on the Double-Take
RecoverNow Installation splash screen or click uninstall.exe.
a. Run uninstall.bin.
3. Run the Uninstall wizard to remove Double-Take RecoverNow from
your nodes.
The Double-Take RecoverNow Uninstall Wizard runs and displays
the Double-Take RecoverNow Welcome screen.
4. Click Next.
110
want to uninstall RecoverNow.
6. Click Next.
software uninstalled on this node.
8. Click Next.
111
The Node Login screen re-displays.

9. Log in to the node that you added, and press Next.
The Specify Nodes screen re-displays.
10. Click Next.
The Retrieving Installation Information screen briefly displays. Then
the Uninstall Options screen displays.
11. Select the components you want to uninstall from each node and click
Next.
112
Once the uninstall wizard has verified that RecoverNow has been
shutdown the Shutdown Verification Complete screen displays.
13. Click Next

If you have efixes, the Fix Removal Required screen displays. You must
manually remove the efixes, the wizard does not perform this task.
113
14. After you remove the efixes, click Next

The Verify Fix Removal screen displays.
15. When the fix verification successfully completes, the Fix Removal
Verification Complete screen displays.
16. Click Next to uninstall RecoverNow and VSP.
The Uninstall RecoverNow screen displays.
114
When the uninstall process completes, the Uninstall Complete screen

displays.
115
Use smit to Uninstall RecoverNow

To uninstall RecoverNow:
IMPORTANT
You must shutdown RecoverNow and unload the drivers before

you run the uninstall program. Use the following command:
1. Enter smit
The System Management screen displays.
116
Use smit to Uninstall RecoverNow
2. Select Software Installation and Maintenance.
3. Select Software Maintenance and Utilities.
117
4. Select Remove Installed Software
5. Select List to the right of SOFTWARE name.
6. Select the software you wish to remove, and press OK.
Install the Vision Solutions Portal and Double-Take RecoverNow

Portal Application on Windows
The Vision Solutions Portal and Double-Take RecoverNow Portal
Application Installation Wizard for Windows provides a way to install,
upgrade, or reinstall the Vision Solutions Portal and the RecoverNow portal
application on your local Windows machine.
118
Install the Vision Solutions Portal and Double-Take RecoverNow Portal Application on Windows
NOTE
The RecoverNow software can only be installed on an AIX

machine. Only VSP and the portal application can be installed on
a Windows machine.
Download the RecoverNow installation program from:
www.visionsolutions.com.
Double-Take RecoverNow installation CD.
7. Run 4.0.00.01.RECOVERNOW.WINDOWS.EXE to install the portal
application and the Vision Solutions Portal to your local Windows
machine.
The Installation Wizard runs and displays the Vision Solutions Portal
Welcome screen.
8. Click Next
The Terms And Conditions screen displays. Read and accept the terms
of the License Agreement.
119
9. Click Next.
The Checking Vision Solutions Portal screen displays.
10. Once the VSP and portal application status is verified, the Installation
120
Install the Vision Solutions Portal and Double-Take RecoverNow Portal Application on Windows
11. You can decide to start the portal server when Windows starts and after
the installation completes.
12. Click Next.
NOTE
If the portal application is currently active, the Shut Down Vision

Solutions Portal screen displays. The installation wizard will
enable you to shut down VSP.
13. Click Next.
121
The Installing screen displays, showing the status of the installation.
14. When the installation completes, the Summary screen displays.
15. After you have installed the RecoverNow portal application, you can
log into VSP. You can select the highlighted machine-name address to
launch VSP and log into VSP. See Logging in to Vision Solutions
Portal on page 127.
122
Uninstall the Vision Solutions Portal and Double-Take RecoverNow Portal Application Uninstallation Wizard on Windows
Uninstall the Vision Solutions Portal and Double-Take

RecoverNow Portal Application Uninstallation Wizard on
Windows
Use this wizard to uninstall product portal applications or Vision Solutions
Portal and all product portal applications from your local Windows
machine. Vision Solutions Portal will be shut down and portal users will be
automatically logged out during the uninstall.
1. Navigate to C:\Program Files (x86)\VisionSolutions\http
2. Click Uninstall.exe.
The Uninstall Wizard for Vision Solution Portal runs and displays the
Vision Solutions Portal Welcome screen.
3. Click Next.
123
The Checking Vision Solutions Portal screen displays.
4. Once the VSP and portal application status is verified, the Options
screen displays.
5. Decide which components you wish to uninstall and click Next.

The Shut Down Vision Solutions Portal screen displays.
124
Uninstall the Vision Solutions Portal and Double-Take RecoverNow Portal Application Uninstallation Wizard on Windows
6. When the shut down of VSP completes, the Ready to Uninstall screen
displays.
7. Click Next.
The Preparing to Uninstall screen displays.
125
8. When the uninstall processing completes, the Summary screen displays.
9. Click Done, to complete the Vision Solutions Portal uninstallation.
Post-Installation Tasks
These sections contain the post-installation tasks:
126
Logging in to Vision Solutions Portal on page 127
Increase Your AIX File Limit on page 128
Enable Automatic Startup on page 128
Enable Automatic Startup on page 128
Logging in to Vision Solutions Portal

You must have a valid user ID and password for the node on which the
Vision Solutions Portal (VSP) server is installed. Do the following:
1. Enter the following URL in your web browser:
http://server:port
The server is the IP address or host name for the node on which the
VSP server is installed and active. The default port number is 8410. For
example, if the VSP server was installed on node vsp-53, you would
copy the following url into the address field in your browser window:
http://vsp-53:8410
2. The portal appears showing the Log In page. Log in using your user ID
and password. Depending on the platform, the user ID and password
may be case-sensitive.
NOTE
If you have a problem logging into VSP refer to the

Troubleshooting chapter in Getting Started with Vision Solutions
Portal (2.0.00.07_VSP_Getting_Started.pdf) packaged
with RecoverNow for AIX.
After you have logged in, the portal opens to the Home page. A default
portal connection exists for the node on which you logged in.
127
For detailed information on working with VSP, refer to Getting Started with
Vision Solutions Portal (2.0.00.07_VSP_Getting_Started.pdf)
packaged with RecoverNow for AIX.
Increase Your AIX File Limit

If you are using 4,000 or more Log File Containers (LFCs), you need to
increase your AIX file limit of 2,000. Make the file limit equal to the
number of LFCs in the /etc/security/limits file.
If the number of LFCs in a configuration exceeds 5,000, you may need to
increase the default data value in /etc/security/limits to 524288, which can
support up to 20,000 LFCs. The default value is 262144.
To determine an exact value:
1. Temporarily set data to -1 in /etc/security/limits.
2. Restart scrt_lca or scrt_aba.
3. Use the following command to monitor the maximum SIZE reached
over time:
while true;do ps v <scrt_lca or scrt_aba PID>|awk
'NR>1' >>/tmp/agent_size.out;sleep 60;done&
If this value is inadequate, one of the following will assert or shutdown

without reason:
scsetup -MC <Context ID>
scrt_lca
scrt_aba
Enable Automatic Startup

After you install RecoverNow on your Production, Recovery, and
Replicated (if available) Servers, you need to enable automatic startup of
RecoverNow so that it is started as part of the boot process. RecoverNow
needs to be started before the application starts so that it can protect the
application data.
1. Using the cat command, determine the entry for the protected
application in the /etc/inittab file.
2. Note the entry that precedes it. The identifier for this entry is an
argument in the mkitab command. The mkitab command inserts the
RecoverNow boot command after the identifier into the /etc/inittab file.
This causes RecoverNow to start automatically before the protected
application during a reboot.
128
3. Execute the following command from a shell as root:

mkitab -i <identifier> scrt:2:wait:"/usr/scrt/bin/sccfgd_boot
>/var/log/EchoStream/sccfgd_boot.log 2>&1"
This command inserts the sccfgd_boot command after the identifier

in the /etc/inittab file.
Related Topics
Refer to Configuring Replication Groups on page 131.
Refer to Starting and Stopping on page 177.
129
130
Configuring Replication Groups
After you install RecoverNow, VSP and the portal application, you can
use the RecoverNow Replication Group Wizard or the Command line to
configure new replication groups and change, rename, and delete existing
ones.
Using VSP to Configure Replication Groups on page 132
Create and Configure a New Replication Group on page 132
Configuration Initialization Progress (New Configuration) on

page 150
Change a Replication Group Configuration on page 151
Configuration Initialization Progress (Change Configuration) on

page 161
Rename a Replication Group on page 6-164
Delete a Replication Group on page 6-166
Replication Group Configuration Window on page 169
Using the Command line to Perform Configuration Tasks on

page 169
131
Chapter 6: Configuring Replication Groups
Using VSP to Configure Replication Groups

You can use the Replication Group Configuration Wizard to:
Create and Configure a New Replication Group on page 132
Change a Replication Group Configuration on page 151
Rename a Replication Group on page 164
Delete a Replication Group on page 166
Create and Configure a New Replication Group

To create and configure a new replication group, do the following:
1. Navigate to the Replication Group portlet.
2. Click Configuration
The Replication Group Configuration window displays.
132
For detailed information refer to the RecoverNow online help.
3. Click New.
This starts the Replication Group Configuration wizard and the New
Replication Group Servers panel displays.
The New Replication Group Servers panel contains the following

fields:
133
Field
Description
ServersSection for specifying the host name or IP address for the servers in
this replication group.
Production
Select from the list of portal connections that are associated

with the instance. The host name from the portal connection
is used for the server. This ensures the newly configured
replication group will appear in the instance.
Recovery
Options are:
Select the portal connection
Specify, and enter the host name or IP address for the
server in the recovery role.
Recovery host
name or IP
address
Specify a host name or IP address for the server in the

recovery role.
Replicated
(optional)
Options are:
Replicated host
name or IP
address
Specify a host name or IP address for the server in the

recovery role.
Failover server
If you need to move production due to an unplanned outage

or for maintenance, select the server where you would move
production.
None
Select the portal connection
Specify, and enter the host name or IP address for the
server in the replicated role.
4. Click Next.
134
The New Replication Group Servers panel displays. Use this panel to
log into the failover server, specified in the previous panel.If you have
not already logged in to all of the nodes, this panel displays.
5. Specify the username and password and click Log In. Log in to each
server to retrieve information.
NOTE
The user must be either root or a user that is in the scrt group.
Passwords cannot be blank
6. Click Log In.
The New Replication Group Servers panel displays.
135
The New Replication Group login panel contains the following fields:
Field
Description
Log in Status
Displays the list of servers in the replication group and their

login status.
Server
Displays the name of the server being logged in to. This is

the host name from the portal connection if a portal
connection was used to specify the server.
User
Specify the user ID to log in with. This defaults to the user

ID from the portal connection if a portal connection was
used to specify the server.
Password
Specify a password for the specified user ID.
Log In
Click this button to log in to the specified server.
Status
Displays the login status for the specified server.
7. Click Next.
136
The New Replication Group Names panel displays.
The New Replication Group Names panel contains the following fields:
Field
Description
Replication
group name
Specify the name of the replication group. Replication

group names:
Can be 256 characters in length and can contain any
alphanumeric character.
Must be unique across the servers specified for the
replication group. This is the same as for context IDs.
Must start with an alphanumeric character. Hyphens ( - )
and underscores ( _ ) are allowed. Spaces or other special
characters are not allowed.
Primary context
ID
Defaults to the next available context ID on the specified

servers. Context IDs must be unique across the servers
specified for the replication group.Valid values are 1 to 999.
Failover context
ID
Defaults to the primary context ID number plus 16. If a

value greater than 983 is specified for the Primary context
ID, no default is provided.Valid values are 1 to 999.
NOTE
Only displays when the Failover server field on

the Servers panel has a value other than Do Not
Failover.
8. Click Next.
137
The New Replication Group Logical Volumes panel displays. Initially, a

message indicates that RecoverNow is retrieving logical volume
information from the production server. To proceed, you must add at
least one production server logical volume you want protected, to this
replication group
9. Click Add.
10. The Add Logical Volumes dialog displays. For detailed information
refer to the RecoverNow online help.
138
Select one or more logical volumes to protect on the production

server.
Select the volume group on the recovery or replicated servers where
the replicated data will be located.
Click OK.
The New Replication Group Logical Volumes panel displays with
the selected logical volumes you want protected.
11. Click OK.

The New Replication Group Logical Volumes panel displays. It
contains the following columns:
Column
Description
Logical Volume
Displays the logical volumes configured for this replication

group.
Volume Group -
Displays the name of the volume group on the production

server for the logical volume being protected.
Production
Volume Group Recovery
Displays the name of the volume group on the recovery

server where the replica for the logical volume being
protected is located. When there is a replicated server, the
Volume Group - Recovery and Volume Group - Replicated
are set to the Default Volume Group and cannot be changed.
139
Column
Description
Volume Group Replicated
Displays the name of the volume group on the replicated

server where the replica for the logical volume being
protected is located. Displayed only if the replication group
has a replicated server configured. When there is a
replicated server, the Volume Group - Recovery and Volume
Group - Replicated are set to the Default Volume Group and
cannot be changed.
Size (GB)
Displays the size in gigabytes (GB) of the logical volume.
Type
Displays the type of logical volume. For example, raw, jfs,

jfs2, and jfs2log.
Mount Point
Displays the mount point for the logical volume.Typically

around 20 characters but can be as long as 2048. If length
exceeds 76, the text is truncated with an ellipsis in the
middle
FS Log
Displays the FS log for the logical volume. If length

exceeds 76, the text is truncated with an ellipsis in the
middle.
12. Click Next on the New Replication Group Logical Volumes panel.
The New Replication Group Replication IP Addresses panel displays.
Use this panel to specify IP labels or addresses that will be used
specifically for replication. By default, replication uses the IP addresses
of the servers. There are two options:
If you select, Use server IP addresses for replication, the New
Replication Group Replication IP Addresses panel displays in the
following format..
140
If you select, Use server IP addresses for replication, the Replication

Group Replication IP Addresses panel displays in the following
format.
The Replication Group Replication IP Addresses panel contains the

following fields:
NOTE
Production and Recovery server sections are always displayed.

The Replicated server section is only displayed if a replicated
server is configured.
Field
Description
Replication IP
Addresses
Indicates if the user wants to use the server IP address for

replication or specify alternates. Possible values:
Use server IP addresses for replication
Use specified IP addresses for replication
Production ServerSection heading for the production server.

Production
Server - Host
name
Identifies the name of the server in the production role.
Production
Server - IP
address
Identifies the resolved IP address from the host name.

Possible values are any valid IPv4 and IPv6 addresses.
Recovery ServerSection heading for the recovery server.
141
Field
Description
Recovery Server
- Host name
Identifies the name of the server in the recovery role.
Recovery Server
- IP address
Identifies the resolved IP address from the host name.

Possible values are any valid IPv4 and IPv6 addresses.
13. Click Next.

The New Replication Group Containers panel displays. Use this panel
to configure how data is moved between servers. Containers are used by
internal processes and replication to move the changed data between
servers. A larger total container size provides a larger rollback window.
Smaller sized containers may replicate more frequently. Specify the
quantity and size of the containers, the default volume group where the
containers are located, and the number of logical volumes to use to
balance IO and improve replication performance.
The Replication Group Containers panel contains the following fields:
142
Field
Description
Server
Identifies the name of the server in the production, recovery,

and replicated roles
Number of
containers
Specify the number of containers to use on each server.

Default values are 64 for the production server, and 128 for
the recovery and replicated servers.
Number of
containers on
replicated
This value is the same as the recovery server and cannot be

changed. Updating the recovery server value also updates
this value. Only displayed if a replicated server is
configured.
Size of each
container
Specify the size of each container in MB. Since the size

must match on all configured servers, the recovery and
replicated values are display only. Possible values are 2, 4,
8, 16, 32, 64, 128, 256, and 512. Default is 16.
Total size
Displays the total space required for the containers on each

server.
Default volume
group
Specify the default volume group for each server.
Logical volumes
Specify the number of logical volumes to create when

creating containers. By default, the logical volumes are
created on the default volume group. However if you
choose, Use alternate volume groups or physical volumes
for replication containers, then the logical volumes are
created on the alternate volume groups or physical volumes.
Default values are 2 for the production server, and 3 for the
recovery and replicated servers.
Use alternate
volume groups
or physical
volumes for
replication
containers
Indicates if the you want to specify volume groups and

physical volumes that will be used for the containers used
for replication. If checked, the Replication Containers panel
is displayed. If not checked, the Replication Containers
panel is skipped. This box, is unchecked, by default.
14. Click Next. There are two possible actions:

If you leave the 'Use alternate volume groups or physical volumes
for replication containers' checkbox unchecked on the Containers
panel, the New Replication Group Container Options panel displays.
See page 145.
If you check the Use alternate volume groups or physical volumes
for replication containers checkbox, the New Replication Group
Replication Containers panel displays. If the checkbox is not
checked, this panel is skipped.
143
NOTE
The Production and Recovery server sections are always

displayed. The Replicated server section is only displayed if a
replicated server is configured.
Use the New Replication Group Replication Containers panel to select the
volume groups and physical volumes where you want to locate the
containers used specifically for replication
The New Replication Group Replication Containers panel contains the

following fields:
Field
Description
Production ServerSection heading for each server (production, recovery, and

replicated if configured)
144
Server
Displays the name of the server.
Total container
size
Displays the total space (in MB) required for the containers
on the server.
Volume Group
Displays the list of volume groups on the specified server.
Add
Use this button to adds the volume group to the list and
defaults the physical volume to Any.
Volume Group
Displays the name of a selected volume group.
Field
Description
Physical Volume
Displays a list of physical volumes (disks) on the specified

volume group.
Remove
Removes the volume group from the list. This action is

available after you add a volume group.
15. Click Next.

The New Replication Group Container Options panel displays. Use this
panel to specify if you want to use compression during replication. Specify
if you also want to send partially filled containers after a specified amount
of time. Otherwise, containers are sent when they become full.
The New Replication Group Container Options panel contains the

following fields:
145
Field
Description
Use compression
By default, the Use compression during replication

checkbox contains a checkmark. To disable this option,
remove the checkmark from the checkbox. In the
Replication Configuration Summary window, possible
values are Yes (enabled) or No (disabled).
LFCs are compressed before being sent to the recovery
server, where they are then uncompressed. When you
compress LFCs a smaller amount of data is sent across the
network. This activity requires additional CPU bandwidth
to compress and uncompress the LFC on each server.
Send partially
filled containers
automatically
By default, the Send partially filled containers

automatically checkbox contains a checkmark. To disable
this option remove the checkmark from the checkbox. In the
Replication Configuration Summary window, possible
values are Yes (enabled) or No (disabled).
The Send partially filled containers automatically option
enables you to control the frequency of shipping containers
(LFCs) to the recovery server during low I/O periods.
The container is examined when the frequency to check
value is reached. If at that time the minimum percent filled
value has not been reached, the container will not be sent.
When the container reaches the minimum percent filled
value, containers will not be immediately sent. There is a 5
second delay before the containers are sent. The 5 second
delay is provided to account for the possibility that the
container could become completely full. When the
container is sent, the frequency to check value is reset and
the entire process starts again.
Note you can also use the command line to Send partial
containers automatically. Refer to Send partially filled
containers automatically on page 170.
146
Field
Description
Frequency to
check
Indicates how often to check for partially filled containers

in seconds.
Keep in mind the following:
Only enabled if the Send partially filled containers
automatically checkbox is checked.
Default value is 300 seconds
Valid range is 1 to 86,400, only numeric values are
accepted. You must specify a value.
Minimum
percent filled
Indicates how full the container must be before it can be

sent.
Keep in mind the following:
Only enabled if the Send partially filled containers
automatically checkbox is checked.
Default value is 50%.
Valid range is 0 to 99, only numeric values are accepted.
You must specify a value.
16. Click Next.

The New Replication Group Snapshot Buffer panel displays. When a
snapshot is created, the snapshot buffer is used to hold the changes between
the location in the rollback window where the snapshot was created and the
current time. As changes are replicated, the snapshot buffer fills up.
The New Replication Group Snapshot Buffer panel contains the following
fields:
147
Field
Description
Snapshot
Buffers - Size
Indicates how much space to reserve for the snapshot buffer.

The value is a percent of the size of the logical volumes that
have been selected to protect. Valid values are integers from
1 to 100. Default is 10.
Warning
threshold
Indicates how full the snapshot buffer must be before you

are warned that it is filling up.Valid values are integers from
1 to 100. Default is 75.
Location
Indicates that the snapshot buffers are located on default

volume group of the production, recovery, and replicated
servers.
Production
Production server information. Only displayed if a failover

server is configured.
Replicated
Replicated server information. Only displayed if a

replicated server is configured.
17. Click Next.

The New Replication Group Tivoli Storage Manager panel displays. The
Tivoli Storage Manager (TSM) can archive containers which allows you to
rollback or take snapshots farther back in time. Full backups of the server
where the TSM client is running can also be performed.
The New Replication Group Tivoli Storage Manager panel contains the
following fields:
148
Field
Description
Enable Tivoli
Storage Manager
(TSM)
Indicates if Tivoli Storage Manager is enabled for this

replication group. Checkbox is unchecked by default.
TSM client Server
Displays the name of the server where the TSM client is

running. This is always the recovery server. Enabled only
when Enable Tivoli Storage Manager (TSM) is checked.
TSM client User ID
Specify the user ID for TSM to use to log into the server
where the TSM client is running. Enabled only when Enable
Tivoli Storage Manager (TSM) is checked.
TSM client Password
Specify the password for TSM to use to log into the server
where the TSM client is running. Enabled only when Enable
TSM client Options file
Specify the location of the TSM options file on the TSM

client server. The default location is
/usr/tivoli/tsm/client/ba/bin/dsm.opt. You can
specify any valid path and file name. Enabled only when Enable

TSM client Domain
Specify the domain for TSM to use. Enabled only when

Enable Tivoli Storage Manager (TSM) is checked.
TSM server
Specify the host name or IP address of the server where the

TSM server software is running. Valid values are any valid
IPv4 and IPv6 addresses or host name. Enabled only when
Enable Tivoli Storage Manager (TSM) is checked.
18. Click Next.
149
The New Replication Group Summary panel is a summary of this

replication group's configuration. To save and initialize this configuration
on the specified servers, click Finish. To cancel, click Cancel.
The content of this panel is the same as the Configuration Summary section
in the Replication Group Configuration window. Refer to the RecoverNow
online help for additional information for the Replication Group
Configuration window. Refer to Replication Group Configuration
Window on page 169.
IMPORTANT
When you initially configure a replication group the Finish button

will be enabled, so that you can save the configuration to the
servers and initialize the configuration.
Configuration Initialization Progress (New Configuration)

When a configuration is created, the Configuration Summary area lists the
steps required to initialize the configuration and shows which steps have
been completed or are in progress. When the initialization is successfully
completed, the summary area displays the normal Configuration Summary
content. Configuration initialization cannot be stopped once it is started.
When the new configuration is saved and validated, you can view the
progress of the configuration initialization in the Configuration
Initialization Progress section in the Replication Group Configuration
window. As each step is successfully completed, a checkmark appears next
to the step. When you create a new configuration, RecoverNow runs
150
commands for each step. The table below describes steps and commands
that are run when you create a new configuration.
Order
Step
Command
1.
Save configuration.
This was done when you clicked

Finish in the New Replication Group
wizard on the Summary panel.
2.
Copy configuration to S (S is
production server, recovery
server or replicated
server).You will see a row
for each server in the
configuration.
sccfgd_putcfg <Context ID>
3.
Create containers and logical

volumes on S (S is
server or replicated server).
You will see a row for each
server in the configuration.
4.
Create failover context on S

(S is production server,
recovery server or replicated
server). You will see a row
for each server in the
configuration.
rtdr -C <Primary Context

ID> -F <Failover Context
ID> -s <Failover HostName>
setup
Notes:
These steps may be run on one or more servers.
step 4 only runs if a failover server was specified in the
configuration.
Related topics
Configuration Initialization Progress (Change Configuration) on

page 161.
Change a Replication Group Configuration

Before you change the configuration for a replication group, keep in mind
the following:
Configuration changes cannot be saved until the replication group is

stopped. You will be allowed to continue through the Change
Replication Group wizard and make changes. However, when you click
Finish on the Summary panel, a message will appear that instructs to
stop the replication group. To stop the replication group navigate to the
151
Replication Groups portlet in the Vision Solutions Portal window and

stop the replication group.
You cannot change the replication group name or context ID with the
Change Replication Group wizard.
A replication group cannot be changed if the production, recovery, and

replicated servers are not at version v4.0.01.00.
To change a replication group, do the following:

2. Click Configuration.
The Replication Group Configuration window displays. For detailed
information refer to the RecoverNow online help.
3. Click Change from the Actions dropdown. There are two possible
results:
a. The Change Replication Group dialog displays with a warning:
152
Replication group cannot be changed because it is either failed over

over or the partition has been migrated using Live Partition
Mobility.
b. The Change Replication Group Servers panel displays with a

warning.
Replication group is currently active. Configuration changes
cannot be saved until the replication group is stopped.
Configuration changes cannot be saved until the replication group is
stopped. However, when you click Finish on the Summary panel, a
message will appear that instructs you to stop the replication group.
To stop the replication group, navigate to the Replication Groups
portlet in the Vision Solutions Portal window and stop the
replication group.
4. Click Next.
153
The Change Replication Group Servers panel displays. Use this panel to
log into the failover server, specified in the previous panel. This panel
displays if you are not logged in.
5. Specify the username and password and click Log In. Log in to each
server to retrieve information When you run commands, context IDs are
used to identify the replication group. The context IDs specified have
been defaulted to unique IDs on the servers in this replication group.
6. Click Next.
The Change Replication Group Servers panel displays.
7. Click Next.
154
The Change Replication Group Names panel displays.
8. Click Next.
The Change Replication Group Logical Volumes panel displays.
Click Add to add a logical volume. For details refer to step 10 on
page 138.
Select the logical volume you wish to change or remove for this
replication group and click Next.
155
Use the Change Logical Volumes dialog, shown below, to choose a

different volume group for the replica logical volumes, on the recovery
or replicated servers. By default, the default volume group specified is
used for this purpose. For detailed information refer to the RecoverNow
online help.
Use the Remove Logical Volumes dialog, shown below, to remove the
selected logical volumes from the replication group. These logical
volumes will no longer be protected. For detailed information refer to
the RecoverNow online help.
9. Click OK on either the Change or Remove Logical Volumes dialogs.

The Change Replication Group Logical Volumes panel displays.
10. Click Next.
The Change Replication Group Replication IP Addresses panel
displays. By default, replication uses the IP addresses of the servers.
You can specify IP labels or addresses that will be used specifically for
replication. There are two options:
156
If you select, Use server IP addresses for replication, the following

panel appears.
If you select, Use specified IP addresses for replication, the

following panel appears.
11. Click Next.

The Change Replication Group Containers panel displays. Containers
are used by internal processes and replication to move changed data
between servers.
157
IMPORTANT
If you make configuration changes on the Container panel you

could lose CDP. For example, if you change the Size of each
container field, you will lose CDP. This will be reflected in the
Change Replication Group Summary panel displays, as shown
page 161.
12. Click Next.

The Change Replication Group Container Options panel displays.
Specify if you want to use compression during replication. Specify if
you also want to send partially filled containers after a specified amount
of time. Otherwise, containers are sent when they become full.
158
13. Click Next.

The Change Replication Group Snapshot Buffer panel displays. The
Tivoli Storage Manager (TSM) can archive containers which allows
you to rollback or take snapshots farther back in time. Full backups of
the server where the TSM client is running can also be performed.
14. Click Next.

The Change Replication Group Tivoli Storage Manager panel displays.
Tivoli Storage Manager (TSM) can archive containers which allows
159
you to rollback or take snapshots farther back in time. Full backups of

the server where the TSM client is running can also be performed.
15. Click Next.

The Change Replication Group Summary panel displays and shows a
summary of this replication group's configuration. If you have made
configuration changes on the Containers panel (see page 158), proceed
to Configuration changes will reset the rollback window (CDP is lost)
on page 161.
160
IMPORTANT
When you decide to make configuration changes to the replication

group, they can only be saved if the replication group is stopped.
Use the Replication Group portlet on the Replication page to stop
the replication group.
16. To save and initialize this configuration on the specified servers, click
Finish. To cancel, click Cancel. To view a summary of the changes you
made, refer to Replication Group Configuration Window on
page 169. The content of this panel is the same as the Configuration
Summary section in the Replication Group Configuration window.
Refer to the RecoverNow online help for additional information for the
Replication Group Configuration window. Refer to Replication Group
Configuration Window on page 169.
Configuration changes will reset the rollback window (CDP is lost)
If you have made configuration changes on the Containers panel (see
page 158), the Change Replication Group Summary panel displays with the
warning: Configuration changes will reset the rollback window. This means
you will lose CDP.
Configuration Initialization Progress (Change Configuration)

When a configuration has been changed, the Configuration Summary area
lists the steps required to initialize the configuration and shows which steps
have been completed or are in progress. When the initialization is
successfully completed, the summary area displays the normal
161
Configuration Summary content. Configuration initialization cannot be

stopped once it is started.
When the configuration is saved and validated, you can view the progress of
the configuration initialization in the Configuration Initialization Progress
section in the Replication Group Configuration window. As each step is
successfully completed, a checkmark appears next to the step. When you
change an existing configuration, RecoverNow runs commands for each
step.
The step and command sequence can change depending upon the
configuration changes that you make. The table below describes steps and
commands that are run when you change a configuration:
162
Order
Step
Command
1.
Save configuration.
This was done when you clicked Finish

in the Change Replication Group
wizard on the Summary panel.
2.
Copy configuration to S (S
is production server,
recovery server or
replicated server). You will
see a row for each server in
the configuration.
sccfgd_putcfg -C <Context
ID>
3.
Delete statemap bitmap

logical volumes.
scsetup -R -C <Context ID>

-t smbitmap
4.
Delete replication
container logical volumes
on S (S is production
server, recovery server or
the configuration.

-t lfc
or
-t lfc -t smfc
5.
Delete log record file

container and snapshot
logical volumes on S (S is
recovery server or
the configuration.

-t lrfc -t pool
6.
Delete replication
container and snapshot
recovery server or
the configuration.

-t lfc -t pool
or
-t lfc -t smfc -t lrfc -t
pool
Order
Step
Command
7.
Delete all containers and

8.
Delete snapshot logical

volumes on S (S is recovery

-t pool
9.
Delete snapshot container

recovery server or
the configuration.

-t jcontainer
10.
Delete replication
container, snapshot, and
log record file container
logical volumes.

-t lfc -t lrfc -t pool
11.
Create containers and

scsetup -M -C <Context ID>
12.
Set all logical volumes as

replicated.
scconfig -W -C <Context ID>
13.
Set new logical volume X
scsetup -M -C <Context ID>

-L
as requiring replication (X
is a device such as /dev/
rLV_p520-95_0001).
14.
Reset Send process (LCA)
scrt_lca -fr -C <Context

ID>
15.
Reset Apply process

(ABA).
scrt_aba -fr -C <Context

ID>
16.
Load drivers on S (S is
production server or
recovery server). You will
the configuration.
scconfig -l -C <Context ID>
163
Order
Step
Command
17.
Delete logical volume

device special files on S (S
is production server,
recovery server or
the configuration.
scsetup -X -C <Context ID>
18.
Create failover context on

S (S is production server,
recovery server or
the configuration.
rtdr -C <Primary Context

ID> -F <Failover Context
ID> -s <Failover HostName>
setup
19.
Delete failover context on

S (S is production server,
recovery server or
the configuration.
sccfgd_delcfg <Failover
Context ID>
NOTE
The steps that are run depend on what is changed in the

configuration.
Related Topic:
Configuration Initialization Progress (New Configuration) on page 150
Rename a Replication Group

To rename a replication group, do the following:
164

.
3. Click Rename from the Actions dropdown. There are two possible
results:
a. The Rename Replication Group dialog displays with a warning:
Replication group cannot be renamed because it is either active and
must be stopped, failed over, or the partition has been migrated
using live partition mobility. Use the Replication Group portlet on
the Replication page to stop the replication group.
165
b. The Rename Replication Group displays:
Specify the new replication group name, and press OK. To view a
summary of the changes you made, refer to Replication Group
Configuration Window on page 169.
Delete a Replication Group

To delete a replication group, do the following:
166
3. Click Delete from the Actions dropdown. There are two possible
results:
a. The Delete Replication Group dialog displays with a warning:
The Replication group cannot be deleted because it is either active
and must be stopped, failed over, or the partition has been migrated
using live partition mobility Use the Replication Group portlet on
the Replication page to stop the replication group.
b. The Delete Replication Group displays with a warning:
167
The Logical volumes configured in this replication group will no

longer be protected.
Click OK to delete the replication group. To view a summary of the

changes you made, refer to Replication Group Configuration
Window on page 169.
Using Earlier Versions of RecoverNow to Configure Replication

Groups
If you configured replication groups with a previous version of
RecoverNow, if the context ID is 1 then the replication group appears in
VSP as RG-1.
168
Using the Command line to Perform Configuration Tasks
Replication Group Configuration Window

The Replication Group Configuration window lists all the replication
groups in the instance. The Replication Group Configuration window
Summary section displays a summary of the configuration for the selected
replication group. See the RecoverNow online help for information about
each field.

You can use the command line to perform the following configuration tasks:
Initialize a Context on page 169
Send partially filled containers automatically on page 170
Verify your Configuration using the sccfgchk Command on page 171
Initialize a Context
Use the command line to initialize a context on the production, recovery,
and replication servers. Execute the following command on each server.
scsetup -C <Context ID> -M
169
Send partially filled containers automatically

The Send partially filled containers automatically option can be managed
via the command line using the scconfig command. Refer to scconfig on
page 274. You can also use the Replication Group Configuration Wizard.
Refer to Using VSP to Configure Replication Groups on page 132.
Specifically, refer to the New Replication Group Container Options panel,
and the field, Send partially filled containers automatically on page 146.
The Send partially filled containers automatically option enables you to
control the frequency of shipping containers (LFCs) to the recovery server
during low I/O periods.
The container is examined when the frequency to check value is reached. If
at that time the minimum percent filled value has not been reached, the
container will not be sent. When the container reaches the minimum percent
filled value, containers will not be immediately sent. There is a 5 second
delay before the containers are sent. The 5 second delay is provided to
account for the possibility that the container could become completely full.
When the container is sent, the frequency to check value is reset and the
entire process starts again.
When changing either of the scconfig options (-a, -b), the command returns
output displaying the values for both options. If the frequency to check
value is changed to zero, the command output will also display Send
Partial Container Automatically is not active.
Configurations created prior to RecoverNow Version 4.0

When displaying (using the -c option), the Send partially filled containers
automatically for configurations created prior to RecoverNow v4.0, the
scconfig -C <ContextID> -c command displays:
Send Partial Container Automatically is not active.
Values for Context ID = <ContextID>:
Frequency to check = Not Specified
Percent filled = Not Specified
Configurations created using RecoverNow Version 4.0

There are two possible outputs for when:
170
Send partially filled containers automatically option is enabled on

page 171
Send partially filled containers automatically option is disabled on

page 171
Send partially filled containers automatically option is enabled

automatically for configurations created using RecoverNow v4.0 and the
Send partially filled containers automatically option enabled, the scconfig
-C <ContextID> -c command displays:
Send Partial Container Automatically is active.
Frequency to check = 300
Percent filled = 50
Send partially filled containers automatically option is disabled

automatically for configurations created using RecoverNow version 4.0
and the Send partially filled containers automatically option disabled the
scconfig -C <ContextID> -c command displays:
Send Partial Container Automatically is not active.
Frequency to check = 0
Percent filled = 0
Verify your Configuration using the sccfgchk

Command
Execute the following command on each node after the configuration has
been initialized. For information on the command syntax, refer to
sccfgchk on page 283.
sccfgchk -C <Context ID>
171
Support for LVM commands when the RecoverNow drivers are

loaded
The AIX Logical Volume Manager (LVM) manages Logical Volumes (LVs)
that RecoverNow protects.
Because RecoverNow manages Logical Volumes, not all the AIX LVM
commands are supported. The table below shows the LVM commands
supported when the RecoverNow drivers are loaded.
172
LCM Command
Status
chlv
OK
chlvcopy
OK
chvg
OK
cplv
OK. LV must be closed
defragfs
OK
exportvg
OK
extendlv
OK
extendvg
OK
fileplace
Not supported. DSI
importvg
OK
joinvg
Not supported. Unable to synchronize

logical volumes.
lslv
OK
lsvg
OK
lvmstat
Not supported. Cannot enable Statistics

collection.
migratepv
Not supported. Uses mirror and syncvg
mirrorvg
OK. But it will not begin sync when the

drivers are loaded.
mirscan
OK
mklv
OK
mklvcopy

logical volume.
mkvg
OK
readlvcopy
OK
Support for LVM commands when the RecoverNow drivers are loaded
LCM Command
Status
redefinevg
OK
reducevg
OK
reorgvg

logical volumes.
rmlv
OK, if the LV is not in PVS.
rmlvcopy
OK
snapshot
Not supported
splitlvcopy
OK
splitvg
OK
syncvg

logical volumes
synclvodm
OK
unmirrorvg
OK
varyoffvg
OK. All LVs must be closed
varyonvg
OK
173
RecoverNow Default Ports

RecoverNow always checks the /etc/services file for port assignments to be
used by a Context ID. If no port assignments are found in the /etc/services
file, RecoverNow will use the default ports listed below. If the default port
assignments are in use by another application, then specific port
assignments must be added to the /etc/services file for a Context ID.
The default port assignments shown are not added to the/etc/services file
when RecoverNow is installed.
sc<Context
sc<Context
sc<Context
sc<Context
sc<Context
sc<Context
sc<Context
sc<Context
ID>aa_channel
ID>ra_channel
ID>ca_channel
ID>aba_dchannel
ID>lca_dchannel
ID>aa_achannel
ID>aba_channel
ID>lca_channel
5747/tcp
5748/tcp
5749/tcp
5750/tcp
5751/tcp
5752/tcp
5753/tcp
5754/tcp
#
#
#
#
#
#
#
#
Archive Agent
Restore Agent (scrt_rs)
Restore Client Agent
Assured Backup Agent
Log Control Agent
Archive Agent
Log Control Agent
Adding specific port assignments

If multiple Context IDs are configured, each Context ID must have a unique
port assignment. Port assignments for Primary and Failover Context IDs
must also be unique.
For example, if the Primary Context ID is 25 and the Failover Context ID is
250, add the following entries to the /etc/services file on all servers.
sc25aba_channel
sc25lca_channel
sc25aa_channel
sc25ra_channel
sc25ca_channel
sc25aba_dchannel
sc25lca_dchannel
sc25aa_achannel
6901/tcp
6902/tcp
6903/tcp
6904/tcp
6905/tcp
6906/tcp
6907/tcp
6908/tcp
#
#
#
#
#
#
#
#

Log Control Agent
Archive Agent
Log Control Agent
Archive Agent
sc250aba_channel
sc250lca_channel
sc250aa_channel
sc250ra_channel
sc250ca_channel
sc250aba_dchannel
sc250lca_dchannel
sc250aa_achannel
6909/tcp
6910/tcp
6911/tcp
6912/tcp
6913/tcp
6914/tcp
6915/tcp
6916/tcp
#
#
#
#
#
#
#
#

Log Control Agent
Archive Agent
Log Control Agent
Archive Agent
The default port assignment shown below for the Configuration Daemon is
added to the/etc/services file when RecoverNow is installed.
scconfigd
174
7835/tcp
Activating syslog Debug
If the default port assignment for the configuration daemon is in use by

another application, then the port number used by the configuration daemon
must be changed.
To change the port number used by the Configuration Daemon:
Use stopsrc to stop the Configuration Daemon on all servers

stopsrc cs scrt_scconfigd
Change the port number for the scconfigd entry in the /etc/services file
on all servers to an unused port number.
Use startsrc to start the Configuration Daemon on all servers

startsrc s scrt_scconfigd
Activating syslog Debug

1. During the installation process, the following debug line is
automatically added to /etc/syslog.conf.
*.debug var/log/EchoStream/es_syslog.out rotate size
10m files 7 compress # EchoStream
2. tail f var/log/EchoStream/es_syslog.out
After you complete the RecoverNow post-installation tasks, refer to

Starting and Stopping on page 177 for information on how to start and
stop RecoverNow.
175
176
Starting and Stopping
RecoverNow provides capabilities to start up and shut down both the

production and the recovery servers.
Using VSP to Start and Stop RecoverNow on page 177
Using the command line to Start and Stop RecoverNow on

page 179
Using VSP to Start and Stop RecoverNow

This topic contains two sections:
Using VSP to Start RecoverNow on page 177
Using VSP to Stop RecoverNow on page 178
Using VSP to Start RecoverNow

The start action takes any necessary actions to get replication started. This
includes starting the processes and loading the drivers. For detailed
To start RecoverNow:
1. Log in to VSP. See Logging in to Vision Solutions Portal on
page 127.
3. Do one of the following for a specific replication group:
Click Start from the Action toolbar.
Select Start from the Actions dropdown.
177
Chapter 7: Starting and Stopping
The Start Replication Group dialog displays:
4. Click OK, to start the replication group for the selected servers.
The Start Replication Group dialog remains displayed until the action
completes successfully.
Using VSP to Stop RecoverNow

The start action takes any necessary actions to get replication started. This
includes starting the processes and loading the drivers. For detailed
NOTE
Applications must be stopped before stopping replication. File

systems are unmounted and data is no longer protected when
replication is stopped.
To stop RecoverNow:
1. Log in to VSP. See Logging in to Vision Solutions Portal on
page 127.
3. Do one of the following for a specific replication group:
Click Stop from the Action toolbar.
Select Stop from the Actions dropdown.
178
Using the command line to Start and Stop RecoverNow
The Stop Replication Group dialog displays:
4. Click OK, to stop the replication group for the selected servers.
The Stop Replication Group dialog remains displayed until the action
completes successfully.

This topic contains two sections:
Using the command line to Start RecoverNow on page 179
Using the Command line to Stop RecoverNow on page 180
Using the command line to Start RecoverNow

On the production server, start RecoverNow:
NOTE
Steps 1 and 2 are done for the first start after RecoverNow is
configured.
1. Stop any applications that are using the RecoverNow PVS (Production
Volume Set) LVs (Logical Volumes).
2. Make sure that any filesystems associated with the PVS LVs are
unmounted. Use the AIX umount command.
rtumnt -C <Context ID>
179
3. Start RecoverNow. Enter the following command:

rtstart -C <Context ID>
The following output will display:

# rtstart -C 34
Loading Double-Take Recover Now Production drivers
Starting scrt_lca
fsck -F vxfs -o p -y /dev/vx/rdsk/rootvg/vol2
/dev/vx/rdsk/rootvg/vol2:file system is clean - log replay
is not required
Mounting /dev/vx/dsk/rootvg/vol2 on /vol2...
fsck -F vxfs -o p -y /dev/vx/rdsk/rootvg/vol1
/dev/vx/rdsk/rootvg/vol1:file system is clean - log replay
is not required
Mounting /dev/vx/dsk/rootvg/vol1 on /vol1...
4. Start the application.

On the recovery and replicated servers, start RecoverNow by performing
the following steps:
5. Start RecoverNow.
This displays:
# rtstart -C 34
Loading Double-Take RecoverNow Recovery Server Drivers
Starting scrt_aba
Monitoring the Log Files

Use the tail command to check the log files on all servers to check that
processes are running properly.
1. Examine log files on the production server.
tail f /var/log/EchoStream/scrt_lca-<ContextID>.out
2. Examine log files on the recovery and replicated servers.

tail f /var/log/EchoStream/scrt_aba-<ContextID>.out
Using the Command line to Stop RecoverNow

On the production server, stop RecoverNow.
1. Stop the application.
2. Unmount the protected volumes.
180
3. Stop RecoverNow, synchronize data, and unload the drivers on the

production server.
rtstop -FSC <Context ID>
4. On the recovery and replicated servers, stop RecoverNow and unload

the drivers:
181
182
Creating Snapshots
Overview
RecoverNow enables you to restore a complete copy of the data on the
production server to any time in the past. You can quickly restore a
database that has crashed and rollback the data to a point before a logical
corruption occurred.
Using Snapshots to Protect Your Data on page 183
Using VSP to Create a Snapshot on page 184
Using the Command Line to Create a Snapshot on page 186
Using Snapshots to Protect Your Data

If damage to your data is limited to a single volume or group of volumes,
you can choose to rollback the affected volume(s). You do not have to
restore the entire data set.
RecoverNow lets you create and use snapshots for production restores
and snapshot-based backups to media such as tape. You can also create a
snapshot when you want to use a copy of the data on the recovery server.
Having a snapshota read or write copy of the dataon the recovery
server enables you to investigate and use the data without affecting the
operation of the production server. For example, you can:
Perform data mining without affecting performance on the production

server.
Repair individual objects such as files, tables, and records on the

production server without stopping the application or replacing all of
the application files.
Test whether the snapshot is the correct one to use for rolling back the
data on the production server.
183
Chapter 8: Creating Snapshots
Using VSP to Create a Snapshot

There are two ways to create a snapshot:
1. Do one of the following
Navigate to the Recovery page, and select Create Snapshot from the
Actions dropdown.
184
Using VSP to Create a Snapshot
Navigate to the Snapshot Details portlet, and select Create from the
dropdown.
The Create Snapshot dialog displays.
2. In Location in rollback window, specify the type of information to use

to locate in the rollback window, where to take the snapshot:
Point in TimeSpecifiy the date and time, within the rollback
window, from where the snapshot will be created. This is displayed
when Point in Time is selected in the Location in rollback window
field. This value is always displayed in the dropdown. This only
applies to the Date and time field.
185
Container IDIdentify the container ID, within the rollback

window, from where the snapshot will be created. This is displayed
when Container ID is selected in the Location in rollback window
field. This value is always displayed in the dropdown. This only
applies to the Container ID field. Container IDs must be even
numbers in the range specified in the rollback window. When a
change is made on the production server an even container ID is
created to represent that change. The even container ID is then
applied to the recovery server. When you create a snapshot, you use
the even container ID, that contains the change.
Event MarkerSelect one of the event markers, within the rollback
window, for where the snapshot will be created. Only one event
marker can be selected at a time.
Most Recently Applied ContainerSpecify the most recent
confirmed apply (committed) container, within the rollback window,
from where the snapshot will be created. This option is used when
all the data is synchronized and no changes are being made. You can
use this option to create a snapshot at that point in which you know
you have stable and consistent data.
3. Date and TimeSpecify the date and time, within the rollback window,
from where the snapshot will be created. The default is the most recent
date in the rollback window. It is only displayed when Point in Time is
selected.
4. Mount file systemsSpecifies whether or not to mount file systems in
the snapshot. The default is Yes.
5. Check file system consistencySpecify whether or not to check the
consistency of the file system after mounting it. This can only be
specified if yes is specified for mount file systems. The default is No.
You can use the fsck command to check and interactively repair
inconsistent file systems.
6. Click OK.
Using the Command Line to Create a Snapshot

You can use the command line to create a snapshot based on the current
redo log, a specific date and time, or a specific LFCID.
This section contains the following:
186
Creating Snapshots Based on the Current Redo Log on page 187
Creating Snapshots Based on a Specific Date and Time on page 189
Creating a Snapshot Based on a Specific LFCID on page 191
Creating Snapshots Based on the Current Redo Log

You can create a snapshot based on the current redo log. You can use this
snapshot for data mining and object repair on the production server.
1. On the recovery server, make sure all snapshot filesystems are
unmounted before trying to release the snapshot.
2. Make sure no snapshots already exist on the recovery server.

scrt_ra -C <Context ID> -W
3. On the recovery server, enter the following command to create a

snapshot based on the current redo log:
scrt_ra -C <Context ID> -X
You should see output similar to the following:

Making SNAP /dev/rsnc1lif_bk_1, 66.306
Making SNAP /dev/rsnc1dbmf_bk_1, 66.318
Making SNAP /dev/rsntestlv, 66.450
Making SNAP /dev/rsnrtlog, 66.454
Making SNAP /dev/rtestlv, 66.448
Making SNAP /dev/rrtlog, 66.452
Snap Devs (read only, raw)
Minor rj wj
Snap level
Vdev level
Vdevs (read/write, block)
------------------------------ ----- -- -- -------------/dev/rsnc1lif_bk_1
304 ON -10
0 N/A
/dev/rsnc1lif_bk_2
308 ON -10
0 N/A
/dev/rsnc1lif_bk_3
312 ON -10
0 N/A
/dev/rsnc1dbmf_bk_1
316 ON -10
0 N/A
/dev/rsntestlv
448 ON ON
10
10 /dev/testlv
/dev/rsnrtlog
452 ON ON
10
10 /dev/rtlog
4. Mount the volumes on the recovery server. Enter the following

command:
rtmnt -C <Context ID>
187
Determining Filesystems to mount...

fsck -fp -y /dev/rtestlv
log redo processing for /dev/rtestlv
syncpt record at 7028
end of log 7028
syncpt address 7028
number of log records = 1
number of do blocks = 0
number of nodo blocks = 0
/dev/rtestlv (/test): ** Unmounted cleanly - Check
suppressed
Mounting /test...
188
Creating Snapshots Based on a Specific Date and Time

In this section you:
Verify that the DATEMSK environment variable is defined
Establish the date format for the DATEMSK environment variable
Create Snapshots Based on a Specific Date and Time
Verify that the DATEMSK environment variable is defined

Before you can create a snapshot you must establish the date format for the
DATEMSK environment variable that is used by the recovery server.
To verify that the DATEMSK environment variable is defined:
1. Type, echo $DATEMSK
If no value is returned, the DATEMSK environment variable is
undefined. Proceed to Establish the date format for the DATEMSK
environment variable on page 189.
If, for example, /tmp/mdm, is returned, the date format for
DATEMSK environment is defined. To view the contents of
/tmp/mdm, type: cat /tmp/mdm.
Establish the date format for the DATEMSK environment variable

To specify a date format:
1. Create a file that contains the date format for the DATEMSK
environment variable. In this example, create /tmp/mdm.
NOTE
The DATEMSK environment variable must be set to the full-path

of a file that contains the date format template.
2. Enter the following:
export DATEMSK=/tmp/mdm
The following shows two examples based on the different date formats.
If the content of /tmp/mdm
is...
Then the date format for DATEMSK is...
%m/%d/%y %H:%M:%S
scrt_ra -C310 -D "05/05/09 16:15:00"
%y.%m.%d %H%M%S
scrt_ra -C310 -D "09.05.05 161500"
where:
189
%mMonth
%dDay
%yYear
%HHour
%MMinute
%SSecond
IMPORTANT
The date format must be specified exactly as it is in the

DATEMSK environment variable. For example, if you do not
specify seconds (%S) in DATEMSK, you cannot specify it in the
command.
Create Snapshots Based on a Specific Date and Time

You can create a snapshot based on a time in the past. You can use this
To create a snapshot:
2. Make sure that a snapshot does not already exist on the recovery server.
3. On the recovery server, enter the following command to create a

snapshot based on a specific date and time.
scrt_ra -C <Context ID> -D date
The date format depends on the DATEMSK environment variable

that is used by the recovery server.
If you enter 05/15/09 09:33:40 as the date and 34 for the Context ID,
the output is similar to the following:
scrt_ra -C34 -D "05/15/09 09:33:40"
You have requested a virtual incremental LFC restore to
time (1159536820) Fri May 15 09:33:40 2009
c(ontinue) or a(bort)? c
Making SNAP /dev/rsntestlv, 66.450
190
Making SNAP /dev/rsnrtlog, 66.454

Making SNAP /dev/rtestlv, 66.448
Making SNAP /dev/rrtlog, 66.452
Trying to match LFC to time: 1159536820
Found time 1159536820 in-between LFC #12 and LFC #14
Fetching LFC #17
Applying LFC #17
Fetching LFC #15
Applying LFC #15
Fetching LFC #13
Applying LFC #13
Fetching LFC #14
Applying LFC #14
The BackingStore Vdevs are at level 12+. ABA is at
level 18.
4. Mount the volumes on the recovery server:

You will see output similar to the following:

Verifying Operating Context ID.
Determining role of this system in the Context.
Mounting /pfs00...
Replaying log for /dev/esplv00.
Mounting /pfs01...
Mounting /pfs02...
Creating a Snapshot Based on a Specific LFCID

You can create a snapshot based on a specific LFCID. You can use this
2. Make sure that no snapshot already exists on the recovery server.

3. On the recovery server, create a snapshot based on a LFCID. The

LFCID should precede the sequence of the current redo log, but should
not be so far in the past that it has been archived to media such as tape.
scrt_ra -C <Context ID> -t <LFCID>
If the LFCID is 30, then the output is similar to the following:
191
Making SNAP /dev/vx/rdsk/rootdg/snvol2, 215.6

Making SNAP /dev/vx/rdsk/rootdg/snvol1, 215.10
Making SNAP /dev/vx/rdsk/rootdg/snc34lif_bk_1, 215.14
Making SNAP /dev/vx/rdsk/rootdg/snc34dbmf_bk_1, 215.26
Making SNAP /dev/vx/rdsk/rootdg/vol2, 215.4
Making SNAP /dev/vx/rdsk/rootdg/vol1, 215.8
Fetching LFC #33
Applying LFC #33
Fetching LFC #31
Applying LFC #31
The BackingStore Vdevs are at level 30. ABA is at level
34.
4. Mount the volumes on the recovery server.

rtmnt -C <Context ID> -f

log redo processing for /dev/rtestlv
end of log 9028
syncpt address 9028
number of log records = 1
number of do blocks = 0
number of nodo blocks = 0
/dev/rtestlv (/test): ** Unmounted cleanly - Check
suppressed
Mounting /test...
For information on database repair and database resurrection, refer to

Chapter 10, Working with RecoverNow Applications on page 211.
192
Administrative Tasks
This chapter describes RecoverNow administrative tasks:
Verify the configuration using the sccfgchk command on page 193
Updating the RecoverNow Configuration on page 194
Verifying Available Free Space on page 194
Extending a RecoverNow-Protected Filesystem on page 194
Increasing the Snapshot Journal Space on the Recovery Server on

page 197
Setting up JFS Log Isolation on page 198
Removing a Filesystem from a RecoverNow Protected jfslog on

page 199
Setting Up Error Notification on page 199
Marking State Maps Dirty and Synchronizing the PVS or one LV in

the PVS on page 204
Alternative Method for Performing Initial Synchronization on

page 205
Using IBM Power Systems Live Partition Mobility with

RecoverNow on page 206
Verify the configuration using the sccfgchk command

Use the sccfgchk command to check a configuration before you start
RecoverNow. Execute the following command on each node after the
configuration has been initialized. For information on the command
syntax, refer to sccfgchk on page 283.
sccfgchk -C <Context ID>
193
Chapter 9: Administrative Tasks
Updating the RecoverNow Configuration

If you update the RecoverNow configuration you must stop RecoverNow
processes and unload the RecoverNow drivers on the production site. This
means that you must also stop the production application and close and
unmount the production data.
After you install RecoverNow, VSP and the portal application, you can use
the RecoverNow Replication Group Wizard or the command line to
configure new replication groups and change, rename, and delete existing
ones. Refer to Configuring Replication Groups on page 131.
Verifying Available Free Space

Anytime a protected filesystem needs to be extended always check:
The State map limit is not exceeded.
On the production server, the protected Logical Volumes (LV)V MAX

LPs: size is not exceeded.
On the recovery server, the replica LV (pt LV) MAX LPs: size is
not exceeded.
On the protected and replica Volume Groups (VGs), there is enough

free space.
Refer to Extending a RecoverNow-Protected Filesystem on page 194 for

additional information.
Extending a RecoverNow-Protected Filesystem

RecoverNow-protected filesystems can be extended when RecoverNow is
active using the smit chfs command. When the filesystem is extended:
On the production server, the replica (pt LV) is automatically

extended when data is written to the expanded area.
NOTE
To manually extend the replica (pt LV), use the

extend_replica_lv command to force the expansion of a Replica
LV (Logical Volume):
extend_replica_lv -C <Context ID> -L <PVS LV>
194
On the recovery server, the write journal associated with the file system
is not extended.
Extending a RecoverNow-Protected Filesystem
The filesystem can only be extended within the limits of the state map size.
The default state map supports a filesystem size of 1 TB
If a protected filesystem needs to be extended, verify the following.
1. The state map limit will not be exceeded.
2. The protected LV MAX LPs: will not be exceeded.
3. The replica LV (pt LV) MAX LPs: will not be exceeded.
4. There is enough free space in the protected and replica VGs.
The following command provides the state map size.
rtattr -C <Context ID> -o smb<LV> -a Size
The default state map size is 33280, which allows a filesystem to expand to
1 TB.
The state map limit is calculated as follows. The default region size is 4
MB.
max filesystem size = (SMBitmap size -512) * 8 * region size
For example, if SMBitmap size is 3584 and region size is 4 MB.
max filesystem size = 3072 * 8 * 4 MB
max filesystem size = 98304 MB
Extending a Protected File System Beyond the State

Map Limit
To extend a protected filesystem beyond the state map limit:
1. On the production server, stop applications that are using the
RecoverNow protected logical volumes.
2. On the recovery server, stop applications that are using logical volumes
from an active snapshot.
3. On the production server, use rtstop to unmount protected filesystems,
transfer any current LFC data to the recovery server and unload the
RecoverNow Production Server Drivers.
/usr/scrt/bin/rtstop -FSC <Context ID>
NOTE
If you cannot unmount filesystems, specify the fuser -c

command to locate the processes that are holding the filesystem(s)
open.
195
4. On the recovery server, use rtstop to unmount any snapshot filesystems

and unload the RecoverNow recovery server drivers.
/usr/scrt/bin/rtstop -FC <Context ID>
NOTE
If you cannot unmount filesystems, specify the fuser -c

command to locate the processes that are holding the filesystem(s)
open.
5. On the production and recovery servers, use rtattr to change the state
map size. The new size must be a multiple of 512.
rtattr -C <Context ID> -o smb<LV> -a Size -v <new size>
6. On the production and recovery servers, use scsetup to delete device

special files from /dev.
scsetup -XC <Context ID>
7. On the production and recovery servers, use scsetup to remove

configured RecoverNow LV's:
scsetup -RC <Context ID>
8. On the production and recovery servers, use sccfgd_delcfg to delete

the Failover Context ID if one was configured.
sccfgd_delcfg <Failover Context ID>
9. On the production and recovery servers, use scsetup to create new

configured logical volumes.
10. On the production and recovery servers, use rtdr to create a failover
context if one was configured.
rtdr -C <Primary Context ID> -F <Failover Context ID>
setup
11. On the production server, use scconfig to wipe the state maps clean
(0.000% dirty).
scconfig -WC <Context ID>
12. On the Production and Recover Servers, use rtstart to start

RecoverNow.
196
Increasing the Snapshot Journal Space on the Recovery Server
13. On the production server, use smit chfs command to extend the
filesystem.
Increasing the Snapshot Journal Space on the Recovery Server

The snapshot journal space can be increased on the recovery server without
impacting the production environment. To increase the Snapshot Journal
space on the recovery server:
1. Stop RecoverNow on the recovery server.
2. Unmount any snapshot filesystems.
/usr/scrt/bin/rtumnt -C <Context ID>
3. Release a snapshot if it exists.

/usr/scrt/bin/scrt_ra -C <Context ID> -W
4. Stop RecoverNow.
/usr/scrt/bin/rtstop -C <Context ID> -Fk
5. Use rtattr to increase the size of an existing snapshot journal volume.

NOTE
Any snapshot journal can be increased in size since all snapshot

volumes are available for use during snapshot creation/use.
6. Determine the name of the snapshot journal to increase.
7. Use the following command to find the snapshot journal volume sizes:
/usr/scrt/bin/rtattr -C <Context ID> -t
SCRT/containers/WJournal -a Size
The ObjectName of a snapshot volume that you want to increase.

8. Select one of the snapshot journals and increase the size of the volume.
/usr/scrt/bin/rtattr -C <Context ID> -o <ObjectName>
-a Size -v <new size value in bytes>
9. Find the logical volume name for the snapshot journal.

rtattr -C <Context ID> -o <ObjectName from step 8 on
page 197>
-t SCRT/containers/WJournal -a FileName
197
10. Remove the existing snapshot journal volume that you want to increase
rmlv -f <ObjectAttributeValue from step 9 on
minus the /dev/r prefix>
page 197
11. Create the new snapshot journal volume.

/usr/scrt/bin/scsetup -C <Context ID> -M
12. Restart RecoverNow.

/usr/scrt/bin/rtstart -C <Context ID>
Setting up JFS Log Isolation

When you use jfs or jfs2 filesystems with outline logs, ensure that the
outline logs that are used by the RecoverNow-protected filesystems are also
protected by RecoverNow. Also, ensure that filesystems that RecoverNow
does not protect do not use these protected outline jfslogs.
For example, if there are 4 logical volumes /lvfs1, /lvfs2, /lvfs3, /lvfs4 that
use the same jfslog /dev/fsloglv00 and RecoverNow only protects /lvfs1,
You need to need have a:
Separate jfslog for the unprotected logical volumes
Separate jfslog for the protected logical volume
Isolated jfslog RecoverNow protects and is used by /fs1.

NOTE
When using inline logs with jfs2 filesystems, there is no jfslog

isolation requirement.
To maintain jfslog isolation:
1. Unmount the filesystem that you will assign to the new jfslog.
umount /fs1
2. Create a new logical volume with a type of jfslog.

mklv -t jfslog -y< newjfslog> <vgname> 1 ; where
<newjfslog> is the name of the new jfslog and
<vgname> is the volume group where the jfslog is
created
3. Format the new jfslog.

logform /dev/<newjfslog>
198
Removing a Filesystem from a RecoverNow Protected jfslog
4. Respond yes to allow the jfslog to be formatted.

5. Assign the new jfslog to the filesystem.
chfs -a log=/dev/<newjfslog>
/fs1
Removing a Filesystem from a RecoverNow Protected jfslog

When you decide to remove a filesystem from a PVS, this filesystem needs
to use a different jfslog from the RecoverNow protected jfslog. Follow the
steps below to associate the newly removed filesystem with a non-protected
jfslog.
Assumption: The filesystem /jfsold is being removed from the RecoverNow
PVS. The RecoverNow protected filesystems are unmounted, the
RecoverNow processes are stopped and the RecoverNow data tap is
unloaded.
1. If a jfslog exists in the volume group that is not currently part of the
PVS, you can assign that jfslog to the filesystem that is being removed
from the PVS.
umount /jfsold
chfs -a log=/dev/<nonrtjfslog>
/jfsold
where nonrtjfslog is a jfslog that exists in the volume group but is not
part the PVS.
2. If a jfslog does not exist in the volume group:
mklv -t jfslog -y< newjfslog>
<vgname> 1
where <newjfslog> is the name of the jfslog that you are creating for the
non-protected filesystem to use
3. Format the jfslog.
logform /dev/<newjfslog>
4. Assign the jfslog to the non-protected filesystem

chfs -a log=/dev/<newjfslog>
/jfsold
Setting Up Error Notification

Use the AIX Error Notification feature to notify the administrator via email
or take action when certain errors are recorded in the AIX error log.
199
The following example script /home/scrt/enm_1 can be used to record

this information in /tmp/enm_1.out and notify the administrator via email
when certain errors occur.
#!/bin/ksh
PROGNAME=${0##*/}
exec 1>> /tmp/${PROGNAME}.out
exec 2>&1
printf "error notify method\n"
errpt -Al ${1}
errpt -Al ${1} | mail -s Error root
To enable AIX Error Notification for one or more of the RecoverNow

errors, you must add a stanza to the errnotify ODM. When the error is
logged, the error notification daemon will call the notify method specified
in the errnotify ODM stanza. The errnotify ODM is located in the
/etc/objrepos directory.
For example, the following errnotify stanzas are added to file
/tmp/ern_1.add for error labels SCRT_LFC_READ_ERROR,
SCRT_LFC_WRITE_ERROR, "SCRT_NETWORK_ERROR" and
"SCRT_ABORT_ERROR.
errnotify:
en_name = "SCRT_LFC_READ_ERROR"
en_class = "S"
en_type = "PERM"
en_method = "/home/scrt/enm_1 $1"
errnotify:
en_name = "SCRT_LFC_WRITE_ERROR"
en_class = "S"
en_type = "PERM"
errnotify:
en_name = "SCRT_NETWORK_ERROR"
en_class = "S"
en_type = "PERM"
errnotify:
en_name = "SCRT_ABORT_ERROR"
en_class = "S"
en_type = "PERM"
200
ODM Commands
The errnotify stanzas are added to the errnotify ODM using the
following command.
odmadd /tmp/ern_1
To list the errnotif stanzas added for RecoverNow errors do the

following command.
odmget -q"en_name LIKE SCRT_*" errnotify|more
To delete all the errnotify stanzas added for RecoverNow errors:

odmdelete -q"en_name LIKE SCRT_*" -o errnotify
To delete a particular stanza:

"SCRT_LFC_WRITE_ERROR" "errnotify"
stanzas:
odmdelete -qen_name=SCRT_LFC_WRITE_ERROR -o errnotify
Error Notify Messages

Table 1shows the Error Notify Messages that are written to the AIX error
log.
Table 1: Error Notify Messages

Error
Description
SCRT_AAGENT_ABORTED_ERROR:
PERM S Archive Agent Aborted

Operation
SCRT_AAGENT_API_ERROR:
PERM S Error conversing with Archive

Program
SCRT_ABORT_ERROR:
PERM S Generic/Fatal RealTime Error
SCRT_ARCHIVEMGR_ERROR:
PERM S Archive Manager Error
SCRT_BSFC_READ_ERROR:
PERM S Read error from replicated store
SCRT_BSFC_WRITE_ERROR:
PERM S Write Error to replicated store
SCRT_CONFIG_ERROR:
PERM S Configuration Error
SCRT_CONFIG_UPDATE_ERROR:
TEMP S Configuration update failure
SCRT_DBMFC_APPLY_ERROR:
PERM S Logical error applying LFC to

DBMFC
SCRT_DBMFC_READ_ERROR:
PERM S Reading from DevBlockMap

failed
SCRT_DBMFC_WRITE_ERROR:
PERM S Writing to DevBlockMap failed
SCRT_DEVREAD_ERROR:
PERM S Failure to read to required device
201
202
Error
Description
SCRT_DEVWRITE_ERROR:
PERM S Failure to write to required device
SCRT_LFC_READ_ERROR:
PERM S Read error on an LFC
SCRT_LFC_WRITE_ERROR:
PERM S Write error on an LFC
SCRT_LIFC_FULL_ERROR:
TEMP S LIFC is full
SCRT_LIFC_READ_ERROR:
PERM S Read of LIFC failed
SCRT_LIFC_WRITE_ERROR:
PERM S Write of Log Record to LIFC

failed
SCRT_LRFC_READ_ERROR:
PERM S Read of LogRecord from LRFC

failed
SCRT_LRFC_WRITE_ERROR:
PERM S Writing of LogRecord to LRFC

failed
SCRT_NETWORK_ERROR:
PERM S Cannot connect to peer process

over network
SCRT_POOL_EMPTY:
TEMP S Storage pool empty
SCRT_POOL_HWM:
TEMP S Storage pool usage at/above high

water mark
SCRT_SCJ_COWREAD_ERROR:
PERM S Copy-On-Write read failed
SCRT_SCJ_COWWRITE_ERROR:
PERM S Journal Copy-On-Write write

failed
SCRT_SCJ_FULL_ERROR:
PERM S Journal Copy-On-Write log is full
SCRT_SCLM_WANCHORCB_ERROR
PERM S Log anchor write call back failed
SCRT_SCLM_WANCHOR_ERROR:
PERM S Log anchor write failed
SCRT_SCSM_BMWRITE:
PERM S Write to statemap bitmap device

failed
SCRT_SCSM_LOGWRITE:
PERM S Write to statemap log device

failed
SCRT_SHAREDD_EXPORT_ERROR:
PERM S Failure to export shared disk
SCRT_SHAREDD_IMPORT_ERROR
PERM S Failure to import shared disk
SCRT_SMFC_READ_ERROR
PERM S StateMap File Container Read

Error
SCRT_SMFC_WRITE_ERROR
PERM S StateMap File Container Write

Error
SCRT_SM_ERROR
PERM S StateMap creation error

Error
Description
SCRT_SNAP_OFF:
PERM O Snapshots taken down
SCRT_SRC_ERROR
PERM S System Resource Controller

Error
203
Marking State Maps Dirty and Synchronizing the PVS or one LV

in the PVS
Use the scconfig command to:
Mark state map bitmaps dirty for all devices or specified devices while
RecoverNow is active.
Perform a full synchronization of the Production Volume Set or one LV

in the Production Volume Set. Refer to Chapter 14 CLI Commands
on page 261 for more information.
scconfig -C <Context ID> -B
Where
-C is the context
Context ID
is the context ID number
marks statemap bitmaps dirty for all devices or specified devices while
RecoverNow is active and performs a full synchronization of the Production
Volume Set or one LV in the Production Volume Set.
-B
204
Alternative Method for Performing Initial Synchronization
Alternative Method for Performing Initial Synchronization

To optimize the time to transfer the PVS data onto a remote recovery server
at a remote site:
NOTE
You can also use the -L option with the scconfig command for
specific logical volumes to mark the state map zero percent dirty.
1. On the production server:
Stop any application(s)
Stop RecoverNow and unload the drivers:
2. On the recovery server

Stop any application(s) that are using a snapshot
Unmount the filesystems on a snapshot
Release any snapshots

scrt_ra -WC <Context ID>
Stop RecoverNow and unload the drivers.

3. Save the protected data on the production server to tape or disk. You
must save at the LV level in block sequence.
Save to tape:
dd if=/dev/db2 of=/dev/rmt0 bs=1024
Save to disk:
dd if=/dev/db2 of=/dev/db2bu bs=16m
4. Clean the state map on the production server.

scconfig -WC <Context ID>
5. Start RecoverNow on the production server.

6. Restore the data from tape or disk to the Replica on the recovery server.
NOTE
The restore must be done to the pt LVs.
205
Restore data from tape:

dd of=/dev/ptdb2 if=/dev/rmt0 bs=1024
Restore data from disk:

dd of=/dev/ptdb2 if=/dev/db2bu bs=16m
7. Start RecoverNow on the recovery server. The changes made after the
save to tape or disk synchronize to the recovery server.
Using IBM Power Systems Live Partition Mobility with

RecoverNow
This section contains:
Overview on page 206
Automatically Registering the es_migrate script on page 206
Manually Registering the es_migrate script on page 207
Before you Migrate a Partition on page 208
Migrating a Partition on page 208
Overview
RecoverNow supports Live Partition Mobility for partitions running AIX
5300-07 or later or AIX 6.1 or later or AIX 7.1 or later on POWER6 or
POWER7 technology-based systems. Live Partition Mobility allows a
partition that is replicating with RecoverNow to be migrated to another
system without interrupting replication.
Automatically Registering the es_migrate script

To support IBM Power Systems Live Partition Mobility, RecoverNow must
be Migration-aware.
To make RecoverNow Migration-aware, when you install RecoverNow, the
es_migrate script is automatically registered on systems capable of Live
Partition Mobility, with the Dynamic Logical Partition (DLPAR) function
of AIX.
206
Using IBM Power Systems Live Partition Mobility with RecoverNow
IMPORTANT
The es_migrate script is registered at installation time if the

partition is capable of migration. In which case you do not have to
register the es_migrate script. You should initially verify if it is
registered, if not, proceed to register the es_migrate script.
Manually Registering the es_migrate script

If you have installed RecoverNow when the partition was not
migration-capable, and then upgraded so the partition is migration-capable,
you must manually register the es_migrate script to make the node capable
of Live Partition Mobility.
To register the es_migrate script, execute the following command:
/usr/sbin/drmgr -f -i /usr/scrt/bin/es_migrate
The command output displays:

DR script file /usr/scrt/bin/es_migrate installed successfully
207
Before you Migrate a Partition

Before you migrate a partition, use the drmgr l command to verify the
existence of the following stanza:
# /usr/sbin/drmgr -l
DR Install Root Directory: /usr/lib/dr/scripts
Syslog ID: DRMGR-
Migrating a Partition
Each time you migrate a partition, RecoverNow is licensed to run on the
migrated partition for up to 30 consecutive days. If the license is valid for
less than 30 days, RecoverNow will be licensed to run on the number of
days remaining on the original partition. When you migrate back to original
partition the license expiration date is returned to its original value.
IMPORTANT
Vision Solution recommends that you do not make changes to the

RecoverNow configuration when the partition has been migrated.
If the partition has been migrated, and configuration changes are
necessary, you must load and work with the configuration from the
migrated partition. When you change a configuration where one or
more nodes have been migrated, always rediscover the Host ID
after loading the configuration.MPORTANT
208
209
210
Working with RecoverNow

Applications
10
This chapter describes working with RecoverNow applications.

Overview on page 211
Database Recovery Operations Scenarios on page 211

Database Resurrection Concept on page 212
Database Repair Concept on page 213
Performing a Volume Restore on page 213
Performing a Production Restore on page 220
Using VSP for a Production Restore on page 220
Using the Command Line to Perform a Production Restore on

page 223
Overview
You can apply the redo and undo logs from the recovery server to rollback
application data on the production server and restore the data to an earlier
point in time. When you rollback the application data the information is
synchronized with the replica on the recovery server similar to any other
write action.
Database Recovery Operations Scenarios

This section describes two database recovery scenarios that demonstrate
how to utilize RecoverNow's production and virtual restore capabilities.
These two recovery scenarios are database resurrection and database
repair.
Database resurrection is a restore from a database "crash and burn"

scenario, where the database could not come up.
211
Chapter 10: Working with RecoverNow Applications
Database repair is a restore of a burning, but living database with

corrupted data.
Keep in mind that the majority of all database recovery operations require
database repairs that are typically performed by DataBase Administrators
(DBAs). Also, the remaining minority of recovery operations need database
resurrections that are done by IT administrators. In the burning repair
scenario, for example, RecoverNow makes the DBA's job more precise.
Database Resurrection Concept

To perform a database resurrection, you may use RecoverNow's production
restore operation to recover an image of the actual production database to
some point in the past directly on the production disk itself (not a snapshot
image of the data).
LAN
Production
Server
Recovery
Server
Data tap
Application
Data Storage
Data tap
Undo
Logs
Undo
Replica
Storage
Logs
In the resurrection scenario, the database is crashed and will not be coming
back up. If you need to rollback to 55 seconds, you do not need to restore
last days full backup and replay a full week of database redo (roll forward)
logs. In this case, RecoverNow can rollback a totally crashed and burned
database in a matter of momentsmany orders of magnitude faster.
Keep in mind that during a production restore, you do not need the database
instance up. In fact, the database cannot be up while RecoverNow rolls its
image around on disk. By definition, the database is down since it crashed.
As a result, the best and fastest way to restore your database is while it is
212
Performing a Volume Restore
still down. The RecoverNow production restore operation provides this

capability.
Database Repair Concept

To perform a database repair, you may use the RecoverNow's virtual
restore operation, recovering a snapshot image of the database to some
point in the past on the writable snapshot, and then a pick and pull
operation to extract pieces of data from that snapshot for reinsertion into the
production database.
LAN
Production
Server
Recovery
Server
Data tap
Data tap
Snapshot
Log File
Application
Data Storage
Containers
Log File
Containers
Replica
Storage
COW
In this repair scenario, only some pieces of data are corrupted but the
production database is still running. After you restore a historical,
non-burning image of the database on the snapshot, you can pull pieces out
of it and put them back into the live production database to repair the bad
pieces of data. You can put out the fire with information. RecoverNow
restores automatically on the back-end snapshot.
Using the RecoverNow snapshot to visualize complete historical images of
the database enables DBAs to forgo spending time on a daily basis saving
pieces of database for possible use at a later time. Also, DBAs have access
to the exact historical image they want (typically by reviewing logs, etc.).

You can restore failed production volumes without having to do a full
production restore by doing the following tasks:
213
Step 1: Stop Double-Take RecoverNow on the Production Server on

page 214
Step 2: Create New Volumes to Replace Failed Volumes on page 214
Step 3: Create a Snapshot on the Recovery Server on page 215
Step 4: Validate the Information Provided by the Snapshot on

page 216
Step 5: Destroy the Snapshot on page 217
Step 6: Recreate the Snapshot on the Recovery Server on page 217
Step 7: Copy Volumes on Recovery Server to Replacement Volumes

on Production Server on page 217
Step 8: Restart Double-Take RecoverNow on the Production Server

on page 217
Step 9: rollback the Appropriate Context on the Production Server on

page 218
Step 10: Mount the Volumes for the Context on page 219
Step 11: Destroy the Snapshot on page 220
These tasks are described in detail in the subsequent sections.
Step 1: Stop Double-Take RecoverNow on the Production Server

Stop RecoverNow on the production server by executing the following
command:
rtstop C <Context ID> -F
Step 2: Create New Volumes to Replace Failed Volumes

The new volumes must have the same names, be in the same volume group,
and be of the same size as the original volumes. Execute the following
command on the production server to display this information:
sclist C <Context ID> t pdfc
EXAMPLE:
In this example, the Context ID is 1. The volume that needs to be replaced is
/dev/rrtctx1. Display information about the volume.
sclist C 1 t pdfc
The output shows that /dev/rrtctx1 belongs to volume group rtvg1. The
volume size is shown in bytes (1073741824).
Object: <loglv00>, Type: <SCRT/containers/PDFC>, Serial <355>:
214
HostName (string) = <production> (No Default)

FileName (string) = </dev/rloglv00> (No Default)
HeaderFileName (string) = </usr/scrt/loglv00.hdr> (No Default)
Size (longlong) = <16777216> (No Default)
DiskGroupHint (string) = <rtvg1> (No Default)
Type (string) = <jfslog> (raw)
Location (string) =
<No Value> (No Default)
AccessGroupID (int) = <0> (0)
Object: <rtlv2>, Type: <SCRT/containers/PDFC>, Serial <362>
FileName (string) = </dev/rrtlv2> (No Default)
HeaderFileName (string) = </usr/scrt/rtlv2.hdr> (No Default
Type (string) = <jfs> (raw)
Location (string) = </ctx12> (No Default)
Object: <rtctx1>, Type: <SCRT/containers/PDFC>, Serial <369>:
FileName (string) = </dev/rrtctx1> (No Default)
HeaderFileName (string) = </usr/scrt/rtctx1.hdr>(No Default)
Type (string) = <jfs> (raw)
Location (string) = </ctx1> (No Default)
Step 3: Create a Snapshot on the Recovery Server

Create a snapshot on the recovery server that contains a copy of each
volume to be restored. The snapshot provides the data to be copied to the
replacement volumes. Select a time for the snapshot that is before the time
that the disk failed. To determine the time at which the disk failed, you need
to inspect the system logs.
To create the snapshot, enter one of the following commands on the
recovery server:
scrt_ra C <Context ID> D <time>
or
scrt_ra C <Context ID> t <LFCID>
The allowable formats for times entered with the D option are specified in
the file identified by the DATEMSK environment variable.
EXAMPLE:
In this example, the LFCID is 158. The Context ID is 1.
scrt_ra C 1 -t 158
215
The following output results.

Making SNAP /dev/rsnloglv00, 41.6
Making SNAP /dev/rsnrtlv2, 41.10
Making SNAP /dev/rsnrtctx1, 41.14
Making SNAP /dev/rloglv00, 41.4
Making SNAP /dev/rrtlv2, 41.8
Making SNAP /dev/rrtctx1, 41.12
Fetching LFC #173
Applying LFC #173
Fetching LFC #171
Applying LFC #171
Fetching LFC #169
Applying LFC #169
Fetching LFC #167
Applying LFC #167
Fetching LFC #165
Applying LFC #165
Fetching LFC #163
Applying LFC #163
Fetching LFC #161
Applying LFC #161
Fetching LFC #159
Applying LFC #159
The BackingStore Vdevs are at level 158.
ABA is at level 174.
Step 4: Validate the Information Provided by the Snapshot

Validate the information provided by the snapshot by running the
application on the recovery server.
IMPORTANT
Do not try to validate the information by running the application

on the production server after the data is restored. If the data turns
out to be incorrect, you need to rollback the data included in all
previous attempts to restore data.
If the data is valid, stop the application. Note the Context ID and the time or
LFCID used to create the snapshot.
If the data is not valid, then stop the application and create the snapshot
again as described in Step 3: Create a Snapshot on the Recovery Server
on page 215. Continue until you have a snapshot with valid data.
216
Step 5: Destroy the Snapshot

Verifying the data in the snapshot may involve mounting the snapshot on
the recovery server and incurring I/O or other causing other changes to the
snapshot. Thus, you need to destroy and recreate the snapshot before you
use it for recovery.
Destroy the snapshot by using the following command:
scrt_ra C <Context ID> -W
EXAMPLE:
The Context ID in this example is 1. Enter the following command:
scrt_ra -C 1 W
This results in output similar to the following:

Snap Devs (read only, raw) Minor rj wj Snap level
block)
-------------------------- ----------- -------------------------------/dev/rsnloglv00
4 -- -0
/dev/rsnrtlv2
8 -- -0
/dev/rsnrtctx1
12 -- -0
/dev/rsnc1lif_bk_1
16 -- -0
/dev/rsnc1lif_bk_2
20 -- -0
/dev/rsnc1lif_bk_3
24 -- -0
/dev/rsnc1dbmf_bk_1
28 -- -0
/dev/rsnc1dbmf_bk_2
32 -- -0
/dev/rsnc1dbmf_bk_3
36 -- -0
Vdev level Vdevs(read/write,

---------0
0
0
0
0
0
0
0
0
/dev/loglv00
/dev/rtlv2
/dev/rtctx1
N/A
N/A
N/A
N/A
N/A
N/A
Step 6: Recreate the Snapshot on the Recovery Server

Recreate the snapshot on the recovery server using the same command that
you used before.
scrt_ra C <Context ID> D <time>
or
scrt_ra C <Context ID> t <LFCID>
Step 7: Copy Volumes on Recovery Server to Replacement Volumes

on Production Server
If you have rsh enabled on the production server, then you can use the dd
command to copy the data.
EXAMPLE:
dd if=/dev/rrtctx1 ibs=1024k |
rsh production_server_name -l root dd of=/dev/rrtctx1
obs=1024k
Step 8: Restart Double-Take RecoverNow on the Production Server

Use the following command to restart RecoverNow:
217
rtstart C <Context ID> -M
Restarting RecoverNow on the production server begins a region

synchronization between the production server and the recovery server.
Monitor the system log or the RecoverNow log until the following message
appears:
--------------------------------------------------------------- Dynamic SuperTransaction recovery complete ---------------------------------------------------------------
Step 9: rollback the Appropriate Context on the Production Server

Enter the following command on the recovery server:
scrt_rc C <Context ID>
You will be prompted for the time or LFCID that you used to create the
snapshot.
EXAMPLE:
scrt_rc C 1
The following is a sample rollback session.

-----------------------------------------------Double-Take RecoverNow Production Restore Shell
-----------------------------------------------Ensuring LCA and ABA are up.
LCA checked in.
ABA checked in.
Checking initial conditions...
Checking production status...
Bringing up restore server...
RestoreServer checked in.
Checking backend status...
Initializing LCA restore session...
LCA is in restore mode.
Initializing RS restore session...
Session is in production lead restore mode.
--- Restore session initialized! --type "help" at the prompt for a list of available commands.
rc>
218
At this prompt, enter restore.

rc> restore
Restore target type:
t - time
e - event marker
l - lfc
a - abort
At this prompt, enter l for LFCID or t for time. Use the same information
that you used to create the snapshot in Step 3: Create a Snapshot on the
Recovery Server on page 215.
> l
LFC target >158
You have requested an incremental LFC restore
from Tue May 12 08:16:24 2009 (1080058584)
to Mon May 10 11:17:29 2009 (1079983049),
LFCIDs 250+ to 158.
c(ontinue) or a(bort)? c
Rolling LFC restore status.
--------------------------Production at LFCID 248.
Production at LFCID 246.
...
Production restored to LFCID 158.
Backingstore remains stable at LFCID 250.
rc> commit
committing...
RestoreServer is down, exit code 0.
Restore Client session complete.
Step 10: Mount the Volumes for the Context

On the production server, enter the following command to mount the
volumes for the context:
rtmnt C <Context ID>
You may need to use fsck on the file systems before they can be mounted. If
this is necessary, then unmount the volumes using the rtumnt command, run
fsck, and then mount the volumes using the rtumnt command shown above.
219
Step 11: Destroy the Snapshot

On the recovery server, use the following command to destroy the snapshot:
scrt_ra C <Context ID> W
Performing a Production Restore

RecoverNows production restore operation allows you to rollback the
protected volumes on the production server. You can roll them back to a
specific LFCID or to a specific time. Before you commit the rollback,
however, you need to verify that you have chosen the correct time or LFCID
to rollback to.
NOTE
Even if you decide later that you would like to pick a better time or
a LFCID, you can roll forward or rollback to that point. This is
possible because RecoverNow keeps all of the change information
on the recovery server.
Using VSP for a Production Restore

Use the Rollback Production Server wizard to rollback the production
server to a specific location in the rollback window. For rollback progress,
see the Production Server Rollback portlet on the Recovery page.
Before starting the rollback, validate the rollback location using a snapshot
and stop all applications that are using the logical volumes. Use the
Replication Group portlet on the Replication page to stop the replication
group. During the rollback, file systems must be unmounted. For rollback
progress, see the Production Server Rollback portlet on the Recovery page.
NOTE
Data changed on the production server between the rollback

location and your most recent changes will be lost.
To start a production server rollback:
1. Select a replication group in the Recovery portlet.
Select Rollback, from the Action dropdown menu on the Recovery
portlet.
Click Rollback on the Production Server Rollback portlet.
220
The Rollback Production Server (Unmount File Systems) wizard starts.
221
3. Unmount file systemsSpecify whether to automatically or manually

unmount the file systems. During the rollback, file systems must be
unmounted. For rollback progress, see the Production Server Rollback
portlet on the Recovery page.
4. Click Next
The Rollback Production Server (Start Rollback) wizard displays.
5. Use the Location in rollback section, to specify the type of information

to use to locate in the rollback window, where the production server will
222
be rolled back to. Select either Point in Time, Container ID, or Event
Marker:
Point in TimeSpecify the date and time, within the rollback
window, where the production server will be rolled back to.
Container IDSpecify the container ID, within the rollback
Event MarkerSpecify the event marker, within the rollback
6. Mount file systemsSpecify whether to automatically or manually
unmount and mount and the file systems.
7. Click Finish to start the rollback.
The Production Server Rollback portlet shows the progress of the
rollback.
Using the Command Line to Perform a Production

Restore
To perform a production restore, complete the following tasks:
1. Stop the application on the production server.
2. Unmount the protected volumes on the production server. Enter the
following command:

Determining Filesystems to unmount...
Unmounting /dev/testlv from /test...
Sync: transferring current LFC to Recovery Server.
Waiting for synchronization of data to complete.
All data has been synchronized to the Recovery Server.
3. On the recovery server, start the production restore shell.

scrt_rc -C <Context ID>
This should result in output similar to the following:

LCA checked in.
ABA checked in.
223

rc>
4. Enter restore. This results in the following output:

t - time
e - event marker
l - lfc
a - abort
5. You can choose to rollback the production server to a specific time or a

specific LFCID. If the current redo LFCID is 36 and you choose to
rollback to LFCID 30, you should see output similar to the following:
from Fri May 15 15:02:08 2009 (1088200928)
to Wed May 13 15:38:03 2009 (1088030283),
LFCIDs 36+ to 30.
c(ontinue) or a(bort)?
6. Enter c to continue.
Rolling LFC restore status
-------------------------Production at LFCID 34
Production at LFCID 32
Production at LFCID 30
Backingstore remains stable at LFCID 36
rc>
224
NOTE
Although you have rolled back the production server to LFCID

30, the snapshot on the recovery server still contains information
up to and including LFCID 36. This allows you to verify that
LFCID 30 is the one you want to rollback to. If it is not, enter
abort. Determine the proper LFCID and start the production
restore shell as described in step 3.
7. If you are satisfied that LFCID 30 is correct, then enter commit. You
should see output similar to the following:
committing....
8. On the production server, mount the protected volumes. Enter the

following command:

fsck -fp -y /dev/rlvesp00
/dev/rlvesp00 (/pfs00): ** Unmounted cleanly - Check
suppressed
Mounting /pfs00...
fsck -fp -y /dev/rlvesp01
The current volume is: /dev/lvesp01
Primary superblock is valid.
Mounting /pfs01...
225
226
Working with Archived Data
11
RecoverNow supports media management products such as Tivoli

Storage Manager on AIX platforms. This chapter contains:
Overview of Working with Archived Data on page 227
Performing a Snapshot-based Backup to Archive Media on

page 227
Performing a Production Restore from Archived Logs on page 228
Overview of Working with Archived Data

You can have additional data protection by keeping complete copies of
the data on archive media such as tape. To create copies on the archive
media use snapshots from the recovery server.
Archived copies of the data are useful to:
Perform a production restore on the production server. You may want

to do this if the data on the recovery server is unavailable or if you
need to rollback to a point in the data that is no longer stored on the
recovery server.
Create a snapshot on the recovery server. You may want to do this if

you need to restore the data on the recovery server or if you want
additional copies of the data for data mining.
Create additional copies of the data on alternate devices, such as,

logical volumes.
Performing a Snapshot-based Backup to Archive Media

This section explains how to put a complete copy of your protected data
on archive media. You may back up your data to archive media as often as
you like.
On the recovery server, enter the following command:
scrt_vfb -C <Context ID>
227
Chapter 11: Working with Archived Data
Performing a Production Restore from Archived Logs

You can perform a production restore from archived LFCs when the LFCs
on the recovery server do not go back far enough or when they are
unavailable. Like any other production restore, you must stop the
application on the Production Server and unmount the protected volumes
before you roll them back.
To perform a production restore from archived LFCIDs, do the following:
1. Stop the application on the production server.
2. Unmount the protected volumes on the production server. Enter the
following command:
rtumnt -C <Context ID><Context ID>

Determining Filesystems to unmount...
Unmounting /dev/testlv from /test...
Sync: transferring current LFC to Recovery Server
Waiting for synchronization of data to complete.
3. On the recovery server, start the production restore shell. Enter the
following command:
scrt_rc -C <Context ID>
This results in output similar to the following:

LCA checked in.
ABA checked in.
228
Performing a Production Restore from Archived Logs
rc>
4. To display available restore target windows. Enter targets.

Current Snap Time:
1201274570 (LFCID: 1466): Fri Jan 25 10:22:50 2008
Available internal & external (tape) rollback windows:
-------------------------------------------------------Start: 1201274570 (LFCID: 1466): Fri Jan 25 10:22:50 2008
End: 1201273198 (LFCID: 1352): Fri Jan 25 09:59:58 2008
Available VFBs:
-------------------------------------------------------1201269025 (LFCID: 1342): Fri Jan 25 08:50:25 2008
1193078943 (LFCID: 1986): Mon Oct 22 14:49:03 2007
1192817678 (LFCID: 1456): Fri Oct 19 14:14:38 2007
rc>
5. Enter restore. This results in the following output:

t - time
e - event marker
l - lfc
a - abort
>
6. Choose l for Log File Container.

This results in the following output:
LFC target >
If you are restoring from archived data, then the LFCID that you choose
identifies data that is no longer stored on the recovery server.
7. Enter a LFCID <example 1360>
This results in the following output:
from Fri Jan 23 10:22:50 2009 (1201274570)
to Fri Jan 23 10:01:26 2009 (1201273286),
LFCIDs 1466+ to 1360.
c(ontinue) or a(bort)?
8. Enter c to continue.
229
Chapter 11: Working with Archived Data

Rolling LFC restore status.
--------------------------Production at LFCID 1464.
Backingstore remains stable at LFCID 1466.
rc>
NOTE
Although you have rolled back the production server to LFCID

1360, the snapshot on the recovery server still contains
information up to and including LFCID 1464. This allows you to
verify that LFCID 1360 is the one you want to rollback to. If it is
not, enter abort. Determine the proper LFCID and start the
production restore shell as described in step 3 on page 228.
9. If you are satisfied that LFCID 1360 is correct, enter commit.
You will see output similar to the following:
committing...
10. On the production server, mount the protected volumes. Enter the
following command:

fsck -fp -y /dev/rtestlv
The current volume is: /dev/testlv

J2_LOGREDO:log redo processing for /dev/testlv
Mounting /test...
230
Introduction to Disaster Recovery
12
Introduction on page 231
RecoverNow Disaster Recovery Overview on page 232
RecoverNow Disaster Recovery Management on page 233
Introduction
RecoverNow provides highly available data recovery and protection
through failover, resync, and failback operations on both local and remote
recovery servers, providing solutions for local failovers, as well as remote
failovers.
In a local failover operation, the production server fails over to a local
(on-site) recovery server, whereas in a remote failover operation, the
production server fails over to a remote (off-site) recovery server. Failing
over to a remote server is defined as Disaster Recovery, since it applies to
the situation where an entire site is lost due to a disaster such as a flood or
hurricane. Disaster recovery is also used for situations where failing over
to a local server is adequate for keeping data available and applications
running.
231
Chapter 12: Introduction to Disaster Recovery
RecoverNow Disaster Recovery Overview

The RecoverNow disaster recovery capability is the process of switching
the production server to a recovery server that is located either locally or
remotely from the production server. The failover operation could be due to
a system hardware or software failure, a scheduled maintenance period, or a
disaster at your production or local site.
RecoverNow Recovery Process on page 232
How RecoverNow Disaster Recovery Works on page 232
RecoverNow Recovery Process

The RecoverNow disaster recovery process consists of three operations:
failoverswapping from the production server to the recovery server
resync bring the production server data up to date with the recovery
server data
failbackswapping back to the production server from a local or a

remote recovery server.
A failover operation is typically followed by a resync and then a failback

operation, which is the process of returning the role of production server to
its original location after a system failover or a disaster has taken place.
These operations use remote mirror Backing Store File Container (BSFC)
replicas as the failover-production data. This requires only a
configuration change and no data movement, resulting in reducing the time
that is required to resync the volumes after switching sites.
Use of the statemap tracking mechanism increases the efficiency of the data
resync, as it minimizes the amount of data transfer that must take place. The
Production application continues to run during the resync operation.
How RecoverNow Disaster Recovery Works

The resync operation eliminates the need to perform a full volume resync
copy from your recovery server to the production server. This reduce the
time required to resume operations at your production site.
Since the failover process puts the volumes into a suspended state, changes
are tracked within a statemap. Assuming that recording these changes is
enabled, only the changed data is sent to the production site to synchronize
the volumes. This reduces the time required to complete the failback
operation.
232
RecoverNow Disaster Recovery Management

NOTE
Although the production application must be down during the

failover operation, this transition is usually very fast.
Failover Context Overview on page 233
Failover Context Specifications on page 234
How the Failover Context Works on page 234
rtdr Failover Command on page 235
Failover Context Overview

If the production server fails, you can temporarily run the production
systems on the recovery server or the replicated server through the failover
operation.
NOTE
You can choose to configure the recovery server or a replicated

server, if you have one in your configuration, as the failover server
where you temporarily run the production system.
This transition requires that the configuration of the failover operation is
specified in a failover context. You create a failover context to configure a
context that is specific to your failover operations.
The failover context provides RecoverNow with the configuration
information used to operate as if the original recovery or replicated server
was actually a production server, and the original production server, when it
comes back on-line, was actually the recovery or replicated server.
You set up the failover context in association with the primary context for
the production server. This failover context enables you to perform the
system failover and failback operations, including managing all the required
transitions from the Production environment to the recovery or replicated
server.
To accomplish this, RecoverNow uses the block-ordered replica maintained
on the recovery and replicated servers as the failover disk, on which it
brings up the production server.
233
Failover Context Specifications

To configure a context, specify:
Sizes, names, locations of all raw logical volumes (LVs)
Names of all special files associated with devices (access points to LVs)
Filesystem information
Host information (host name, host ID, and IP addresses
Information about relationships between a production volume and its

associated replica logical volume, as well as the designated production
server, and recovery server
All of this information is identified with a single Context ID
Failover Context and Primary Context Relationship

Although closely related to the primary context, a failover context has all of
its own attributes for handling a failover and needs its own content identifier
or Context ID.
A failover context is the same as its associated primary context, where all
relationships and flows are reversed. Many of the logical volumes used by
the primary context, such as LFCs, are shared with the failover context.
However, some of the volumes are not shared and therefore may be in
different locations depending on which server the production is operating
on. For example, on the production server during normal operations, or on
the recovery or replicated server during failover operations.
Setup a Failover Context

After you setup a failover context by using the rtdr command, all other
attributes used with the rtdr command can then take the primary Context
ID. This provides an environment in which it appears as if you are dealing
with one context, either in a primary or a failover mode. The underlying
configurations are really two different, tightly related, contexts. You only
need to remember the primary Context ID to failover, resync, and failback.
Failover Context Naming Conventions

A primary or a failover context can be up to three digits long (1 - 999). You
can assign a number as a primary Context ID, for example 100. You can
then add 1 to the ID to identify the failover Context ID; for example, 101.
How the Failover Context Works

The primary context has all the information for the servers in a
Production-recovery server configuration, as well as in a
production-recovery-replicated server configuration.
234
Similar to a primary context, the failover context has all the information for
all the servers in a configuration. However, the difference is that the
configuration settings the failover context contains are derived from the
primary context and that it shares several attributes with the primary
context.
NOTE
You must setup one Failover Context for each additional server in
a configuration where you need to create only one Primary
Context for your production server.
By default, in a replicated configuration (Production, Recovery,
Replicated), the recovery server becomes the failover server. In a failover
event, it becomes the production server. All three servers will still be
connected when a resync operation is active.
In the same replicated configuration, to change the failover server to the
replicated server, change the configuration associated with the failover
context by specifying the hostname of the replicated server in the following
command:
rtdr -C <Primary Context ID> -F <Failover Context ID> setup
rtdr Failover Command

This section describes the RecoverNow Disaster Recovery failover
command rtdr and its associated attributes.
Use the RecoverNow disaster recovery and failover rtdr command to:
Setup a failover context
Validate data integrity prior to a failover
Perform failover, resynchronize, and failback operations.
Syntax
rtdr -C <ID> -fhmnqv failover | resync | failback
rtdr -C <ID> -F <ID> [-s <hostname>] [-fhmnv] setup
-C Primary Context ID of Production, Recovery, Remote
Replica servers
-F Failover Context ID
-f Forced execution *
-h Help, prints usage
-m Man style help
-n No execution, just print commands
-q quiet, do not ask for confirmation
235
-s Select failover site server from multiple Recovery

Servers (default is the first replication hop's server).
<hostname> must be a configured SCRT/info/host HostName
attribute.
-v Verbose output
IMPORTANT
Use the -f attribute with the rtdr command with caution. If you
do not resync, this attribute forces a failback and leaves the
Production Volume Set (PVS) file systems and replica out of sync.
RecoverNow Disaster Recovery Operations

The previous section contains an overview of the RecoverNow Disaster
capability and its associated failover context and commands. In the event of
a failover, follow the procedures described in the following sections.
236
Failover Process Overview on page 237
Preparing Before a Failover on page 237
Before Performing Failover Operations on page 238
Failover Process Overview

The failover process involves:
1. Creating a primary context using the procedures described in the
Chapter 6 Configuring Replication Groups on page 131.
NOTE
If you have changed a primary context you must delete the

corresponding failover context and recreate the failover context.
setup
2. Creating a failover context using the rtdr command. This command

creates a failover context definition associated with the primary context.
Refer to Preparing Before a Failover on page 237.
3. Validating data integrity of the replica prior to a failover. If necessary,
perform a failover restore, as shown in Before Performing Failover
Operations on page 238.
4. Performing the following operations after you validate the replica:
Failover
Resync
Failback
Refer to Disaster Recovery Operations on page 241 for instructions
on how to perform these operations.
Preparing Before a Failover

To prepare for a potential system failure, you set up a failover context so
that your production server can switch operations to the designated recovery
or replicated server.
Setting up a Failover Context

You must create a failover context to be associated with a configured
primary context prior to a system failover. The Failover Context ID must be
unique on the associated servers. Refer to Failover Context Naming
Conventions on page 234 for information on valid Failover Context ID
names.
To create a failover context associated with the primary context:
On the production and target servers execute:
237
where:
<Primary Context ID> is the primary context on the production server,
and <Failover Context ID> is the failover context on the recovery server.
NOTE
If there is more than one target server and you do not want to use
the default for the Failover Server, then use the -s <Hostname>
option on the rtdr command to specify the Failover Server. Refer
to Syntax on page 235 for the rtdr command.
Before Performing Failover Operations

Do not perform a failover restore on a restore target that has not been
validated. After a system failover occurs, RecoverNow cannot rollback to a
point in time before the failover.
This section provides guidelines to execute before performing failover
operations:
Validating Data Integrity

Validating the data integrity of the replica is critical. Prior to performing the
failover operations, validate the data integrity of the replica on the recovery
server and restore it. To validate the data, first create a snapshot of the
replica and then analyze it with the application itself.
Verify Restore Point

On the recovery or replicated server execute:
1. Create the snapshot:
scrt_ra -C <Primary Context ID> -X
2. Mount filesystems:
3. Check the data.

4. Unmount filesystems:
5. Remove the snapshot:

238
Create a Virtual Restore Snapshot

Once you create a snapshot of the replica, you can analyze the integrity of
the data. If analysis indicates data corruption, remove the snapshot and use a
virtual restore to locate and validate an optimal restore point. A virtual
restore leaves a snapshot in place for analysis at the restored point in time.
1. To perform a virtual restore on the recovery server:
scrt_ra -C <Primary Context ID> -t|-D|-S
For example, to make a restore snapshot to October 27, 2008 at

17:21:57:
scrt_ra -C <Context ID> -D "10/27/08 17:21:57"
3. Check the data.


You can perform failover operations after you validate the replica.
239
240
Disaster Recovery Operations
13
Using VSP to Perform Failover and Failback on page 241
Using the Command line to Perform Failover and Failback on

page 252
Using VSP to Perform Failover and Failback

The following steps show how to perform a failover (planned or
unplanned) or failback procedure. Refer to the RecoverNow online help
for specific information for each dialog. The following sections describe:
Run a Procedure on page 241
Run Planned Failover Procedure on page 242
Run Unplanned Failover Procedure on page 245
Run Failback Procedure on page 249
Run a Procedure
The Procedures portlet and Steps portlet on the Procedures page will
guide you through each step.
1. Use one of the following methods to run a procedure. Use the method
that is most convenient for you based on the page you are currently
on:
From the Replication Group portlet, select Planned Failover,
Unplanned Failover, or Failback from the Actions dropdown for a
replication group.
From the Procedures portlet, select Run from the Actions
dropdown for a specific replication group.
From the Procedures portlet, select the Procedure name (Planned
Failover, Unplanned Failover, or Failback) and select Run from
the Action toolbar on the Steps portlet.
241
Chapter 13: Disaster Recovery Operations
2. Use one of the following methods to run the next step in the procedure
or retry a failed step:
From the Replication Group portlet, select Resume Procedure from
the Actions dropdown for a specific replication group.
From the Procedures portlet, select Resume from the Actions
dropdown for a specific procedure and replication group.
From the Steps portlet, select Resume from the Action toolbar.
Run Planned Failover Procedure

Use the planned failover procedure to switch the roles of the production and
failover server when both production and recovery servers are available.
Refer to the RecoverNow online help for details for each portlet and dialog.
NOTE
Before failing over, all applications using the logical volumes

must be stopped. You may want to validate that your applications
run properly on the failover server by using a snapshot and
running your applications with the snapshot.
242
The Steps portlet shows the steps for the Planned Failover procedure
selected in the Procedures portlet.
The following table shows what steps are run to perform a planned failover.
The Sequence Number shows the sequence number of the step in the
procedure. The sequence begins with integers starting at 10 and
incremented by 10. The purpose of sequence numbers is to help identify
problem steps when communicating with customer care.
243
Table 2.
Run Planned Failover
Sequence
Number
Step
Dialog for this step
Command for this step
10
Unmount the file

systems on the
current production
server
Run Planned Failover

Procedure dialog: Unmount file
systems on current production
server on page 244
rtumnt -C <Context id>
20
Stop replication on
the current
production server
Resume Planned Failover

Procedure dialog: Stop
replication on current
production server on
page 245
rtstop -C <Context id> -F -S
30
Failover the
replication group.
Server roles change

dialog: Failover replication
group: Server roles change on
page 245
rtdr -C <Context id> -q failover
40
Start replication on
the new replicated
server (Only
displayed if the
replication group has
a replicated server
configured.)

dialog: Start replication on new
replicated server on page 245
rtdr -C <Context id> -q resync
50
the new recovery
server

recovery server on page 245
60
the new production
server

production server on
page 245
NOTE
When the procedure completes, and you are ready to move

production back to the configured production server, run the
Failback procedure.
Run Planned Failover Procedure dialog: Unmount file systems on

current production server
This step unmounts the file systems on the current production server. You
requested to move production to the failover server. Before running this
procedure, all applications using the logical volumes in this replication
group must be stopped. Applications are not available until this procedure
completes.
244
Resume Planned Failover Procedure dialog: Stop replication on

This step stops replication on the current production server. Replication
stops after all data in the backlog has been replicated. After replication stops
and before you continue, it is recommended that you create a snapshot on
the failover server and validate that your applications run properly.
Resume Planned Failover dialog: Failover replication group:

Server roles change
This step changes the roles of the servers in the replication group. Data is
not replicated and failback is not available until you complete the remaining
steps in this procedure.
Resume Planned Failover dialog: Start replication on new

replicated server
This step starts replication processes on the replicated server. Data is not
replicated and failback is not available until you complete the remaining
Resume Planned Failover dialog: Start replication on new recovery

server
This step starts replication processes on the recovery server. Data is not
NOTE
If a replicated server is not configured for this replication group,

and this is the last step in the procedure, when this step completes,
applications can be started on the production server.
Resume Planned Failover dialog: Start replication on new

production server
This steps starts replication processes on the new production server. Data
can start replicating. When the step completes, and you are ready to move
production back to the configured production server, run the Failback
procedure. You can also start your applications.
Run Unplanned Failover Procedure

An unplanned failover could be due to a hardware or software failure or
power outage with the production server. Because the production server is
not available, you may need to fail over to the failover server. This
procedure guides you through the failover, including validating the rollback
location, using a snapshot. Refer to the RecoverNow online help for details
for each portlet and dialog.
245
The Steps portlet shows the steps for the Unplanned Failover procedure
selected in the Procedures portlet.
The following table shows what steps are run to perform an unplanned
failover. The Sequence Number shows the sequence number of the step in
the procedure. The sequence begins with integers starting at 10 and
246
incremented by 10. The purpose of sequence numbers are to help identify

problem steps when communicating with customer care.
Table 3.
Run Unplanned Failover
Sequence
number
Step
10
Create a snapshot
on failover server.
Run Unplanned Failover Procedure

dialog: Create snapshot on failover
server on page 248
scrt_ra -C <Context id> -S

timestamp (if by Point in
Time or Event Marker)
or
scrt_ra -C <Context id> -t
Container ID (if by
Container ID ), followed by
rtmnt -C <Context id> -f
20
Delete a snapshot
on failover server.
Resume Unplanned Failover Procedure

dialog: Delete snapshot on failover
server on page 248
scrt_ra -C <Context id> -W
30
Rollback the
failover server
Resume Unplanned Failover Procedure

dialog: Rollback failover server on
page 248
scrt_rc -C <Context id> -S

timestamp (if by Point in
Time or Event Marker)
or
scrt_ra -C <Context id> -t
Container ID (if by
Container ID )
40
Failover the
replication group.
Server roles
change.
Resume Unplanned Failover dialog:

Failover replication group: Server
roles change on page 248
rtdr -C <Context id> -q failover
50
the new replicated
server.(Only
displayed if the
replication group
has a replicated
server configured.)

Start replication on new replicated
server on page 249
rtdr -C <Context id> -q

resync
60
the new recovery
server.

Start replication on new recovery server
on page 249

resync
70
the new production
server
Resume Planned Failover dialog: Start

replication on new production server on
page 245

resync
247
Run Unplanned Failover Procedure dialog: Create snapshot on

failover server
Because the production server is not available, you may need to rollback the
failover server before failing over. Validating the rollback location using a
snapshot is recommended. To create a snapshot:
1. Specify a location in the Location in failover server rollback window.
2. Click OK, to confirm the snapshot location and to the start the
unplanned failover.
3. Run your applications with the snapshot to validate the rollback
location.
4. You may need to use more than one snapshot to find a valid location.
Resume Unplanned Failover Procedure dialog: Delete snapshot on

failover server
This step deletes the snapshot on the failover server that was used to
validate the rollback location. All applications using the snapshot must be
stopped. File systems are unmounted when the snapshot is deleted.
You can decide whether or not you have a validated rollback location and
how to proceed. If the you decide to create another snapshot, the procedure
returns to the Create snapshot step and waits for you to resume the
procedure. The dialog for this step is the Resume - Create Snapshot dialog
and not the Run Procedure dialog.
In the section, Do you have a validated rollback location?
You can
Select Yes. Continue to next step. If selected, click OK to delete the
snapshot and go to the next step in the procedure. Note, that you will
need to use the Resume action.
Select No. Return to previous step and create another snapshot with
a different rollback location. Note, that you will need to use the Run
action.
Resume Unplanned Failover Procedure dialog: Rollback failover

server
This step will rollback the failover server to the location you specify. If you
do not want to rollback the failover server, select Do Not Rollback from the
Location to rollback failover server field.
Resume Unplanned Failover dialog: Failover replication group:

Server roles change
This step changes the roles of the servers in the replication group. Data is
not replicated and failback is not available until you complete the remaining
248
steps in this procedure. Applications can be started when this step

completes.
Resume Unplanned Failover dialog: Start replication on new

replicated server
This step starts replication processes on the replicated server. Data is not
NOTE
The servers in the replication group name have changed now that
the replication group has failed over.

recovery server
This step starts replication processes on the recovery server. Data is not
NOTE
If a replicated server is not configured for this replication group,

and this is the last step in the procedure, when this step completes,
applications can be started on the production server.

production server
This steps starts replication processes on the new production server. Data
can start replicating. When the step completes, and you are ready to move
production back to the configured production server, run the Failback
procedure. You can also start your applications.
Run Failback Procedure

The failback procedure is used to move production back to the configured
production server. The failback procedure is only available when the
replication group is in a failed over state.
249
The Steps portlet shows the steps for the Failback procedure selected in the
Procedures portlet.
The following table shows what steps are run to perform a failback. The
Sequence Number shows the sequence number of the step in the procedure.
The sequence begins with integers starting at 10 and incremented by 10.
The purpose of sequence numbers are to help identify problem steps when
250
communicating with customer care. Refer to the RecoverNow online help

for details for each portlet and dialog.
Table 4.
Failback
Sequence
number
Step
10
Unmount the file systems

on the current production
server
Run Failback Procedure

dialog: Unmount file systems
on current production server
rtumnt -C <Context id>
on page 251
20
Stop replication on the

Resume Failback Procedure

dialog: Stop replication on
current production server on
rtstop -C <Context id> -F -S
page 251
30
Failback the configured

production server

dialog: Failback configured
production server on page 252

failback
40

recovery server.

recovery server on page 252

failback
50

replicated server.(Only
displayed if the
replication group has a
replicated server
configured.)

replicated server on page 252

failback
Run Failback Procedure dialog: Unmount file systems on current

production server
This step unmounts the file systems on the current production server. You
requested to move production to the failover server. Before running this
procedure, all applications using the logical volumes in this replication
group must be stopped. Applications are not available until this procedure
completes.
Resume Failback Procedure dialog: Stop replication on current

production server
This step stops replication on the current production server. Replication
stops after all data in the backlog has been replicated. After replication stops
and before you continue, it is recommended that you create a snapshot on
the configured production server and validate that your applications run
properly.
251
Resume Failback Procedure dialog: Failback configured

production server
This step returns the production server role to the configured production
server and starts the replication processes. Data is not replicated until the
remaining steps in this procedure are completed.
NOTE
After this step completes, applications can be started on the

configured production server.
Resume Failback Procedure dialog: Failback configured recovery

server
This step returns the recovery server role to the configured recovery server
and starts the replication processes. Data can replicate from the production
server to the recovery server.
Resume Failback Procedure dialog: Failback configured replicated

server
This step will failback the configured replicated server from whatever role it
is currently in to the replicated server role. Data can replicate from the
recovery server to the replicated server.
Using the Command line to Perform Failover and Failback

In the event of a failover, follow the procedures described in the following
sections.
252
Failover Operations

The failover process involves:
1. Creating a primary context using the procedures described in the
Chapter 6 Configuring Replication Groups on page 131.
NOTE
If you have changed a primary context you must delete the

corresponding failover context and recreate the failover context.
setup
2. Creating a failover context using the rtdr command. This command

creates a failover context definition associated with the primary context.
Refer to Preparing Before a Failover on page 253.
3. Validating data integrity of the replica prior to a failover. If necessary,
perform a failover restore, as shown in Before Performing Failover
Operations on page 254.
4. Performing the following operations after you validate the replica:
Failover
Resync
Failback
Refer to Failover Operations on page 255 for instructions on how to
perform these three operations.

To prepare for a potential system failure, you set up a failover context so
that your production server can switch operations to the designated recovery
or replicated server.
Setting up a Failover Context

You must create a failover context to be associated with a configured
primary context prior to a system failover. The Failover Context ID must be
unique on the associated servers. Refer to Failover Context Naming
Conventions on page 234 for information on valid Failover Context ID
names.
To create a failover context associated with the primary context:
On the production and target servers execute:
253
where:
<Primary Context ID> is the primary context on the production server,
and <Failover Context ID> is the failover context on the recovery server.
NOTE
If there is more than one target server and you do not want to use
the default for the Failover Server, then use the -s <Hostname>
option on the rtdr command to specify the Failover Server. Refer
to rtdr on page 264 for the rtdr command.

Do not perform a failover restore on a restore target that has not been
validated. After a system failover occurs, RecoverNow cannot rollback to a
point in time before the failover.
This section provides guidelines to execute before performing your failover
operations:

server and restore it. To validate the data, first create a snapshot of the
replica and then analyze it with the application itself.
Verify Restore Point

On the recovery or replicated server execute:
1. Create the snapshot:
scrt_ra -C <Primary Context ID> -X
3. Check the data.


254
Create a Virtual Restore Snapshot

Once you create a snapshot of the replica, you can analyze the integrity of
the data. If analysis indicates data corruption, remove the snapshot and use a
virtual restore to locate and validate an optimal restore point. A virtual
restore leaves a snapshot in place for analysis at the restored point in time.
1. To perform a virtual restore on the recovery server:
scrt_ra -C <Primary Context ID> -t|-D|-S
For example, to make a restore snapshot to October 27, 2008 at

17:21:57:
scrt_ra -C <Context ID> -D "10/27/08 17:21:57"
3. Check the data.


You can perform failover operations after you validate the replica.
Failover Operations
This section describes how to perform the failover operations from the
production server to the recovery or replicated server when the production
server has failed.
Performing a Failover Restore on page 256
Performing Resync on page 256
Performing Failback on page 258
Manual Resynchronization Process if production server Data Is Lost

on page 258
IMPORTANT
Do not perform a failover restore on an invalidated restore target.

After validating the replica, the failover procedures are failover,
resync, and failback.
255
Performing a Failover Restore

You can now use a failover restore to roll the data back. You can:
rollback to the Actual Data Replica on page 256
Failover to the Latest Point in the Data on page 256
rollback to the Actual Data Replica

To rollback to the actual data replica, not a snapshot of the replica, to a
validated point in time and then failover:
scrt_ra -C <Primary Context ID> -F [-t | -D | -S]
NOTE
Do not do execute scrt_ra -X -F if you want to failover to the

latest point in time.
For example, to restore the replica to a previously validated target time
(1169584788 seconds from Jan. 1, 1970):
scrt_ra -C <Primary Context ID> -F -S 1169584788
Failover to the Latest Point in the Data

To failover to the recovery server, on the recovery server execute:
rtdr -C <Primary Context ID> failover
NOTE
This stops the aba for the primary context.
Start the Application

After performing the failover restore procedure, start the application on the
recovery or replicated server. At this point, the application operates in the
production environment until the failback operation takes place. Next, the
resynch operation tracks all data modification and sends them back to the
the original production server.
Performing Resync
A resync operation is required when the Production Volumes and Recovery
Replica Volumes diverge. This occurs after a failure of the production
server and a failover to the recovery or replicated server. When the
application is started on the recovery or replicated server the updates result
in a divergence from the data on the production server.
256
After restoring the original production server, use the resync operation to
ensure that the production server data is current with the recovery server
data.
To resynchronize the revived production server from the recovery or
replicated server, on all servers (Production, Recovery, and Replicated):
rtdr -qC <Primary Context ID> resync
This command takes the following actions:
On the replicated server:

start aba for failover context
On the production server:

start aba for failover context
start lca for failover context if there is a replicated
server
On recovery server:
start lca for failover context
IMPORTANT
The resync operation assumes the original production data was not
lost, and is available in its entirety after the production server is
revived. If the production data is lost, the statemap on the recovery
or replicated server must be marked as dirty prior to resync. This
forces a complete region recovery and initializes the production
data.
Resync with Region Recovery

This procedure marks statemap bitmaps as dirty for all devices in the
failover context.
On the recovery or replicated server, which is now the acting production
server, execute:
rtstop -C <Failover Context ID> -F
scconfig -C <Failover Context ID> -M
rtstart -C <Failover Context ID> -M
Wait for the region recovery completes before performing the failback
procedure, as described in Performing Failback on page 258.
257
NOTE
Performing a complete region recovery should be avoided since

this will require production down time and significant network
resources.
You can mark only specific state maps dirty, using the -L option of the
scconfig command. Refer to the RecoverNow Chapter 14 CLI
Commands on page 261for more information.
Performing Failback
IMPORTANT
Before you failback ensure that you stop the application on the
recovery server.
After a resync operation completes, failback to return to the original
production, recovery, and replicated server roles. All servers must be active
and running to execute resync and failback operations.
To initiate failback to the original production server, on the Recovery or
Replicated and production servers execute:
rtdr -qC <Primary Context ID> failback
Manual Resynchronization Process if production server Data Is

Lost
If the failover from the production server to the recovery server was
necessary because all protected data was lost on the production server, then
the data must be restored to the production server before doing a failback.
This data loss could occur if a disk or disk subsystem failed or was
destroyed on the production server.
The data can be restored to the production server from the recovery or the
replicated server using the following procedure after the necessary volume
groups and logical volumes have been recreated.
1. On the recovery or replicated server stop the application.
2. On the production server execute the following command which will
start scrt_aba:
rtdr -qC <Failover Context> resync
3. On the recovery or replicated server execute the following command

which will unmount any filesystems if the context has filesystems and
unload the drivers.
rtstop -FC <failover context>
258
4. On the recovery or replicated server execute the following command

which will mark the state maps dirty.
scconfig -MC <failover context>
5. On the recovery server execute the following command which will

synchronize the data to the production server.
rtdr -qC <Failover Context ID> resync
6. On the recovery server execute the following command which will

show the state maps.
scconfig -PC <failover context>
7. If the state maps are not clean on the recovery server wait until all the
data is synchronized to the production server.
create a snapshot to allow the integrity of the data to be checked.
scrt_ra -XC <failover context>
9. On the production server execute the following command if the context

has filesystems. This command will mount the filesystems.
rtmnt -C <failover context>
10. On the production server verify the validity of the data.

unmount any filesystems which were mounted in step 9.
rtumnt -C <failover context>
remove the snapshot created in step 8.
scrt_ra -WC <failover context>
13. On the recovery and replicated server execute the following command
which will failback to the primary context.
rtdr -qC <Failover Context ID> failback
failback to the primary context.
rtdr -qC <Failover Context ID> failback
259
260
CLI Commands
14
RecoverNow provides a comprehensive list of Command Line Interface

(CLI) commands:
extend_replica_lv on page 262
rtattr on page 263
rtdr on page 264
rtmark on page 268
rtmnt on page 268
rtstart on page 269
rtstop on page 270
rtumnt on page 271
sclist on page 271
scconfig on page 274
scsetup on page 276
scrt_ra on page 278
scrt_rc on page 279
scrt_vfb on page 281
sccfgd_cron_schedule on page 282
sccfgd_putcfg on page 283
sccfgchk on page 283
sztool on page 284
261
Chapter 14: CLI Commands
extend_replica_lv
Usage
You can use the extend_replica_lv command to force the expansion of a
Replica LV (Logical Volume) that is associated with a specified PVS
(Production Volume Set) LV, so that the Replica LV will be equal in size to
the PVS LV. This command will only run on the production server and the
LCA must be active.
Syntax
extend_replica_lv -C <Context ID> -L <PVS LV>
extend_replica_lv -h help
-C <Context ID>
-L <PVS LV>
-h Help, prints this usage
NOTE
This command is only required for PVS LVs that are extended and
have no associated filesystem, or PVS LVs that have an associated
filesystem with an outline log and the filesystem is extended.
262
rtattr
rtattr
Syntax
rtattr -C ID [-a attribute] [-o object] [-t type]
rtattr -C ID -a attribute -v value {-o object | -t type}
rtattr -h
-a Attribute for query/edit (ObjectAttributeName)
-C <Context ID>
-h Help, prints this usage
-o Object for query/edit (ObjectName)
-t Type for query/edit (ObjectType)
-v Value for edit (ObjectAttributeValue)
You can use the v parameter with the commands to edit. If you do not
specify the v parameter only query is available.
Usage
Use this command to query and change attributes in the RecoverNow ODM
(Object Data Manager) files:
SCCuAttr
SCCuObj
SCCuRel
Example 1
View all the machine hostids:
rtattr -C <Context ID> -a HostId
SCCuAttr:
ObjectName = "backup"
ConfigObjectSerial = 4
ObjectType = "SCRT/info/host"
ObjectAttributeName = "HostId"
ObjectAttributeValue = "0xc0a801f7"
ObjectAttributeType = "ulong"
SerialNumber = 4006
ObjectNlsIndex = 0
SC_reserved = 0
ContextID = 1
SCCuAttr:
ObjectName = "replica"
263
SerialNumber = 8006
ObjectNlsIndex = 0
SC_reserved = 0
ContextID = 1
SCCuAttr:
ObjectName = "production"
SerialNumber = 16006
ObjectNlsIndex = 0
SC_reserved = 0
ContextID = 1
Example 2
View only the production servers hostid:
rtattr -C <Context ID> -o production -a HostId
SCCuAttr:
ObjectNlsIndex = 0
SC_reserved = 0
ContextID = 1
rtdr
Syntax
rtdr -C <ID> [-fhmnqv] failover | resync | failback
rtdr -C <ID> -F <ID> [-s <hostname>] [-fhmnv] setup
-C Context ID (of the "primary" context)
-F Failover Context ID
-f Forced execution (use with caution)
-h Help, prints usage
-m Man style help
-n No execution, just print commands
-q quiet, do not ask for confirmation
-s Select failover site server from multiple recovery
servers (default is first replication hop's server.)
264
rtdr
<hostname> must be a configured SCRT/info/host HostName

attribute.
-v Verbose output
NOTE
The f option prompts for confirmation unless combined with q.
Usage
This command manages RecoverNow's disaster recovery processes as well
as failover and failback operations. Given a primary context <X>
configured on both a Production and a Recovery Server, note:
To create an associated failover context <Y>, on both the production

and recovery servers execute:
rtdr -C <X> -F <Y> setup
To failover to the Recovery server, on the recovery server execute:

rtdr -C <X> failover
To re-synchronize the revived production server, on both the production

and recovery servers execute:
rtdr -C <X> resync
To failback to the production server, first on the recovery server, and

then on the production server execute:
rtdr -C <X> failback
To select and configure from one of many recovery servers (a cascaded

replication configuration) as the target machine for all subsequent
failover, resync and failback operations execute:
rtdr -C <X> -F <Y> -s <hostname> setup
NOTE
A failover context associated with a configured primary context

must be created and setup prior to executing a failover. The
failover Context ID is arbitrary, but must be unique on the
associated servers.
To create and setup a failover context, on both the production and

recovery servers:
rtdr -C <X> -F <Y> setup
265
Prior to failover, you should validate the data integrity of the Replica,
and if necessary, validate the data if necessary.
To validate data integrity of the Replica, create a snapshot of the
replica, then analyze it with the application itself. To create the
snapshot, on the recovery server:
scrt_ra -C <X> -X
If your analysis indicates data corruption, use a virtual restore to locate

and validate an optimal restore point. A virtual restore leaves a snapshot
in place for analysis at the restored point in time. To perform a virtual
restore, on the recovery server:
scrt_ra -C <X> [-D | -S | -t]
Given a corrupt replica, and a validated restore point in time, use a

failover restore to roll the actual data replica, not a snapshot of the
replica, back to the validated point in time. To perform a failover
restore, on the recovery server execute:
scrt_ra -C <X> -F [-D | -S | -t]
NOTE

After validating the replica, the disaster recovery procedure is
failover, resync, then failback.
After failover, start the application on the recovery server. It will be the
acting production environment until failback. All data modifications
will be tracked and shipped back to the original production server by
resync.
After reviving the original production server, use resync to bring the
production server data up to date with the recovery server data.
After the resync completes, use failback to return to the original
production and recovery server roles. Both the production and recovery
servers must be live to execute resync and failback.
NOTE
Resync assumes the original production data was not lost, and is
available in its entirety after the production server is revived. In
the event that production data was lost, statemap on the recovery
server must be dirtied prior to resync to force a complete region
recovery, and re-initialize the production data.
266
rtdr
To dirty all statemaps, in the failover context on the recovery server (the
acting production server):
rtstop -C <X> -F
scconfig -C <X> -M
NOTE

After validating the replica, the disaster recovery procedure is to
failover, resync, then failback.
After failover, start the application on the recovery server. It will be
the acting production environment until failback. All data
modifications will be tracked and shipped back to the original
production server by resync.
After reviving the original production server, use resync to bring the
production server data up to date with the recovery server data.
After resync completes, use failback to return to the original
production and recovery server roles. Both the production and
recovery servers must be live to execute resync and failback.
NOTE
Resync assumes the original production data was not lost, and is
available in its entirety after the production server is revived. In
the event that production data was lost, statemap on the recovery
server must be dirtied prior to resync to force a complete region
recovery, and re-initialize the production data.
To dirty all statemaps, in the failover context on the recovery server (the
acting production server):
rtstop -C <X> -F
scconfig -C <X> -M
rtstart -C <X>
After a system failover, RecoverNow cannot rollback to a point in time

before the failover. Likewise, after a system failback, RecoverNow
cannot rollback to a point in time before the failback. For this reason,
replica data integrity should be validated with a snapshot prior to
executing a failover.
267
rtmark
Syntax
rtmark [-C ] [-s <num>|-d <str>] [-iV] [<file>|-]
rtmark -rC <Context ID>
rtmark -h
-C ID Event is specific to Context ID.
-d <str> Date string, overrides event time.
-h Help, display this message.
-i Interactive query for event attributes.
-r Copies event marks from the production server to the
Recovery Server
-s <num> Seconds since epoch, overrides event time.
-V Print version.
<file> File containing the event mark attributes.
Usage
Event Markers are tags that mark points in time or points in process that are
significant to you for the purposes of recovery. An Event Marker can be
selected as the Recovery Point Objective (RPO) during a data restore. They
are typically needed for applications which cannot take advantage of
RecoverNows Any Point-In-Time (APIT) data restores along with
applications which do not have live transactional durability on disk.
The following is an example of a script that could be called, with as many
arbitrary attributes to the event that you want, in addition to the time and
date attribute automatically assigned by rtmark. The customer-defined
attributes between the cat line and the second EOF would also be added to
the event. The entire event would be replicated to the recovery server, and
available for viewing and selection during restores.
#!/usr/bin/ksh
cat <<-EOF | /usr/scrt/bin/rtmark -C <Context ID> name = test1
description = "This is a test."
owner = dave
priority = 2
another_attribute = Just another attribute
EOF
rtmnt
Syntax
rtmnt [-C ID][-fn]
268
rtstart
Parameters
Parameter
Description
-C ID
Context ID. It must be specified if more than one context is

defined.
-f
Perform fsck of file systems before mounting
-h
Display help message
-n
Do not mount file systems. Give present status.
-p
Halt on errors instead of undoing (partial)
Usage
This command is used to mount all file systems associated with the context
specified.
See Also
rtumnt, sccfgd_putcfg
rtstart
Syntax
rtstart [-C ID][-BMnNp]
Parameters
Parameter
Description
-C ID

defined.
-B
Boot/Autostart - used to ONLY during system startup
-h
Displays help message
-n
Does not actually start anything just to give present status.
-M
(production server only) Disable, mount, and perform fsck of

production volume set (PVS) file systems
-N
(production server only) Disable and perform fsck of production

volume set file systems
-p
Halts on errors instead of undoing (partial)
269
Usage
This command is used to load the RecoverNow data tap and to start the data
replication processes. On the production server, rtstart also mounts the
protected file systems.
See Also
rtstop
rtstop
Syntax
rtstop [-C ID][-FfknS]
Parameters
Parameter
Description
-C ID

defined.
-F
Forces an unload of drivers (use this command with caution).
-f
Forces unmount if the snapshot journals are full.
-h
-n
Does not actually stop anything just give present status
-k
Attempt to kill an agent that is not stopping normally
-S
(production server only) Force log creation agent (LCA) to

synchronize before stopping
Usage
This command is used to stop the data replication process of RecoverNow,
and optionally to unload the data tap.
See Also
rtstart on page 269
270
rtumnt
rtumnt
Syntax
rtumnt [-C ID][-Dfn]
Parameters
Parameter
Description
-C ID

defined.
-D
(production server only prevent log creation agent (LCA) from

synchronizing after unmounting.
-f
Forces unmount if the snapshot journals are full.
-h
Display help message.
-n
Do not unmount. Give present status.
Usage
This command is used to unmount all the volumes associated with the
specified context.
See Also
rtmnt on page 268 and sccfgd_putcfg on page 283
sclist
Syntax
sclist -t TYPE [-bR] [-A ATTR [ ... ]] [-R] [-C ID] [-d X]
sclist -t TYPE -o ATTR=VALUE [-bR] [-A ATTR [ ... ]] [-C ID]
[-d X]
sclist -a [-A ATTR [ ... ]] [-C ID] [-d X]
sclist -r SERIAL [-r SERIAL ...] [-b | -c] [-C ID] [-d X]
sclist -O SERIAL [-O SERIAL ...] [-C ID] [-d X]
sclist [-BeiIjJlLmMpPstSTvVX] [-I] [-D[D]] [-C ID] [-d X]
sclist -h [-z]
sclist -fZ
-a Query all objects
-A ATTR Query specific attribute (repeatable)
-b Be Brief, useful for scripting output
-B List of StateMap bitmap devices
-c Expansive, if possible, expand on output.
-C ID Operate on Context ID.
271
-d X Debug level of X (0-9).

-D Query all driver/device objects from ODM.
(-D for just drivers, and -DD for devices & drivers.)
-e List pooling/journal configuration.
-f File system list.
-h Print Help Message
-i Query driver objects
-I Query whether drivers are loaded
-j Query the Journal objects
-J List of History Journal exported devices
-l Query the Logdev objects
-L List of LogDev access devices
-m Query the StateMap objects
-M List of StateMap access devices
-o ATTR=VAL Query within type list for attribute ATTR equal
to VAL.
-O SERIAL Query specific object with serial number SERIAL
-p Query the Passthru objects
-P List of PassThru access devices
-r SERIAL List given objects relationships.
-R List relationships in attribute list.
-s Query the SCID objects
-S List of SCID devices
-v List of Volume group/disk info for config
-V List of Write Journal exported devices
-t TYPE What objects to query ('sclist -hz' for list)
-X List of SCID exported devices
-z Print all known classes/object types.
-Z Print device names for current machine.
272
Parameter
Description
-a
Query all objects
-A attribute
Query specific attribute (repeatable)
-b
Be brief; useful for scripting output
-B
List of statemap bitmap devices
-C ID
Operate on Context ID
-d
Query all driver objects
-D
Query all driver objects and device objects
-E
Query the pooling status
-f
File system list
-h
Print help message
-i
Query driver objects
sclist
Parameter
Description
-I
Query whether drivers are loaded
-j
Query the journal objects
-J
List of history journal exported devices
-l
Query the Logdev objects
-L
List of LogDev access devices
-m
Query the StateMap objects
-M
List of statemap access devices
-o ATTR=val
Query within type list for attribute ATTR equal to val
-O serial
Query specific object with serial number serial
-p
Query the Passthru objects
-P
List of PassThru access devices
-r serial
List relationships of object with specified serial number
-R
List relationships in attribute list
-s
Query the SCID objects
-S
List SCID devices
-v
List volume group information for configuration
-V
List write journal exported devices
-t type
Type of object to query (enter sclist hz for list of object

types)
-X
List of SCID exported devices
-z
Print all known object types
-Z
Print device names for current machine
Usage
This command provides information about containers used in RecoverNow.
See Also
273
scconfig
Usage
Use this command to manage DataTap devices and drivers.
Syntax
scconfig
scconfig
scconfig
scconfig
scconfig
scconfig
scconfig
scconfig
scconfig
274
-l
-u
-r
-M
-S
-s
-t
-C
-V
[-cfinERtv] [-C ID] [-d X] [-I name]

| -U [-finEv] [-C ID] [-d X] [-I name]
[-nv] [-L name ...] [-C ID] [-d X] [-I name]
| -W | -P | -B [-nv] [-L name ...] [-C ID] [-d X]
[-C ID] [-d X] [-nv]
[-C ID] [-d X]
| -q | -Q | -h
ID -a seconds [-b percent]
Parameter
Description
-a seconds
Send Partial Container Automatically. Frequency to check in

seconds (0-86400).
-b percent
Send Partial Container Automatically. Percent filled (0-99).
-c
Send Partial Container Automatically display current values.
-C ID
Operate on Context ID.
-d X
Debug level of X (0-9).
-E
Exported, defers device export to scsetup.
-f
Create the filter special devices.
-h
Print help message.
-i
Ignore errors and continue.
-I name
Operate on named context instance {prod | backup}. Only

useful in single system configuration.
-l
Load/configure Double-Take RecoverNow kernel drivers and

devices
-L name
Select a device for operations. Use sclist Z to list names

of the devices.
-M
Mark statemap bitmaps dirty for all or specified devs (see -L).
-n
Don't execute, just echo commands.
-P
Print statemap bitmaps for all or specified devices.
scconfig
Parameter
Description
-B
Resync data for all or specified devices.

Warning: Keep in mind the following while scconfig -B is
running:
Do not cycle RecoverNow and the server.
Ensure that you do not lose your terminal session (nohup
scconfig -B &).
-Q
Query kernel statemaps for system synchronization. Returns

"true" if replica is synchronized with production, "false" if not.
-q
Query for a list of installed contexts.
-r
Replay statemap logs ONLY.
-R
Skip replay of statemap logs during log.
-s
Status of kernel devices.
-S
Send Partial Container.
-t
Trace kernel extensions (using this option twice enables

tracing on all kernel drivers).
-u
Unload/unconfigure Double-Take RecoverNow kernel

devices.
-U
Unload Double-Take RecoverNow kernel shared drivers. All

other local contexts must be unconfigured (see -u).
-v
Be verbose or display version when only option.
-V
Display version.
-W
Wipe statemap bitmaps clean for all devices or the specified

devices (see -L).
NOTE
The functionality to send partial containers automatically is also

provided by the Replication Group Configuration Wizard on the
Replication Group Container Options panels page 145 and
page 157 in the section Send partially filled containers
automatically. For additional details, refer to Send partially
filled containers automatically on page 146.
See Also
275
scsetup
Makes or removes the Logical Volumes (LVs) used by RecoverNow in a
specific protection context, such as LFCs. Note however that scsetup will
not remove production LVs in the PVS or their associated replica LVs. Run
this command after defining and saving a context configuration using the
RecoverNow Replication Group Wizard.
After you have defined a context, scsetup creates a log file and containers
(logical volumes) in the specified volume group.
Syntax
scsetup -M [-ijlnprsv] [-C ID] [-d X] [-o role] [ -t TYPE ]
scsetup -R [-inv] [-C ID] [-d X] [-o role] [ -t TYPE ]
scsetup -E [-cinv] [-C ID] [-d X] [-o role]
scsetup -I [-cinv] [-C ID] [-d X] [-o role]
scsetup -L [-inv] [-C ID] [-d X]
scsetup -X [-inv] [-C ID] [-d X]
scsetup -F [-inv] [-C ID] [-d X]
scsetup -h
-C ID Operate on Context ID.
-c Clear destination device files prior to export/import.
-d X Debug level of X (0-9).
-E Export production volumes.
276
Parameter
Description
-c
Clean destination special device file before it is moved
-C ID
Operate on Context ID
-i
Initialize devices (fill with zeros)
-j
Initialize journal containers (fill with zeros)
-L
Perform a logform operation on the statemap log
-l
Skip forming statemap log (if one exists)
-m
Do not spread volume layout if disks are not specified
-M
Create new configured logical volumes
-n
Do not execute commands. Print commands.
-o
Override the volume group or disk group hints in the

configuration with the configured default volume group or disk
group
-R
Remove configured logical volumes (except PDFCs and

BSFCs). Any logical volume with true or replicated production
data will not be destroyed.
-s
Skip setting or clearing bitmaps for statemap (if there are any)
scsetup
Parameter
Description
-v
Verbose mode (using this option twice increases verbosity)
-X
Delete or clean up device special files from the /dev directory
Usage
Preparation for RecoverNow data protection.
-F Failover preparation. PDFC LV names are moved to BSFC LV
names, and vice versa.
-i Ignore volume manager errors.
-I Import production volumes.
Removes RecoverNow data protection, allowing direct access to production

data.
-j Initialize COW journal containers (fill with zeros).
-L Perform a logform on the statemap log
(implied with -M operation).
-l Skip forming statemap log (if one exists).
-M Make configured LV's from scratch.
-n Don't execute, just print commands.
-o role Operate on specific role { prod | back }.
Only necessary in a single server configuration.

-p Post processing during make (-M), skips volume creation.
-r Re-initialize devices (fill with zeros).
-R Remove configured LV's (all except PDFCs and BSFCs).
Any LV with true or replicated production data will not be destroyed.

-s Skip setting/clearing bitmaps for statemap
filters (if there are any).
-t TYPE An exclusive container type to operate on.
Multiple -t specifications are allowed.
NOTE
Type must be of SCRT/container/*, and specified as the associated

Class name (see sclist -hz for a list).
-v Verbose mode (using this option twice increases verbosity).

-X Delete/Clean-up device special files from /dev.
277
scrt_ra
Syntax
scrt_ra
scrt_ra
scrt_ra
scrt_ra
-t
-D
-S
-V
<>
<>
<>
<>
[-C
[-C
[-C
[-d
ID] [-V <>] [-d X] [-fFlLv]

ID] [-V <>] [-d X] [-fFlLv]
ID] [-V <>] [-d X] [-fFlLv]
X] [-aflLv]
Parameters
278
Parameter
Description
-a
Restore to alternate devices.
-C
Operate on context ID.
-d
Debug level of X (0-9).
-D <target date/time>
Restore Vdevs to target date/time.
-e
Print available Event Marks to restore.
-f
Force: Continue even if read journals are already

in place.
-F
Restore to raw BS devices (only for Failover

processing.)
-h
Display this message.
-l
List status of devices.
-L
Lock ABA for the duration of the restore.
-p
Print valid restore time spans.
-P <path to script>
Create Vdevs and run script.
-S <target seconds>
Restore Vdevs to target time (seconds since Jan 1,

1970).
-t <target LFC number>
Restore Vdevs to target LFC number.
-v
Verbose.
-V<vfb level>
Restore Vdevs to target vfb level. The Tivoli

Storage Manager must be defined in the
Double-Take RecoverNow configuration before
using this option.
-w
Wipe write journals (Vdevs).
-W
Wipe write and history journals (Vdevs &

snapshots).
scrt_rc
Parameter
Description
-x
Snap history journals (read only snapshots).
-X
Snap write journals (write snapshots, a.k.a.

Vdevs).
Usage
This command is the Restore Agent and is used to create snapshots on the
recovery server.
See Also
rtmnt on page 268, and rtumnt on page 271
scrt_rc
Syntax
scrt_rc [-C ID] [-d X] [-p X] [-h[v]] [-v] [-V]
-d Debug level of X (0-9)
-h Help, display this message
-C Operation on Context ID (default is 17)
-p Ping agent X (aba|lca|rs), ref is 0 if up
-v Verbose help
-V Print version
Usage
The restore client is an interactive command line interface, or shell, for
production data restore. To enter the shell, type scrt_rc -C<ID> at the unix
command prompt on the recovery (a.k.a. backup) server.
Entering the restore client shell starts a production restore session.
Ultimately, this session should be terminated with either the commit or
abort command. Problems during the restore can be resolved with the
recovery command.
Type help at the rc> prompt for all available commands.
NOTE
The -p option the scrt_rc command will not start the shell, but
instead will return with agent status.
279
Session restore targets

You can restore to three types of targets during a session with the restore
command. These targets are:
LFC level
Date/Time
Seconds since epoch (Jan 1, 1970)
Session Termination
A restore session may be terminated either with an abort or a commit
command. When aborted, all restored devices are brought back to
pre-session levels. When committed, all restored devices remain at the last
target of the session.
A commit does not remove any forward or reverse incremental data from
the RecoverNow time line which allows for a subsequent restore to a time
after the committed target, if necessary. In fact, the restore itself is included
in the time line which allows it to be undone.
Process Overview
RecoverNow performs a production data restore by writing reverse block
incremental data directly into the raw Logical Volumes (LVs) of the
Production Volume Set (PVS), rolling those LVs back in time as a single
consistency group. The PVS is treated as a consistency group since it
encapsulates the entire storage footprint of the protected application. The
application's referential data integrity is always maintained.
All block I/O during the restore occurs at the Logical Volume Manager
(LVM) layer, below all file systems and/or databases associated with the
protected application. In RecoverNow, the reverse block incremental data is
recorded in odd numbered LFCs, the Before Image Log File.
Containers (BILFCs), which are also raw LVs and reside on the
backup/recovery server, or in external tape archives, if any.
The length of the restore window is a function of how many BILFCs are
available to RecoverNow, the size of the BILFC, and the average
application write rate. Tape archives are used to extend the restore window.
During a restore, the PVS LVs must be opened exclusively for writing by
RecoverNow. No other application may have the LVs opened for writing.
All associated databases and file systems must be unmounted.
Two agent daemons work together to perform a production restore. On the
production server, the Log Creation Agent (LCA) receives BILFC
transmissions and makes the BILFC writes to the PVS. On the
280
scrt_vfb
backup/recovery server, the Restore Server (RS) handles the extraction of

LFCs from either the Assured Backup Agent (ABA) or the Archive Agent
(AA) if available, and the transmission of those BILFCs to LCA.
Prior to a production restore, LCA (scrt_lca) and ABA (scrt_aba) must be
running in daemon mode. If external tape archiving is enabled, AA (scrt_aa)
should also be running in daemon mode. RC (scrt_rc) will spawn RS
(scrt_rs) automatically, and stop RS automatically at the conclusion of the
restore.
General Procedure
1. Ensure required agent daemons are running.
2. On the production server - Stop/Unmount the corrupted production
application
3. On the production server - Unmount file systems [rtumnt -Cx]
4. On the production server - Sync RecoverNow [scconfig -Cx -S]
5. On the recovery server - Execute RC [scrt_rc -Cx]
6. On the production server - Mount file systems [rtmnt -Cx]
7. On the production server - Start/Mount production database
Procedure Notes
1. The rtumnt -Cx command will perform a switch [scconfig -Cx -S]
automatically.
2. One After Image Log File Container (AILFC) may be sent during the
restore to fine tune to the nearest second. BILFCs are optimized for I/O
throughput, while AILFCs maintain individual write fidelity.
3. Backup and recovery server are synonymous.
scrt_vfb
The Tivoli Storage Manager must be defined in the RecoverNow
configuration before using this command.
Syntax
scrt_vfb [-bdDflLnUVrR] [-s <path to validation script>] [ -C ID ]
Usage
This command is used to create a virtual full backup.
281
Parameters
Parameter
Description
-b
Backup from backing store devices.
-C
Operate on Context ID.
-D
Don't backup (useful with -d).
-f
Force use of existing snaps.
-h
Help.
-l
Leave snaps/VDevs up when completed.
-L
Lock ABA for entire backup.
-U
Unlock ABA (If VFB crashed leaving ABA frozen).
-R
Release read devices as they complete.
-s
Run validation script prior to backup.
-V
Create VDevs.
sccfgd_cron_schedule
This command manages entries in cron for RecoverNow Virtual Full
Backups (VFB). The Tivoli Storage Manager must be defined in the
RecoverNow configuration before using this command.
Syntax
sccfgd_cron_schedule <Op> <Context_id> [<sched_type>]
[<cron_info>] [<vfb_opts>]
where:
Op -[a|q|d] for (add|query|delete respectively)
sched_type [once|daily|weekly|monthly]
cron_info mm:hh:DoM:MoY:DoW (see man crontab)

where: mm (minutes) 0-59
hh (hours) 0-23
DoM (Day of Month) 1-31
MoY (Month of Year) 1-12
DoW (Day of Week) 0-6 (0=Sun,6=Sat)
vfb_opts
scrt_vfb command options:

[-bdDflLnUVrR] [-s <path to validation script>]
see scrt_vfb command documentation
282
sccfgd_putcfg
Usage
This command is used to schedule a virtual full backup.
Examples
sccfgd_cron_schedule add 3 daily 15:3:*:*:*
sccfgd_cron_schedule delete 3
sccfgd_cron_schedule query 3
sccfgd_putcfg
Syntax
sccfgd_putcfg primary_context_ID failover_context_ID
Parameters
Parameter
Description
primary_cont
ext_ID
Primary Context ID (an existing context that has been created

for normal operation)
failover_cont
ext_ID
Failover Context ID (a new context based on the primary

context that will be created for failovers)
-h
Usage
This command is used to load the RecoverNow configuration file into the
RecoverNow ODM by creating and loading a failover context configuration
based on a previously loaded primary context configuration.
sccfgchk
Syntax
sccfgchk-C <Context ID>
Parameters
Parameter
Description
-c class
Check only listed class (can repeat)

(Use 'sclist -hz' to list possible classes)
-C CTX
Run for this context
283
Parameter
Description
-d X
Debug level of X (0-9)
-h
Print this message
-i
Check production instance uniqueness only
Check file access and sizes
-v
Verbose
Usage
This command is used to check a configuration before RecoverNow is
started. Issue this command on each node after the configuration is
initialized and before it is started.
sztool
Syntax
sztool
Parameters
sztool script
Command
Options
284
Description
sztool
If issued for the very first time, the working directory,

diskinfo file and sztool.cfg file are generated. You should
review the diskinfo file and then modify sztool.cfg,
accordingly. You can then re-run sztool.
sztool -c
Assigns CDP_Window_Hours a value and run against

existing data log file. Only screen output. Example: sztool
-c24.
sztool -d
Generates the /tmp/sztool/diskinfo file. Performed

sztool -g
Generates the sztool.cfg file in /tmp/sztool. Performed

sztool -h
Prints out a help message.
sztool
sztool script
Command
Options
Description
sztool -l
When the log file is created, this command prints out the
calculation results for different LFC sizes based on the
existing log file. For example, sztool -l32 prints out the
results when the LFC size is at 32M. sztool -l16 -l512,
prints out all the calculation results from 32MB to
512MB. You cannot have spaces between -l and the LFC
size number. Only screen output, there is not any delay or
sleep.
sztool -r
Assigns Replication_Out_Hours a value and runs against

the existing data log file. Only screen output. For
example: sztool -r24, or sztool -r24 -l2 -l32.
Usage
You can use the Sizing Tool (sztool) to calculate configuration values before
RecoverNow is installed. It is also useful to run the tool after RecoverNow
is installed to determine if the number of LFCs or WJ percentage needs to
be adjusted. For more information, refer to Chapter 3, Using the Sizing
Tool to Calculate LFC Size on page 43.
285
286
Integration of PowerHA (HACMP)

with RecoverNow
Configuring a Highly Available RecoverNow/PowerHA Production

Server Environment on page 287
Configuring a Highly Available RecoverNow/PowerHA Recovery

Configuring a RecoverNow/PowerHA Production to Recovery

Configuring a Highly Available RecoverNow/PowerHA

Production Server Environment
A RecoverNow Highly Available (HA) production server configuration is
supported with PowerHA (HACMP). This section describes how you
configure RecoverNow in a HA production server Environment.
In this mode:
There are two production nodes with shared disks between them.
Replication is to a third node that lies outside the cluster.
A Resource Group can only be moved between the two production

nodes, and the RecoverNow roles of the nodes never changes.
RecoverNow Configuration Requirements

Review these guidelines before you configure RecoverNow:
The filesystems protected by RecoverNow should not be

automatically mounted at system restart.
Do not enable automatic startup for RecoverNow in a HA

environment.
RecoverNow software must be installed on both PowerHA

Production nodes and the Recovery node.
287
Chapter A:
All volume groups associated with a RecoverNow configuration must

be enhanced concurrent mode volume groups on both PowerHA
Production nodes. Use C-SPOC to create/change the volume group(s),
selecting Fast Disk Takeover or Disk Heart Beat for the Enable Fast
Disk Takeover or Concurrent Access option.
The Highly Available production server RecoverNow File Containers

and the RecoverNow configuration filesystems
/usr/scrt/run/c<Primary Context ID> and
/usr/scrt/run/c<Failover ContextID> can co-exist on the
same enhanced concurrent mode volume group(s) that are part of the
RecoverNow configuration. By default the RecoverNow File
Containers are created on the Default VG, defined using the
RecoverNow Replication Group Wizard, unless otherwise specified.
The filesystems need to be at least 128MB in size and must have a
separate jfslog, jfs2log, or jfs2 inline log since they will not be part of
the PVS.
RecoverNow must be configured with a Service IP address (an alias

address that follows the application when it is moved between nodes) so
that the recovery server can connect to either production server. To
configure a Service IP Address, change the production servers Initial
host adapter IP Address to the Service IP address used by your
application. If your application did not use a Service IP address, create
one. The /etc/hosts file on all nodes must contain the Service IP Address
and associated IP label.
Configuring Highly Available Production Servers

To configure Highly Available production servers:
1. On the primary production server and recovery server, varyonvg all
volume group(s) associated with the RecoverNow configuration.
2. On the primary production server, mount the filesystems associated
with the RecoverNow configuration.
c. mount /usr/scrt/run/c<Primary Context ID>
d. mount /usr/scrt/run/c<Failover ContextID>
3. Use the RecoverNow Replication Group Wizard to configure the
Primary Context.
Refer to the section Chapter 6, Configuring Replication Groups on
page 131.
4. On the primary production server, initialize the context.
scsetup -C <Primary Context ID> -M
288
Configuring a Highly Available RecoverNow/PowerHA Production Server Environment
5. On the primary production server, create the Failover Context ID.

setup
6. On the recovery server, initialize the context.

7. On the recovery server, create the Failover Context ID.

setup
NOTE
You must manually copy and load the RecoverNow configuration

onto the failover production server.
8. On the primary production server, create a file with the Primary Context
ID configuration.
odmget -q ContextID=<Primary Context ID> SCCuObj
SCCuAttr SCCuRel >/tmp/C<Primary Context ID>.cfg
Copy this configuration file to /tmp on the Failover Production

Server.
9. On the primary production server, create a file with the Failover
Context ID configuration.
odmget -q ContextID=<Failover Context ID> SCCuObj
SCCuAttr SCCuRel >/tmp/C<Failover Context ID>.cfg
Copy this configuration file to /tmp on the Failover Production

Server.
10. On the failover production server use rthostid to obtain its HostId.
/usr/scrt/bin/rthostid
11. On the failover production server edit the production HostId stanza in
the /tmp/C<Primary Context ID>.cfg file. Replace the contents
of the ObjectAttributeValue field with the output from the
"rthostid" command.
SCCuAttr:
ObjectAttributeValue = "6CABA7DF"
289
Chapter A:
ObjectNlsIndex = 0
SC_reserved = 0
ContextID = 1
12. On the failover production server edit the backup HostId stanza in the
/tmp/C<Failover Context ID>.cfg file replacing the content of
the ObjectAttributeValue field with the output from the
rthostid command.
SCCuAttr:
ObjectAttributeValue = "5FBBC3EF"
SerialNumber = 4006
ObjectNlsIndex = 0
SC_reserved = 0
ContextID = 11
13. On the failover production server, use sccfgd_putcfg to load the

configurations onto the node.
sccfgd_putcfg <Primary Context ID> /tmp/C<Primary
Context ID>.cfg
sccfgd_putcfg <Failover Context ID>

/tmp/C<Failover Context ID>.cfg
14. On the primary production server, use /usr/sbin/lvlstmajor to

obtain a list of unused device major numbers:
lvlstmajor
46,50,54,57,64,67,70,73..75,82..93,95...
15. On the failover production server, use /usr/sbin/lvlstmajor to

lvlstmajor
51,54,57,61,65,69,82...
16. On the primary production server, use es_ha_config to configure a

device major number for the Primary Context ID. Choose a device
major number that is the same on both production servers.
es_ha_config <Primary ContextID> [NewMajorNumber]
For example, when you execute:
290
es_ha_config 1 82
The Primary Context ID is 1, and the new device major number 82, is
available on both production servers.
17. On the failover production server, use es_ha_config to configure a
major number that is the same on both production servers.
For example, when you execute

es_ha_config 1 82
The Primary Context ID is 1, and the new device major number 82, is
available on both production servers.
18. On the primary production server, unmount the filesystems associated
with the RecoverNow configuration.
unmount /usr/scrt/run/c<Primary Context ID>
unmount /usr/scrt/run/c<Failover Context ID>
19. On the primary production server, varyoff all volume group(s)

associated with the RecoverNow configuration.
20. On both production servers, the filesystems protected by RecoverNow
should not be automatically mounted at system restart. Change the
auto-mount attribute to no.
21. Before starting RecoverNow on the production server, you must stop
your application and unmount the protected filesystems.
22. The Initial Replica Synchronization can be performed several ways.
Use RecoverNow to synchronize the data to the recovery server.
Save the protected data on the production server to tape or portable
disk. Refer to section Alternative Method for Performing Initial
Synchronization on page 205.
Copy the data over the network to the recovery server. Refer to
section Alternative Method for Performing Initial Synchronization
on page 205.
RecoverNow/PowerHA Integration requirements

Keep in mind the following RecoverNow/PowerHA integration
requirements:
1. Application ServerIf there are multiple Resource Groups that can
failover independently, there must be multiple RecoverNow ContextIDs
291
Chapter A:
created, one for each Resource Group. Likewise, each of these

Resource Groups requires scripts to manage the startup and shutdown
of each RecoverNow ContextID. The RecoverNow start script (see
RecoverNow Start Script on page 292) needs to be placed at the
beginning of the application start script. The RecoverNow stop script
(RecoverNow Shutdown Script on page 292) needs to be executed
after the application has been shutdown gracefully.
RecoverNow Start Script

This section shows a sample RecoverNow start script. The start script needs
to be placed at the beginning of the application server startup sequence,
before applications are started.
Sample RecoverNow Start script
#!/bin/ksh
DATE_CMD=$(date +"%Y-%m-%d %T")
CID=1
RTSTARTLOG=/usr/scrt/log/rtstartupc$CID.log
/usr/scrt/bin/rtattr -C${CID} -o production -a HostId -v
$(/usr/scrt/bin/rthostid)
if ! /usr/scrt/bin/rtstart -C${CID} ;
then
echo ${DATE_CMD}: "RecoverNow startup
exit 1
failed" >$RTSTARTLOG 2>&1
fi
echo "${DATE_CMD}: RecoverNow startup successful" >$RTSTARTLOG 2>&1
-> Insert application start script here <exit 0
RecoverNow Shutdown Script

This section shows a sample RecoverNow shutdown script. The shutdown
script needs to be executed at the end of the application server shutdown
sequence, after all applications have been stopped gracefully.
Sample RecoverNow Shutdown script
#!/bin/ksh93
-> Insert application stop script here <DATE_CMD=$(date +"%Y-%m-%d %T")
CID=1
RTSTOPLOG=/usr/scrt/log/rtshutdownc${CID}.log
if ! /usr/scrt/bin/rtumnt -C${CID} ;
then
echo "${DATE_CMD}: rtumnt failed" >$RTSTOPLOG 2>&1
for i in `/usr/scrt/bin/sclist -C${CID} -f`
do
fuser -k $i
done
then
292
echo "${DATE_CMD}: rtumnt failure" >$RTSTOPLOG 2>&1

exit 1
fi
fi
if !
then
echo
exit
fi
echo
exit
/usr/scrt/bin/rtstop
-C ${CID} -kFv
"${DATE_CMD}: RecoverNow shutdown failed" >$RTSTOPLOG 2>&1

1
"${DATE_CMD}: RecoverNow shutdown successful" >$RTSTOPLOG 2>&1
0
2. Configure a Service IP Label/Address if one does not already exist.

Configure HACMP Service IP Labels/Addresses.
Add a Service IP Label/Address
Configurable on Multiple Nodes
Use the IP Label/Address that was used by the RecoverNow
Replication Group Wizard.
3. Application Resource Group Changes
Enter the Service IP Labels/Addresses
Enter the names of all volume groups associated with the
RecoverNow <Primary ContextID> to be varied on when the
resource is initially acquired.
IMPORTANT
The RecoverNow drivers must be loaded before the RecoverNow

protected filesystems are mounted or written to. This is managed
during the execution of the RecoverNow startup process.
PowerHA determines which filesystems to mount based on the
information provided in the Resource Group configuration. If no
filesystems are specified, PowerHA will mount all filesystems in
all Volume Groups that defines in the resource group. This
scenario is not preferred for a RecoverNow environment since
RecoverNow will not be started before the filesystems are
mounted.
Enter the filesystem mount points /usr/scrt/run/c<Primary
Context ID> and /usr/scrt/run/c<Failover Context
ID>. This will force only these filesystems to be mounted when the
resource group is acquired.

4. Run HACMP Extended Verification and Synchronization.
5. On the recovery server use rtstart to manually start the RecoverNow
Primary Context ID.
293
Chapter A:
/usr/scrt/bin/rtstart -C<PrimaryContextID>.
6. Start Cluster Services on both Production nodes:

Verify that the volume groups defined in the Resource Group are
online in concurrent mode.
Verify that the /usr/scrt/run/c<Primary Context ID> and
/usr/scrt/run/c<Failover Context ID> filesystems are
mounted.
Verify that all of the RecoverNow protected filesystems are
mounted.
Verify that the Service IP Address is aliased on the Ethernet
Network Interface.
Verify that RecoverNow is replicating to the recovery server. View
the log files: /var/log/EchoStream/scrt_lca-<Primary
Context ID>.out on the production server and
/var/log/EchoStream/scrt_aba-<Primary Context
ID>.out on the recovery server.
Managing RecoverNow in a HA Production Server

Environment
Keep in mind the following when you manage RecoverNow in a HA
production server environment:
Changing the RecoverNow Configuration on page 294
Managing Failover to the Recovery Server on page 295
Changing the RecoverNow Configuration

Changes that you make to a RecoverNow configuration do not
automatically propagate to the failover production server. If you make any
changes to the configuration, you must:
1. On the primary production server, delete the Failover Context. Refer to
the section Deleting a Context on page 146.
2. On the failover production server, delete the Primary and Failover
Contexts. Refer to the section Deleting a Context on page 146
3. Perform the steps in Configuring Highly Available Production
Servers on page 288 to update the Primary and failover production
server with the current configuration information.
294
Managing Failover to the Recovery Server

There are two scenarios for failover from a HA production server to the
recovery server:
Unplanned Failover on page 295
Planned Failover/Resync/Failback on page 297

NOTE
Both scenarios require that you manually perform Failover

operations.
Unplanned Failover
In this scenario, both Highly Available Production servers are unavailable
due to a disaster. For example, an entire site is lost due to a disaster such as
a flood or hurricane.
Do not perform a failover until you have validated the integrity of the data
on the recovery server. After performing a failover, RecoverNow cannot be
rolled back to a point in time before the failover. This section provides
guidelines to execute before performing your failover operations.
server and restore it if necessary. To validate the data, first create a snapshot
of the replica and then analyze it with the application itself.
1. On the recovery server, create a Snapshot based on the current redo log
and validate the data integrity: Refer to Chapter 8, Creating
Snapshots on page 183.
If analysis indicates data corruption, remove the snapshot and create a
Snapshot based on a Specific LFCID or a Specific Date and Time to
locate and validate an optimal restore point.
2. On the recovery server, create a Snapshot based on a Specific LFCID or
a Specific Date and Time and validate the data integrity. Refer to
Chapter 8, Creating Snapshots on page 183.
Use scrt_ra to print valid restore time spans
/usr/scrt/bin/scrt_ra vpC<Primary ContextID>
Current Snap Time:
1300122094 (LFCID: 305000): Mon Mar 14 13:01:34 2011
Available internal rollback windows:
295
Chapter A:
-------------------------------------------------Start: 1300122094 (LFCID: 305000): Mon Mar 14 13:01:34

2011
End: 1300112259 (LFCID: 304502): Mon Mar 14 10:17:39
2011
Available VFBs:
-------------------------------------------------------
No recorded VFBs.
Once you have located an optimal restore point, remove the snapshot.
Proceed to step 3 to Backup the replica or to step 4 to perform a failover
restore or a Failover to the Latest Point in the Data.
3. On the recovery server, if you have TSM or SysBack, backup the
replica. This provides additional data protection by keeping complete
copies of the data on archive media such as tape. Refer to Chapter 11,
Working with Archived Data on page 227.
4. On the recovery server, perform a failover restore to rollback the replica
to the validated point in time from step 2 or a Failover to the Latest
Point in the Data.
To perform a failover restore, refer to Performing a Failover
Restore on page 256.
To perform a Failover to the Latest Point in the Data, refer to
Failover to the Latest Point in the Data on page 256.
5. On the recovery server, start your application.
Recovery of the Failed Production Server(s)
Cluster services should not be running on these nodes. Recreate the
RecoverNow production environment when one or both production servers
have been recovered, the steps will depend on the extent of the damage to
the production servers.
Managing Resynchronization to the Production Server
destroyed on the production server. Refer to Manual Resynchronization
Process if production server Data Is Lost on page 258.
If none of the protected data was lost on the production server, refer to A
resync operation is required when the Production Volumes and Recovery
Replica Volumes diverge.
296
Performing Failback
Before you failback ensure that you stop your application on the recovery
server. Refer to Performing Failback on page 258.
Planned Failover/Resync/Failback
In this scenario, the administrator has a scheduled maintenance period and
switches operations that run on the production server to the designated
recovery server.
Performing Failover
Perform the following steps for a planned failover:
1. On the production server stop your application.
2. On the production server, use rtstop to unmount the protected
filesystems, transfer any current LFC data to the recovery server and
unload the RecoverNow production server drivers.
/usr/scrt/bin/rtstop FSC<PrimaryContextID>
3. Verify that the synchronization of data has successfully completed, the

rtstop output will display:
4. On the recovery server execute the failover command to initiate the

failover process:
Answer y to the Do you wish to questions:

scsetup: You have requested failover processing.
scsetup: Do you wish to continue? [y|n]
!!! RESET WARNING !!!
You have requested an LCA reset.
All outstanding sealed LFCs will be dumped.
Do you wish to do this? [y|n]
Verify that the failover process completed, the failover output will
display:
---Failover Context ID <Failover ContextID> is
enabled. ---
5. On the recovery server start your application.

Performing Resync
Replica Volumes diverge. This occurs after a failover to the Recovery or
297
Chapter A:
replicated server. When the application is started on the Recovery or

replicated server the updates to the Replica Volumes result in a divergence
from the data on the production server. Refer to Performing Resync on
page 256.
Performing Failback
Before you failback ensure that you stop your application on the recovery
server.
6. Verify that the resync operation has completed, use esmon to check the
LFC usage count
esmon <FailoverContextID>
Example:
esmon 740
Mar 22 17:13:11 Total LFC Size=3025M, Free size=3014M,
Used Size=11M, Usage=1/100
Used Size=11M - 11 megabytes of data has not

been sent to the Production Server
Usage=1/100 1 LFC is in use
The failback process will transfer the used LFCs to the production
server.
7. On the recovery server failback, wait for failback to successfully
complete before performing failback on the production server. Refer to
Performing Failback on page 258.
8. On the production server failback, wait for failback to successfully
complete before starting your application Refer to Performing
Failback on page 258.
Configuring a Highly Available RecoverNow/PowerHA Recovery

Server Environment
A RecoverNow Highly Available (HA) recovery server configuration is
supported with PowerHA (HACMP). This section describes how you
configure RecoverNow in a HA recovery server environment.
In this mode:
298
There are two recovery nodes with shared disks between them.
Replication is from a third node that lies outside the cluster.
A Resource Group can only be moved between the two recovery nodes,
and the RecoverNow roles of the nodes never changes.
Configuring a Highly Available RecoverNow/PowerHA Recovery Server Environment
RecoverNow Configuration Requirements

Review these guidelines before you configure RecoverNow:
Do not enable automatic startup for RecoverNow in a HA environment.
RecoverNow software must be installed on both PowerHA recovery

server nodes and the production server node
All volume groups associated with a RecoverNow configuration must

be enhanced concurrent mode volume groups on both PowerHA
recovery server nodes. Use C-SPOC to create/change the volume
group(s), selecting Fast Disk Takeover or Disk Heart Beat for the
Enable Fast Disk Takeover or Concurrent Access option.
The Highly Available recovery server RecoverNow File Containers and

the RecoverNow configuration filesystems
/usr/scrt/run/c<Primary Context ID> and
/usr/scrt/run/c<Failover Context ID> can co-exist on the
same enhanced concurrent mode volume group(s) that are part of the
RecoverNow configuration. By default the RecoverNow File
Containers are created on the Default VG, defined using the
RecoverNow Replication Group Wizard. The filesystems need to be at
least 128MB in size and must have a separate jfslog, jfs2log, or jfs2
inline log since they will not be part of the PVS.
RecoverNow must be configured with a Service IP address (an alias

address that follows the application when it is moved between nodes).
To configure a Service IP Address, change the recovery servers Initial
host adapter IP Address to the address you choose to use. The
/etc/hosts file on all nodes must contain the Service IP Address and
associated IP label.
Configuring Highly Available Recovery Servers

To configure Highly Available recovery servers:
1. On the production server and primary recovery server, varyonvg all
volume group(s) associated with the RecoverNow configuration.
2. On the primary recovery server, mount the filesystems associated with
the RecoverNow configuration
mount /usr/scrt/run/c<Primary Context ID>
mount /usr/scrt/run/c<Failover Context ID>
3. Use the RecoverNow Replication Group Wizard to configure the

Primary Context. Refer to the Chapter 6, Configuring Replication
Groups on page 131
4. On the production server, initialize the context.
299
Chapter A:
5. On the production server, create the Failover Context ID.

6. On the primary recovery server, initialize the context.

7. On the primary recovery server, create the Failover Context ID.

IMPORTANT
You must manually copy and load the RecoverNow configuration

onto the Failover recovery server.
8. On the primary recovery server, create a file with the Primary Context
ID configuration.
odmget -q ContextID=<Primary Context ID> SCCuObj
SCCuAttr SCCuRel >/tmp/C<Primary Context ID>.cfg
Copy this configuration file to /tmp on the Failover Recovery

Server.
9. On the primary recovery server, create a file with the Failover Context
ID configuration.
odmget -q ContextID=<Failover Context ID> SCCuObj
SCCuAttr SCCuRel >/tmp/C<Failover Context ID>.cfg
Copy this configuration file to /tmp on the Failover Recovery

Server.
10. On the Failover recovery server use rthostid to obtain its HostId.
/usr/scrt/bin/rthostid
11. On the Failover recovery server edit the backup HostId stanza. In the
/tmp/C<Primary Context ID>.cfg file. Replace the contents of
rthostid command.
SCCuAttr:
ObjectAttributeValue = "6CABA7DF"
300
ObjectNlsIndex = 0
SC_reserved = 0
ContextID = 1
12. On the Failover recovery server edit the production HostId stanza in the
/tmp/C<Failover Context ID>.cfg file replacing the contents of
rthostid command.
SCCuAttr:
ObjectAttributeValue = "5FBBC3EF"
SerialNumber = 4006
ObjectNlsIndex = 0
SC_reserved = 0
ContextID = 11
13. On the Failover recovery server, use sccfgd_putcfg to load the

configurations onto the node.
sccfgd_putcfg <Primary Context ID> /tmp/C<Primary
Context ID>.cfg
sccfgd_putcfg <Failover Context ID>

/tmp/C<Failover Context ID>.cfg
14. On the primary recovery server, use /usr/sbin/lvlstmajor to
lvlstmajor
46,50,54,57,64,67,70,73..75,82..93,95...
15. On the Failover recovery server, use /usr/sbin/lvlstmajor to

lvlstmajor
51,54,57,61,65,69,82...
16. On the primary recovery server, use es_ha_config to configure a

major number that is the same on both recovery servers.

es_ha_config 1 82
301
Chapter A:
The Primary Context ID is 1, and the new device major number 82 is

available on both recovery servers.
17. On the Failover recovery server, use es_ha_config to configure a
major number that is the same on both recovery servers.

es_ha_config 1 82
The Primary Context ID is 1, and the new device major number 82 is

available on both recovery servers.
18. On the primary recovery server, unmount the filesystems
unmount /usr/scrt/run/c<Primary Context ID>
unmount /usr/scrt/run/c<Failover Context ID>
19. On the primary recovery server, varyoff all volume group(s)

20. On both recovery servers, the filesystems protected by RecoverNow
should not be automatically mounted at system restart. Change the
auto-mount attribute to no.
21. The Initial Replica Synchronization can be performed several ways:
Use RecoverNow to synchronize the data to the recovery server.
Save the protected data on the production server to tape or portable
disk. Refer to section Alternative Method for Performing Initial
Synchronization on page 205.
Copy the data over the network to the recovery server. Refer to
section Alternative Method for Performing Initial Synchronization
on page 205.
RecoverNow/PowerHA Integration requirements:

Keep in mind the following RecoverNow/PowerHA integration
requirements:
1. Application ServerIf there are multiple Resource Groups that can
failover independently, there must be multiple RecoverNow ContextIDs
created, one for each Resource Group. Likewise, each of these
Resource Groups requires scripts to manage the startup and shutdown
of each RecoverNow ContextID. The RecoverNow start script (see
RecoverNow Start Script on page 303) needs to be placed at the
beginning of the application start script. The RecoverNow stop script
302
(RecoverNow Shutdown Script on page 303) needs to be executed

after the application has been shutdown gracefully.
RecoverNow Start Script

This section shows a sample RecoverNow start script. The start script needs
to be placed at the beginning of the application server startup sequence,
before applications are started.
Sample RecoverNow Start script
#!/bin/ksh
DATE_CMD=$(date +"%Y-%m-%d %T")
CID=1
RTSTARTLOG=/usr/scrt/log/rtstartupc$CID.log
/usr/scrt/bin/rtattr -C${CID} -o production -a HostId -v
$(/usr/scrt/bin/rthostid)
if ! /usr/scrt/bin/rtstart -C${CID} ;
then
echo ${DATE_CMD}: "RecoverNow startup
exit 1
failed" >$RTSTARTLOG 2>&1
fi
echo "${DATE_CMD}: RecoverNow startup successful" >$RTSTARTLOG 2>&1
-> Insert application start script here <exit 0
RecoverNow Shutdown Script

This section shows a sample RecoverNow shutdown script. The shutdown
script needs to be executed at the end of the application server shutdown
sequence, after all applications have been stopped gracefully.
Sample RecoverNow Shutdown script
#!/bin/ksh93
-> Insert application stop script here <DATE_CMD=$(date +"%Y-%m-%d %T")
CID=1
RTSTOPLOG=/usr/scrt/log/rtshutdownc${CID}.log
then
echo "${DATE_CMD}: rtumnt failed" >$RTSTOPLOG 2>&1
for i in `/usr/scrt/bin/sclist -C${CID} -f`
do
fuser -k $i
done
then
echo "${DATE_CMD}: rtumnt failure" >$RTSTOPLOG 2>&1
exit 1
fi
fi
if ! /usr/scrt/bin/rtstop
then
-C ${CID} -kFv
303
Chapter A:
echo
exit
fi
echo
exit
"${DATE_CMD}: RecoverNow shutdown failed" >$RTSTOPLOG 2>&1

1
"${DATE_CMD}: RecoverNow shutdown successful" >$RTSTOPLOG 2>&1
0
2. Configure a Service IP Label/Address if one does not already exist.

Configure HACMP Service IP Labels/Addresses.
Add a Service IP Label/Address
Configurable on Multiple Nodes
Use the IP Label/Address that was used in the RecoverNow
Replication Group Wizard.
3. Application Resource Group Changes
Enter the Service IP Labels/Addresses
Enter the names of all volume groups associated with the
RecoverNow <Primary ContextID> to be varied on when the
resource is initially acquired.
IMPORTANT
The RecoverNow drivers must be loaded before the RecoverNow

protected filesystems are mounted or written to. This is managed
during the execution of the RecoverNow startup process.
PowerHA determines which filesystems to mount based on the
information provided in the Resource Group configuration. If no
filesystems are specified, PowerHA will mount all filesystems in
all Volume Groups that defines in the resource group. This
scenario is not preferred for a RecoverNow environment since
RecoverNow will not be started before the filesystems are
mounted.
Enter the filesystem mount points /usr/scrt/run/c<Primary
Context ID> and /usr/scrt/run/c<Failover Context
ID>. This will force only these filesystems to be mounted when the
resource group is acquired.

4. Run HACMP Extended Verification and Synchronization.
5. Before starting RecoverNow on the production server, you must stop
your application and unmount the protected filesystems. The
RecoverNow drivers must be loaded before the RecoverNow protected
filesystems are mounted or written to. This is managed during the
execution of the RecoverNow startup process.
6. On the production server use rtstart to manually start the RecoverNow
Primary Context ID.
304
/usr/scrt/bin/rtstart -C<PrimaryContextID>
7. Start Cluster Services on both Recovery nodes:

Verify that the volume groups defined in the Resource Group are
online in concurrent mode.
Verify that the /usr/scrt/run/c<Primary Context ID> and
/usr/scrt/run/c<Failover Context ID> filesystems are
mounted
Verify that the Service IP Address is aliased on the Ethernet
Network Interface.
Verify that RecoverNow is replicating to the recovery server, view
log file /var/log/EchoStream/scrt_lca-<Primary
Context ID>.out on the production server and
/var/log/EchoStream/scrt_aba-<Primary Context
ID>.out on the recovery server.
Managing RecoverNow in a HA Recovery Server

Environment
Keep in mind the following when you manage RecoverNow in a HA
production server environment:
Changing the RecoverNow Configuration on page 294
Managing Failover to the Recovery Server on page 305
Changing the RecoverNow Configuration

Changes that you make to a RecoverNow configuration do not
automatically propagate to the Failover recovery server. If you make any
changes to the configuration, you must:
1. On the primary recovery server, delete the Failover Context. Refer to
the section Deleting a Context on page 146.
2. On the Failover recovery server, delete the Primary and Failover
Contexts. Refer to the section Deleting a Context on page 146.
3. Perform the steps in Configuring Highly Available Recovery Servers
on page 299 to update the Primary and Failover recovery server with
the current configuration information.
Managing Failover to the Recovery Server

There are two scenarios for failover from a HA production server to a
Recovery server:
Unplanned Failover on page 306
305
Chapter A:
Planned Failover/Resync/Failback on page 297

NOTE
Both scenarios require that you manually perform Failover

operations.
Unplanned Failover
In this scenario, the production server is unavailable due to a disaster. For
example, an entire site is lost due to a disaster such as a flood or hurricane.
Do not perform a failover until you have validated the integrity of the data
on the active recovery server. After performing a failover, RecoverNow
cannot be rolled back to a point in time before the failover. This section
provides guidelines to execute before performing your failover operations:
failover operations, validate the data integrity of the replica on the active
recovery server and restore it if necessary. To validate the data, first create a
snapshot of the replica and then analyze it with the application itself.
1. On the active recovery server, create a Snapshot based on the current
redo log and validate the data integrity: Refer to Chapter 8, Creating
Snapshots on page 183.
If analysis indicates data corruption, remove the snapshot and create a
Snapshot based on a Specific LFCID or a Specific Date and Time to
locate and validate an optimal restore point.
2. On the active recovery server, create a Snapshot based on a Specific
LFCID or a Specific Date and Time and validate the data integrity:
Refer to Chapter 8, Creating Snapshots on page 183.
Use scrt_ra to print valid restore time spans
/usr/scrt/bin/scrt_ra vpC<Primary ContextID>
Current Snap Time:
1300122094 (LFCID: 305000): Mon Mar 14 13:01:34
2011
Available internal rollback windows:
------------------------------------------------------Start: 1300122094 (LFCID: 305000): Mon Mar 14
13:01:34 2011
End: 1300112259 (LFCID: 304502): Mon Mar 14
10:17:39 2011
306
Available VFBs:
-----------------------------------------------------No recorded VFBs.
Once you have located an optimal restore point, remove the snapshot.
Proceed to step 3 to Backup the replica, or to step 4 to perform a
failover restore or a failover to the Latest Point in the Data.
3. On the active recovery server, if you have TSM or SysBack, backup the
replica. This provides additional data protection by keeping complete
copies of the data on archive media such as tape. Refer to Chapter 11,
Working with Archived Data on page 227.
4. On the active recovery server, perform a failover restore to rollback the
replica to the validated point in time from step 2 or a Failover to the
Latest Point in the Data.
To perform a failover restore, refer to Performing a Failover
Restore on page 256.
To perform a Failover to the Latest Point in the Data, refer to
Failover to the Latest Point in the Data on page 256.
5. On the active recovery server, start your application.
Recovery of the Failed Production Server
Recreate the RecoverNow production environment when the production
server has been recovered, the steps will depend on the extent of the damage
to the production server.
Managing Resynchronization to the Production Server
destroyed on the production server. Refer to Manual Resynchronization
Process if production server Data Is Lost on page 258.
If none of the protected data was lost on the production server, refer to
Performing Resync on page 256.
Performing Failback
Before you failback ensure that you stop your application on the active
recovery server. Refer to Performing Failback on page 258.
307
Chapter A:
Planned Failover/Resync/Failback
In this scenario, the administrator has a scheduled maintenance period and
switches operations that run on the production server to the active recovery
server.
Performing Failover
1. On the production server stop your application.
2. On the production server, use rtstop to unmount the protected
filesystems, transfer any current LFC data to the recovery server and
unload the RecoverNow production server drivers.
/usr/scrt/bin/rtstop FSC<PrimaryContextID>
3. Verify that the synchronization of data has successfully completed, the

rtstop output will display:
4. On the active recovery server execute the failover command to initiate

the failover process:
Answer y to the Do you wish to questions:

scsetup: You have requested failover processing.
scsetup: Do you wish to continue? [yen]
!!! RESET WARNING !!!
You have requested an LCA reset.
All outstanding sealed LFCs will be dumped.
Do you wish to do this? [y|n]
Verify that the failover process completed, the failover output will
display:
---Failover Context ID <Failover ContextID> is
enabled. ---
5. On the active recovery server start your application.

Performing Resync
Replica Volumes diverge. This occurs after a failover to the recovery or
replicated server. When the application is started on the Recovery or
replicated server the updates to the Replica Volumes result in a divergence
from the data on the production server. Refer to Performing Resync on
page 256.
308
Configuring a RecoverNow/PowerHA Production to Recovery Server Environment
Performing Failback
1. Before you failback ensure that you stop your application on the active
recovery server.
2. Verify that the resync operation has completed, use esmon to check the
LFC usage count
esmon <FailoverContextID>
Example:
esmon 740
Mar 22 17:13:11 Total LFC Size=3025M, Free size=3014M,
Used Size=11M, Usage=1/100
Used Size=11M - 11 megabytes of data has not

been sent to the Production Server
Usage=1/100 1 LFC is in use
The failback process will transfer the used LFCs to the production
server.
3. On the active recovery server failback, wait for failback to successfully
complete before performing failback on the production server. Refer to
Performing Failback on page 258.
4. On the production server failback, wait for failback to successfully
complete before starting your application. Refer to Performing
Failback on page 258.
Configuring a RecoverNow/PowerHA Production to Recovery

Server Environment
This purpose of this section:
Have PowerHA for AIX monitor a RecoverNow context. PowerHA for

AIX will monitor the LCA and ABA RecoverNow agents and provides
notification in the event of failure.
If the RecoverNow production server fails, provide the ability to

failover RecoverNow and the users application to the recovery server
after verifying the Replica data.
Provide the ability to control RecoverNow by manipulating the location

and status of the PowerHA for AIX Resource Groups.
Prerequisites
Before you begin, keep in mind the following:
309
Chapter A:
RecoverNow must be operational and configured with Primary and

Failover contexts.
scconfigd must be running on the RecoverNow production and

recovery servers.
Starting RecoverNow and the associated user applications in

"/etc/inittab" is not recommended. If the production server fails and the
user applications are brought up on the recovery server, it would not be
desirable to have them start when the production server is restored to
service.
Overview of the Failover Process

If the Production_Server Resource Group fails it will not automatically
move to the Recovery node because it is anti-collocated, low priority with
the Recovery_Server Resource Group.
Not doing a automatic Failover allows the creation of a snapshot to validate
the Replica data and to restore the Replica to a point in time if required. The
Recovery_Server Resource Group will have to be brought offline before the
Production_Server Resource Group can be brought online on the recovery
server.
When the Production_Server Resource Group is brought online on the
Recovery node it executes the rtdr -C <Primary Context ID> failover
command. When the Production_Server Resource Group is brought offline
(on the recovery server), after the Production node becomes available, it
will send the rtdr -C <Primary Context ID> resync command to the
Production node before stopping.
There may be special cases where you would delay bringing the
Production_Server Resource Group offline on the recovery server.
Normally, this would mean that no data is being synchronized to the
production server. In that case, you can manually execute the rtdr -C
<Primary Context ID> resync command on the production server. This
starts the ABA but it will not be monitored by PowerHA for AIX.
If the Recovery_Server Resource Group fails it will not move, because
there is only one node in the participating node list.
Ensure that you have the Recovery_Server Resource Group online to
prevent an automatic failover of the Production_Server Resource Group to
the recovery server.
310
Sequence to manually bring the Resource Groups

ONLINE:
1. Bring the Recovery_Server Resource Group online on the Recovery
node.
2. Bring the Production_Server Resource Group online on the Production
node.
Sequence to manually bring the Resource Groups

OFFLINE:
1. Bring the Production_Server Resource Group offline on the Production
node.
2. Bring the Recovery_Server Resource Group offline on the Recovery
node.
Planned Failover Procedure

1. Bring the Production_Server Resource Group offline on the Production
node.
node.
3. Bring the Production_Server Resource Group online on the Recovery
node.
Unplanned Failover Procedure

1. Create a snapshot to validate the Replica data and restore the Replica to
a validated point in time, if required. Refer to Creating Snapshots
Based on a Specific Date and Time on page 189 for information on
creating snapshots. Refer to Performing a Failover Restore on
page 256 for information on how to restore the Replica to a validated
point in time.
node.
3. Bring the Production_Server Resource Group online on the Recovery
node.
Failback Procedure
1. Move the Production_Server Resource Group to the Production node.
2. Bring the Recovery_Server Resource Group online on the Recovery
node.
311
Chapter A:
PowerHA for AIX Configuration

Note the following for the example RecoverNow configuration:
Performed using PowerHA for AIX 6.1.0.7
Names used such as, Cluster Nodes, Resource Groups are arbitrary. The
integrator can choose to use any names.
Topology configuration for networks, communication interfaces and

devices is not shown in this example because it is environment specific.
Notification scripts are not provided and are the responsibility of the
integrator.
The implementation of the user application in this example is a

suggestion and could be done differently at the integrators preference.
Has a Primary Context ID of 85 and Failover Context ID of 850.
The following RecoverNow scripts are provided for the PowerHA for AIX
configuration. These scripts require parameters -C <Primary Context ID>
and for the first two scripts optionally -P if called from the
Production_Server Resource Group. These scripts will log to "/usr/scrt/log"
if the "HACMP Log File Parameters" have "Debug Level" set to "high".
/usr/scrt/bin/production_failback_acquire
/usr/scrt/bin/production_failover_release
/usr/scrt/bin/ABA_Monitor
/usr/scrt/bin/LCA_Monitor
312
Sample PowerHA Configuration

Configure HACMP Nodes:
Production
Recovery
Configure Resource Groups:
Resource Group Name
Production_Server
Participating Nodes
Production Recovery]
Startup Policy
Online On Home Node Only
Fallover Policy
Fallover To Next Priority Node In The List
Fallback Policy
Never Fallback
Resource Group Name
Recovery_Server
Participating Nodes
[Recovery]
Startup Policy
Online On Home Node Only
Fallover Policy
Fallover To Next Priority Node In The List
Fallback Policy
Never Fallback
Configure Online on Different Nodes Dependency

High Priority Resource Group(s)
[Recovery_Server]
Low Priority Resource Group(s)
[Production_Server]
Configure HACMP Application Servers

Server Name
Production_85
Start Script
[/users_path_name/Production_Server_Start]
Stop Script
[/users_path_name/Production_Server_Stop]
Application Monitor
Server Name
LCA_Monitor_85 Monitor_85
Recovery_85
Start Script
[/usr/scrt/bin/production_failback_acquire -C <Primary Context ID>]
Stop Script
[/usr/scrt/bin/production_failover_release -C <Primary Context ID>]
Application Monitor
ABA_Monitor_85
313
Chapter A:
Example for Production_Server_Start script

#!/bin/ksh
###############################################################################
#
# Name:
Production_Server_Start
#
# Arguments:
None
#
# Returns:
0 - success
#
# Environment:
None
###############################################################################
###############################################################################
#
Main Entry Point
################################################################################
PROGNAME=${0##*/}
[[ ${VERBOSE_LOGGING} == high ]] &&
{
rm -f /tmp/${PROGNAME}.out
exec 1> /tmp/${PROGNAME}.out
exec 2>&1
PS4='[${PROGNAME}][${LINENO}]'
set -x
}
printf "$(date) ******** Begin ${PROGNAME} ********\n"
/usr/scrt/bin/production_failback_acquire -C <Primary Context ID> -P
if ((${?}!=0))
then
printf "$(date) Production Server start failed.\n"
exit 1
fi
printf "$(date) Production Server start successful.\n"
-> Insert application start script here <if ((${?}!=0))
then
printf "$(date) Double-Take_RecoverNow_85_Application_Start failed.\n"
exit 1
fi
printf "$(date) Double-Take_RecoverNow_85_Application_Start successful.\n"
################################################################################
314
Example for Production_Server_Stop script

#!/bin/ksh
###############################################################################
#
# Name:
Production_Server_Stop
#
# Arguments:
None
#
# Returns:
0 - success
#
# Environment:
None
###############################################################################
###############################################################################
#
Main Entry Point
###############################################################################
PROGNAME=${0##*/}
[[ ${VERBOSE_LOGGING} == high ]] &&
{
rm -f /tmp/${PROGNAME}.out
exec 1> /tmp/${PROGNAME}.out
exec 2>&1
PS4='[${PROGNAME}][${LINENO}]'
set -x
}
printf "$(date) ******** Begin ${PROGNAME} ********\n"
-> Insert application stop script here <if ((${?}!=0))

then
printf "$(date) Double-Take_RecoverNow_85_Application_Stop failed.\n"
exit 1
fi
printf "$(date) Double-Take_RecoverNow_85_Application_Stop successful.\n"
/usr/scrt/bin/production_failover_release -C <Primary Context ID> -P
if ((${?}!=0))
then
printf "$(date) Production Server stop failed.\n"
exit 1
fi
printf "$(date) Production Server stop successful.\n"
###############################################################################
315
Chapter A:
Configure Custom Application Monitors:

Monitor Name
LCA_Monitor_85
Application Server(s) to Monitor

Monitor Mode
Production_85
[Long-running monitoring]
Monitor Method
[/usr/scrt/bin/LCA_Monitor -C <Primary Context ID>]
Monitor Interval
[180]
Hung Monitor Signal
[9]
Stabilization Interval
[900]
Restart Count
[0]
Restart Interval
[0]
Action on Application Failure
[notify]
Notify Method
[/users_path_name/LCA_85_Notify]
Note:
The value for "Stabilization Interval" depends on the time required

to reset the LFCs on Failover.
This depends on the number of LFCs
on the Recovery Server and the system performance.
With 20,000
LFCs it could typically take up to 15 minutes.

Monitor Name
ABA_Monitor_85

Monitor Mode
Recovery_85
Monitor Method
[/usr/scrt/bin/ABA_Monitor -C <Primary Context ID>]
Monitor Interval
[180]
Hung Monitor Signal
[9]
[900]
Restart Count
[0]
Restart Interval
[0]
[notify]
Notify Method
[/users_path_name/ABA_85_Notify]
Monitor Name
Monitor_85
Application_85
Monitor Mode
Monitor Method
[/users_path_name/Double-Take_RecoverNow_85_Application_Monitor]
Monitor Interval
[60]
Hung Monitor Signal
[9]
[900]
Restart Count
[0]
Restart Interval
[0]
[notify]
Notify Method
[/users_path_name/App_85_Notify]
316
Change/Show All Resources and Attributes for a Resource Group

Resource Group Name
Participating Nodes (Default Node Priority)
Application Servers
Resource Group Name
Participating Nodes (Default Node Priority)
Application Servers
Production_Server
Production Recovery
[Production_85]
Recovery_Server
Recovery
[Recovery_85]
HACMP Log File Parameters:

Node Name
Production
Debug Level
high
Formatting options for hacmp.out
Standard
HACMP Log File Parameters:

Node Name
Recovery
Debug Level
high
Formatting options for hacmp.out
Standard
317
Chapter A:
318

Double-Take RecoverNow User Guide

Hochgeladen von

Dokumentinformationen

Copyright

Verfügbare Formate

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

Verfügbare Formate

Double-Take RecoverNow User Guide

Hochgeladen von

Copyright:

Verfügbare Formate

D o u b l e - Ta k e R e c o v e r N o w

Adobe and Acrobat ReaderAdobe Systems, Inc.

Double-Take, GeoCluster, and NSINSI Software, Inc.

Java, all Java-based trademarks, and Solaris-Oracle Corporation.

Microsoft and WindowsMicrosoft Corporation.

Mozilla and FirefoxMozilla Foundation.

NetscapeNetscape Communications Corporation.

Red HatRed Hat, Inc.

Symantec and NetBackupSymantec Corporation.

UNIX and UNIXWarethe Open Group.

Chapter 1Overview of Data Replication Concepts . . . . . . . . . . . . . . . . . . . . 15

Double-Take RecoverNow v4.0.01.00 User Guide

Chapter 2Planning your Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .29

Chapter 3Using the Sizing Tool to Calculate LFC Size. . . . . . . . . . . . . . . . .43

Double-Take RecoverNow v4.0.01.00 User Guide

Chapter 4Supported Configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

Chapter 5Installation Procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

Double-Take RecoverNow v4.0.01.00 User Guide

Logging in to Vision Solutions Portal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127

Chapter 6Configuring Replication Groups . . . . . . . . . . . . . . . . . . . . . . . . .131

Chapter 7Starting and Stopping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .177

Chapter 8Creating Snapshots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .183

Double-Take RecoverNow v4.0.01.00 User Guide

Verify that the DATEMSK environment variable is defined . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189

Chapter 9Administrative Tasks. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193

Chapter 10Working with RecoverNow Applications . . . . . . . . . . . . . . . . . 211

Double-Take RecoverNow v4.0.01.00 User Guide

Step 6: Recreate the Snapshot on the Recovery Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217

Chapter 11Working with Archived Data . . . . . . . . . . . . . . . . . . . . . . . . . . .227

Chapter 12Introduction to Disaster Recovery . . . . . . . . . . . . . . . . . . . . . . .231

Double-Take RecoverNow v4.0.01.00 User Guide

Chapter 13Disaster Recovery Operations . . . . . . . . . . . . . . . . . . . . . . . . . . 241

Double-Take RecoverNow v4.0.01.00 User Guide

Resync with Region Recovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257

Chapter 14CLI Commands. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .261

Double-Take RecoverNow v4.0.01.00 User Guide

Double-Take RecoverNow v4.0.01.00 User Guide

Appendix AIntegration of PowerHA (HACMP) with RecoverNow . . . . . .287

Double-Take RecoverNow v4.0.01.00 User Guide

Overview of Data Replication

Chapter 1, Overview of Data

Use this chapter to learn about the main

Chapter 2, Planning your Environment

Use this chapter to prepare RecoverNow

Chapter 3, Using the Sizing Tool to

Use this chapter to learn how to use the

Chapter 4, Supported Configurations

Use this chapter to learn about the

Chapter 5, Installation Procedures on

Use the procedures in this chapter to

Chapter 6, Configuring Replication

Use the Replication Group Configuration

Chapter 7, Starting and Stopping on

Use the procedures in this chapter to start

Chapter 8, Creating Snapshots on

Use the procedures in this chapter to

Chapter 9, Administrative Tasks on

Use the procedures in this chapter to

Double-Take RecoverNow v4.0.01.00 User Guide

Chapter 1: Overview of Data Replication Concepts

Chapter 10, Working with RecoverNow

Use the procedures in this chapter to

Chapter 11, Working with Archived

Use the procedures in this chapter to