Beruflich Dokumente
Kultur Dokumente
Building a low-cost, self-contained, learning, testing and development environment using EMC Celerra Virtual Storage Appliances (VSAs) with VMware Virtual Infrastructure and VMware Site Recovery Manager (SRM)
Revision 1.0.1 January17, 2009 Bernie Baker Sr. VMware Specialist EMC | VMware Affinity Team (339) 293-2320 baker_bernard@emc.com
Source Material:
http://virtualgeek.typepad.com/virtual_geek/2008/08/celerra-virtual.html http://virtualgeek.typepad.com/virtual_geek/2008/11/celerra-virtual.html http://virtualgeek.typepad.com/virtual_geek/2008/11/celerra-virtual-appliance-howto-301---replicating-between-two-celerra-vsas.html Administering VMware Site Recovery Manager 1.0, Mike Laverick VMware Site Recovery Manager 1.0 Evaluator Guide VMware Site Recovery Manager 1.0 Update 1 Administration Guide EMC Celerra NS Series iSCSI, EMC Celerra Replicator Adapter for VMware Site Recovery Manager, Version 1.0 Release Notes EMC P/N 300-007-023 Configuring iSCSI Targets on EMC Celerra EMC P/N 300-004-153 Using EMC Celera Replicator (V2) P/N 300-004-188 On-Demand Web Replay: EMC Celerra Replicator Delivers Advanced IP Storage Protection http://www.userlocal.com/helpvi.php
THE CONTENTS OF THIS WORK ARE NOT MEANT TO BE A COMPLETE INSTALLATION AND CONFIGURATION GUIDE BUT AN INITIAL CONFIGURATION VIABLE TO SUPPORT LEARNING, TESTING AND DEVELOPMENT OF THE CELERRA VIRTUAL STORAGE APPLIANCE (VSA) IN A NON-PRODUCTION VMWARE VIRTUAL INFRASTRUCTURE ENVIRONMENT WITH VMWARE SITE RECOVERY MANAGER (SRM). NOTE: THIS CONFIGURATION, SPECIFICALLY RUNNING CELERRA VSAS ON VMWARE ESX, NOT SUPPORTED IN PRODUCTION BY EMC OR VMWARE. BEST EFFORT SUPPORT PRTOVIDED BY FORUMS.EMC.COM, VIRTUALGEEK.TYPEPAD.COM AND THE VMWARE AFFINITY TEAM WITHIN EMC. HTTP://FORUMS.EMC.COM HTTP://VIRTUALGEEK.TYPEPAD.COM ALSO, NOTE THAT YOU CANNOT USE THIS FOR ANYTHING BEYOND TESTING AND DEVELOPMENT WITHOUT VIOLATING THE LICENSE AGREEMENT
THANKS TO MY COLLEAGUES ON THE VMWARE AFFINTY TEAM AT EMC FOR TESTING, PROOFING AND SUPPORTING THIS EFFORT
A SPECIAL THANK YOU TO STEPHEN SPELLICY FOR HIS EFFORTS IN THE CREATION OF THIS PAPER
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager
Table of Contents
Overview .................................................................................................................................................5 Diagram ................................................................................................................................................5 Required Server and Network Hardware .........................................................................................5 Required Software ..............................................................................................................................6 Required Infrastructure ......................................................................................................................6 Required Commitment .......................................................................................................................7 General Comment ...............................................................................................................................7 Small Request ......................................................................................................................................7 My Network ...........................................................................................................................................8 Section 1: The Basics .........................................................................................................................9 Assumptions .........................................................................................................................................9 Step 1: Download and Import Celerra Virtual Storage Appliance (VSA) ..................................10 Step 2: Configuring the Celerra VSA VM .......................................................................................13 Step 3: Configuring the Celerra VSA TCP/IP addresses..............................................................14 Step 4: Configuring the Celerra VSA to be Unique ......................................................................18 Step 5: Licensing ...............................................................................................................................27 Step 6: Configuring cge IP addresses ...........................................................................................32 Section 2: Adding Physical Storage ............................................................................................33 Step 1: Add an new VMDK to your Celerra VSA VM ....................................................................33 Step 2: Configure the Celerra VSA to use the new storage .......................................................34 Step 3: Adding Physical Disk to the Celerra VSA ......................................................................37 Step 4: Configure the Celerra VSA to use the new storage .......................................................39 Section 3: Review ..............................................................................................................................40 Section 4: Configuring Celerra Replication .............................................................................41 Step 1: Configuring NTP (Network Time Protocol)......................................................................42 Step 2: Correcting the Celerra VSA Replication Database ..........................................................44 Step 3: Configuring Replication Using the Celerra Manager GUI ..............................................48 Step 4: Preparing ESX Servers for iSCSI Targets and LUNs .......................................................72 Section 5: Site Recovery Manager Installation......................................................................79 Step 1: Site Recovery Manager Database Connectivity ..............................................................80 Step 2: Copying Site Recovery Manager and the SRA ................................................................88 Step 3: Installing Site Recovery Manager .....................................................................................88 Step 4: Installing the Storage Replication Adapter......................................................................91 Step 5: Installing the SRM Plug-in for Virtual Center ..................................................................91
Page 3 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager Section 6: Site Recovery Manager Configuration .................................................................95 Step 1: Configuring the Connection...............................................................................................96 Step 2: Configuring the Array Managers .......................................................................................99 Step 3: Configuring Inventory Mappings (Optional) .................................................................104 Step 4: Configuring Protection Groups ........................................................................................106 Step 5: Configuring Recovery Groups .........................................................................................109 Step 6: Testing (at long last) .....................................................................................................113 Appendix A: Configuring the Replication Target (Command Line Interface) ..........119 Configuring Replication ..................................................................................................................122 Appendix B: iSCSI and NFS Discrete LUN Creation & Assignment ...............................124 Step 1: Creating iSCSI Targets......................................................................................................124 Step 2: Configuring iSCSI LUNs and presenting them to ESX as VMFS Datastores ...........126 Step 3: Configuring NFS LUNs and presenting them to ESX as NFS Datastores ..................137 Appendix C: Basic VI Commands...............................................................................................144 Appendix D: Troubleshooting Tips ............................................................................................146
Page 4 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager
Overview
There are two goals associated with this initiative. The first is this installation and configuration of geographically separate Celerra VSA instances complete with bi-direction replication. The second goal leverages these Celerra VSA for the purpose or testing and developing a working knowledge of VMwares Site Recovery Manager (SRM). The ultimate goal is to build and environment similar to the one depicted in the diagram below.
Diagram
VMware Demo Environment Site Recovery Manager (SRM)
Protected Site
VCPROD1
Recovery Site
VCDR1
VirtualCenter
Virtual Center 2.5 Update 3 Build 119598
VirtualCenter
Virtual Center 2.5 Update 3 Build 119598
Dell PowerEdg 1850 Two Intel 2.8GHz 4 GB RAM Two GbE (Broadcom) Adapters PERC4 (RAID Controller) Two 73GB SCSI RAID1 Dell Remote Access Card (DRAC) DRAC IP: xxx.xxx.xxx.xxx
These Disks are provisioned from within the Celerra VSA VMs. The disk used to setup the Celerra VSA VMs is provisioned from the storage internal to the physical server ESX is running on
VMW Virtual Center 2 Non Replicated Protected Site VM Staging Area Celerra VSA (csdr1)
Dell PowerEdg 1850 Two Intel 2.8GHz 4 GB RAM Two GbE (Broadcom) Adapters PERC4 (RAID Controller) Two 73GB SCSI RAID1 Dell Remote Access Card (DRAC)
TCP/IP Network
Need to build servers on a budget? Check out the following link: http://virtualgeek.typepad.com/virtual_geek/2008/06/building-a-home.html
Page 5 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager
Required Software
VMware ESX 3.5 Update2 (Evaluation License: 60 days by default from VMware download site) VMware Virtual Center 2.5 Update 3 (Evaluation License as per above) VMware Site Recovery Manager 1.0 Update 1 (Evaluation License as per above) The EMC Celerra Virtual Storage Appliance (VSA) Microsoft Windows XP or Windows 2003 Server R2 Enterprise Edition for use with the VMware Virtual Center Server Microsoft Windows XP for use as a utility device as required. It makes sense to use several of these as for SRM VMware SCSI Floppy image for installing XP in a VM on ESX Windows 2003 Server R2 Standard Edition or Enterprise for various servers you may want to create and test against. e.g., Active Directory, DNS, SQL, Exchange, etc. Other Operating Systems such as Solaris x86, Red Hat Enterprise Linux, CentOS or Ubuntu EMC Celerra Replicator Adapter for VMware Site Recovery Manager Microsoft SQL Server 2005 Management Studio Express (SSMSEE) Microsoft SYSPREP (optional) provides for customization of VMs deployed from a template
Required Infrastructure
There are several components in this lab that work better if you have Active Directory, Domain Name Server (DNS), Windows Internet Name Service (WINS) and DHCP enabled. These dont have to be large elaborate implementations. In fact, all of these services could be setup in a single VM on one of your ESX hosts. This about simple domain names like lab.com or vsasrmlab.com. Create a DHCP scope to hand out TCP/IP addresses, a default gateway, DNS and WINS information. Have the client devices register host information automatically with DNS. This is a checkbox in the TCP/IP properties of the Ethernet adapter for the defined virtual machines. For those devices with static IP address add them to the DNS tables as A record, host, entries. Dont forget to click the PTR (reverse lookup) check box. A quick note about disk space; you should plan on allocating disk capacity for the primary and DR site Celerra VSAs as follows: Primary site: as required/desired to support your environment. Default, required minimum, is 40GB. Add space as required. DR Site: plan to allocate a file system that is 2.5-3 times larger that the replication target LUN. With out this space you will have issues creating snapshots required for Site Recovery Manager. Page 6 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager
Required Commitment
The task you are about to embark on will take some time to setup and configure. In my estimation, you should be able to accomplish the complete implementation in a day or two. The first time I went through the task this document didnt exist. I took me and a colleague two days to complete all of the required tasks leveraging a multitude of different documents and subject matter experts. Now that we have done it several times we can have a full environment up and running within 8 hours. That assumes, two people working from scratch, installing ESX, setting up active directory, test machines, Celerra VSA installs and SRM. There is a lot of work to accomplish but if you stick with it, the benefits will be instantly recognizable.
General Comment
If you need any assistance setting up your VMware ESX infrastructure I suggest reviewing these documents: Release Notes: ESX Server 3.5 Update 2 and VirtualCenter 2.5 Update 2 Quick Start Guide Basic System Administration
Small Request
This document represents a great deal of effort on my behalf and that of several of my peers on the EMC VMware Affinity Team. We appreciate your desire to test VMware and EMC in your environment. If you have any questions or comments please forward them to my email address at baker_bernard@emc.com or berniebaker@gmail.com. Please let me know if you find any problems or glaring omissions. I have tried to ensure completeness and accuracy.
Page 7 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager
My Network
Protected Site
VCPROD1 IP Address: xxx.xxx.xxx.xxx
EMC Celerra VSA and VMware Site Recovery Manager (SRM) Lab Environment
Recovery Site
VCDR1 IP Address: xxx.xxx.xxx.xxx
VirtualCenter
Qty 2 Dell PowerEdge 1850 Two Intel 2.8GHz 8 GB RAM Two GbE Adapters PERC4 (RAID) Controller Two 73GB 15K SCSI RAID 1 VMware DRS/HA Cluster (vMotion Enabled) Dell Remote Access Card (DRAC4) DRAC IP: xxx.xxx.xxx.xxx (192.168.1.0 Net) VMware ESX2 IP: eth0: xxx.xxx.xxx.xxx (192.126.1.0 Net) eth1: xxx.xxx.xxx.xxx (10.100.1.0 Net) VMware ESX3 IP: eth0: xxx.xxx.xxx.xxx (192.126.1.0 Net) eth1: xxx.xxx.xxx.xxx (10.100.1.0 Net) Virtual Center 2.5 Update 3 Build 119598
VirtualCenter
Virtual Center 2.5 Update 3 Build 119598
VMW Virtual Center 1 w/ License Server and SRM Server Non Replicated Celerra VSA (csprod1) MS Active Directory (2) MS DNS (2) MS DHCP (1) MS Windows XP (Clients) (4)
VMW Virtual Center 2 w/ License Server and SRM Server Non Replicated Celerra VSA (csdr1) Protected Site VM Staging Area
These Disks are provisioned from within the Celerra VSA VMs. The disk used to setup the Celerra VSA VMs is provisioned from the storage internal a physical server on which ESX is running: Server Name is ESX1 (not pictured)
Qty 2: Dell PowerEdge 1850 Two Intel 2.8GHz 8 GB RAM Two GbE Adapters PERC4 (RAID) Controller Two 73GB 15K SCSI RAID 1 VMware DRS/HA Cluster (vMotion Enabled)
Dell Remote Access Card (DRAC4) DRAC IP: xxx.xxx.xxx.xxx (192.168.1.0 Net) VMware: ESX2 ESX 3.5 U2 Build 82663 Evaluation License VMware ESX3 IP: eth0: xxx.xxx.xxx.xxx (192.168.1.0 Net) eth1: xxx.xxx.xxx.xxx (10.100.10.0 Net) VMware ESX4 IP: eth0: xxx.xxx.xxx.xxx (192.168.1.0 Net) eth1: xxx.xxx.xxx.xxx (10.100.10.0 Net)
iSCSI/TCP/IP Network
EMC Celerra VSA VM IP: eth0: xxx.xxx.xxx.xxx (General/NFS/CIFS) 192 Net eth1: xxx.xxx.xxx.xxx (iSCSI) 10 Net eth2: xxx.xxx.xxx.xxx (Control Station) 192 Net cge0: xxx.xxx.xxx.xxx (Data Mover NFS/CIFS target) 192 Net cge1: xxx.xxx.xxx.xxx (Data Mover iSCSI target) 10 Net
EMC Celerra VSA IP: eth0: xxx.xxx.xxx.xxx (General/NFS/CIFS) -192 Net eth1: xxx.xxx.xxx.xxx (iSCSI) -10 Net eth2: xxx.xxx.xxx.xxx (Control Station) -192 Net cge0: xxx.xxx.xxx.xxx (Data Mover NFS/CIFS target) -192 Net cge1: xxx.xxx.xxx.xxx (Data Mover iSCSI target) -10 Net
All server and Celerra VSA TCP/IP addresses are static. The client TCP/IP assignments accomplished via MS DHCP server running as a VM on ESX1 DNS Domain: baker-iss.com
Page 8 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager
With the intention of streamlining the installation and configuration process, this document focuses on the core steps required to build out multiple Celerra VSAs to support replication and disaster recovery using the Site Recovery Manger (SRM) from VMware. As mentioned on page 2, some of the source material contained within was provided via the Virtual Geek web blog. Ill reference the blog on a few occasions throughout this document. Please note the main URL for this site is: http://virtualgeek.typepad.com
Assumptions
I have assumed some core competency with respect to Linux, VMware ESX and VMware Virtual Center. I have added tips on specific Linux commands and tried to remain consistent with those tips throughout Active Directory implementation to support the environment (I setup two AD virtual machines. One on each ESX host) I also assume you have two preconfigured VMware ESX 3.5 Update2 servers under the management of VMware Virtual Center 2.5 Update3 implemented as virtual machines on the ESX servers each VC manages You will need full administrative access to all of the platforms discussed Need a little help with vi? See Appendix C Basic vi Commands One last thing The Celerra VSA is implemented in a service know as blackbird and is sometimes referred to as a Celerra Simulator.
So have you had enough of the blah blah blah? Lets get started with the Celerra VSAs!! Page 9 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager
THIS IS AN EXTREMELY LARGE FILE (approximately 1.4GB). ALLOW ADEQUATE TIME FOR THE DOWNLOAD TO COMPLETE
Once downloaded: unzip the file make note of the location start the Virtual Infrastructure Client and attach to your production Virtual Center instance (in my case it was vcprod1)
Dont forget to come back and perform these steps again for the recovery site. You will attach to the recovery site Virtual Center instance (mine was vcdr1) and configure the Celerra VSA in the same manner as the production site
Page 10 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager From Virtual Center, Click File, Virtual Appliance, Import
Click on the Import from File radio button, Click Browse, Browse to the .OVF file and Click Open Click Next when returned to the Import Location panel.
Page 11 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager Identify the Name of the VM (csprod1 and csdr1) and the target datastore. In most cases I will choose to put the Celerra VSA on the local datastore of the ESX hosting the virtual machine. If and when you add additional storage to the Celerra VSA the source is inconsequential. It can come from a local source (internal server disk) or a network based source such as a physical EMC Celerra. Setup network mappings (leave as default for now) Summary, Finish, Import process started. Typically the import process takes about 15 minutes to complete.
Note: OVF is approximately 1.4GB. When finished importing the file will be in eagerthick VMDK format and approximately 40GB.
Once imported Click Close
Prior to configuring anything else; clone the Celerra VSA VM to Template from Virtual Center for future use. Right Click on the Celerra VSA VM and Select Clone to Template. (Note: The Celerra VSA VM needs to be powered off in order to complete this step) Give it a name and datastore location; identify the cluster on which to store the template. Store it in Compact format. Page 12 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager
This screen shot also shows additional hard disks assigned to the Celerra VSA VM. Your configuration may or may not require these. For now, take the defaults on the disk configuration. We will discuss additional storage later in this document.
Power ON the Celerra VM. Right Click on the Celerra VM name and Select Power On Page 13 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager
Please note: You will need to click inside the VM console window in order to interact with the command line. If the screen is blank, click inside the window and hit the Enter key. If you need to interact with some other window type CTL-ATL together to release the cursor from the VM window. This is required as VMware Tools is not installed on the VM.
Login to the Control Station; User: root password: nasadmin Type the following command: (do not type [root@localhost ~]# as this is the system prompt) [root@localhost ~]# ifconfig | less
The less command will allow for scrolling up and down within the command output for the purpose of viewing all of the interfaces.
Page 14 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager
The virtual NICs (vNICs) are as follows: dart-eth0: (Internal Interface for DART OS Data Mover Celerra VSA service) dart-eth1:(Internal Interface for DART OS Data Mover Celerra VSA service) eth0: connected to first vNIC
(by default this maps to cge0 and cge1 in the Celerra Manager but this needs to be changed well do that later and explain what cge0 and cge1 are used for)
eth1: connected to the second vNIC (by default not connected to any cge ports but it should be) eth2: management interface for the Celerra VSA Control Station
eth2 may not be listed as this point. If not, dont worry, well add it momentarily.
lo: loopback
Page 15 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager Configure the NICs The following set of commands will configure the TCP/IP address information and reset the Virtual NICs accordingly. Type the following command: [root@localhost ~]# netconfig -d eth0 netconfig will launch a pseudo GUI to accomplish this task: Enter the appropriate values for each interface. eth0 will act as the NFS interface, eth1 will handle the iSCSI traffic and eth2 will be the Celerra VSA Control Station interface. This is the interface you will use for SecureShell (SSH) sessions. It is also the interface for the Celerra Manager GUI.
Repeat the process for eth1 and eth2 [root@localhost ~]# netconfig -d eth1 [root@localhost ~]# netconfig -d eth2 Once completed enter the following commands in ordered succession. [root@localhost [root@localhost [root@localhost [root@localhost [root@localhost [root@localhost ~]# ~]# ~]# ~]# ~]# ~]# ifdown eth0 ifdown eth1 ifdown eth2 ifup eth0 ifup eth1 ifup eth2 Page 16 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager Alternatively you could reboot the Celerra VSA (but that takes more time) using the command: reboot n from the system prompt: [root@localhost ~]# Validate the TCP/IP address information has been assigned correctly: [root@localhost ~]# ifconfig | less (check the IP addresses for proper configuration) Ping each of the interface addresses including the DART interfaces and ensure a positive response: [root@localhost [root@localhost [root@localhost [root@localhost [root@localhost ~]# ~]# ~]# ~]# ~]# ping ping ping ping ping <eth0 ipaddress> <eth1 ipaddress> <eth2 ipaddress> 128.221.252.2 Note this is the simulated data mover 128.221.253.2 Note this is the simulated data mover
Page 17 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager
[root@localhost ~]# cd /opt/blackbird/tools [root@localhost ~]# ls l (this displays the contents of the /opt/blackbird/tools directory)
This regenerates the Celerra VSA Serial number to match the VM UUID The Celerra VSA was created from a clone. This removes the clone Serial Number and replaces it with a unique identifier. This is a critical step as we will setup replication in a later step and each Celerra VSA instance in a replication partnership needs a unique ID
Once completed we need to update the host file using the following command: [root@localhost tools]# vi /etc/hosts
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager Insert an entry below the localhost entry as follows: 192.168.1.100 csprod1 csprod1.mydomain.com csprod1
Use the IP address you specified for the Celerra VSA Simulator Control Station interface (eth2)
save the file by hitting <Esc> then typing :wq! Now for the network file in /etc/sysconfig. This will ensure the host boots with the proper host name.
Add the following lines to the bottom of the existing file: DOMAINNAME=mydomain.com (in my case: baker-iss.com) HOSTNAME=csprod1 save the file by hitting <Esc> then typing :wq! Page 19 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager In order to save a reboot, issue this command: [root@localhost tools]# hostname csprod1 This changes the host name immediately. The previous two file edits (/etc/hosts and /etc/sysconfig/network) will ensure host name retention over a reboot. [root@localhost tools]# exit (may need to repeat this a couple of times to get back to the login prompt) csprod1 login:
Notice the host name has changed, it used to look like this; localhost login:
We will now login with the nasadmin user account. This account is used for all activities not related to the configuration of the Linux operating environment but to the actual Celerra VSA environment. Login to the Control Station (eth2 IP Address); user: nasadmin password: nasadmin
Page 20 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager
Notice that both cge0 and cge1 map to eth0; we will delete and re-add them to resolve this problem
[nasadmin@csprod1 tools]$ ./configure_nic server_2 -d cge0 (Delete cge0) [nasadmin@csprod1 tools]$ ./configure_nic server_2 -d cge1 (Delete cge1)
As of these commands execute the output shows the remaining configured cge interfaces. Since we have run this command twice the output from the second execution shows an empty list.
Page 21 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager
As a second confirmation execute the same command with the list option:
[nasadmin@csprod1 tools]$ ./configure_nic server_2 l
The output should confirm the deletion and non-existence of the previously defined cge interfaces. Now that the cge interfaces have been deleted and the NAS_DB has been exported, we need to reboot the Celerra VSA. When the system comes back up we will re-add these interfaces and map them to the appropriate eth interfaces.
[nasadmin@csprod1 tools]$ export NAS_DB=/nas
This is the Linux Switch User command. By default su assumes the current user is switching to the root user account. The - as in su forces the system to reload the new users profile information.
[nasadmin@csprod1 tools]$ su Password: nasadmin
This command will shutdown the Linux operating system, warm reset the virtual machine and restart the Linux operating system and Celerra VSA
[root@csprod1 tools]# reboot n
Page 22 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager
Note the boot process take approximately 20 minutes to complete. The system will appear to be up but the Celerra VSA (blackbird) service may not be fully operational. To check the status of the service issue the following command:
[nasadmin@csprod1 ~]$ /nas/sbin/getreason
When the status matches the above the Celerra VSA is up and fully operational. We now need to add the cge interfaces back to the configuration and map them to their appropriate eth interface. [nasadmin@csprod1 ~]$ cd /opt/blackbird/tools [nasadmin@csprod1 ~]$ ls l (this displays the contents of the /opt/blackbird/tools directory) [nasadmin@csprod1 tools]$ ./configure_nic server_2 -l
Notice the list is empty. It should look just as it did prior to the reboot.
[nasadmin@csprod1 tools]$ ./configure_nic server_2 -a eth0 (maps eth0 to cge0) [nasadmin@csprod1 tools]$ ./configure_nic server_2 -a eth1 (maps eth1 to cge1)
Page 23 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager [nasadmin@csprod1 tools]$ ./configure_nic server_2 -l
Notice the list now reflects appropriate mapping between cge0/eth0 and cge1/eth1
We need to reboot:
[nasadmin@csprod1 tools]$ su Password: nasadmin [root@csprod1 tools]# reboot -n
Page 24 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager
Remember to wait for the blackbird service to finish loading. To check use the following command: [nasadmin@csprod1 ~]$ /nas/sbin/getreason Once the service had completed loading issue the following commands: [nasadmin@csprod1 ~]$ nas_cel -list Output refers to the hostname (name) as localhost and not csprod1 the net_path may be pointing to the lo (loopback) interface as 127.0.0.1. Well correct this using the next series of commands. First thing we need to do is su to the root user as we discussed a moment ago. What is different now is that we will not pass the - argument to the su command as we want to maintain the nasadmin profile, specifically the file system pathing. [nasadmin@csprod1 ~]$ su (This is not a typo. The command here is su not su -) Password: nasadmin Now that the network interfaces are properly configured and the host name is set properly, the internal Celerra database must be updated. To do so execute the next command. [root@csprod1 ~]# nas_cel -update id=0
Operation in progress (not interruptible) Ignore the warning about the loopback interconnect and not that this step may take a minute or two to complete
[root@csprod1 ~]# nas_cel -list
Page 25 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager As we continue these housekeeping tasks we need to update the SSL certificates associated with the Celerra VSA to avoid errors on the Celerra VSA's web management interface. To do this, enter the following commands: [root@csprod1 ~]# /nas/sbin/nas_config -ssl
The new SSL certificate has been generated and installed successfully.
Time to reboot the VM again. Since we executed the su command earlier we dont need to do it again here. Simply type the reboot command and hit Enter. [root@csprod1 ~]# reboot -n
Page 26 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager
Step 5: Licensing
Login to the Control Station (eth2 IP Address); user: nasadmin password: nasadmin
Remember to wait for the Celerra VSA service to finish loading. To check use the following command: [nasadmin@csprod1 ~]$ /nas/sbin/getreason First we need to initialize the Celerra Licensing Database [nasadmin@csprod1 ~]$ nas_license -init
done
Now we can enable the specific licenses available with in the Celerra Manager. It is important to note that EMC does not require the actual purchase or installation of a license. The Celerra VSA is enabled for all protocols and advanced features. Please recall using this platform for anything other than learning, testing or development is in violation on the license agreement. Launch each of these commands in succession. Each command will signify a successful application of the feature with the message: done [nasadmin@csprod1 ~]$ nas_license l The nas_license l command will output the site_key number only. Next time we run this command it will list the site_key and all of the applied licenses.
Page 27 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager
Page 28 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager As an option, you could have performed the licensing task from the Celerra Manager GUI. if you are so inclined the steps are as follows: Open a web browser to the running Celerra VSA VM Management interface: https://csprod1 Best Practice: Add your Celerra VSAs to a DNS server in your environment. If DNS is not an option, define your machine name is short and FQDN format in your host file. Do the same for the Celerra VSA use as the disaster recovery target in the recovery site.
The Celerra Manager requires Java. Java will be installed for you if it is not present on your machine. Of course this requires a connection to http://www.sun.com for the installation bits. From my personal experience Internet Explorer 7 will always indicate an issue with the SSL certificates issued during connection even after the SSL configuration has been modified. Internet Explorer 6 works great as does Firefox as long as Java is installed properly and operational. I used Firefox 3.0.4 in most cases.
Login with the following credentials; Username: nasadmin Password: nasadmin
Page 29 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager
If you get an HTTP 503 Error: logout, wait a few moments and log back in. This error is caused as a result of incomplete initialization of web services within the Celerra VSA. Give it a couple of minutes to complete. If you get a cookie error there may be a DNS issue. Resolve the DNS issues or use a Fully Qualified Domain Name (FQDN) or TCP/IP addresses. Example: FQDN: TCP/IP: http://csprod1.baker-iss.com http://192.168.1.246 - this was the TCP/IP address of my csprod1
If you receive a secure connection failed error while using the Firefox Web Browser you may need to delete the cached certificate. This may happen if you run thru the document more than once or connect to a Celerra VSA with the same name. (It may happen in a lab. I happen to me and at least on of the document testers. Navigate to Tools, Options, Advanced, Encryption. Click View Certificates, Authorities. Scroll to the certificate in question, Select said certificate and Click Delete. Click OK
Page 30 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager When presented with the Celerra Message of the Day (MOTD) Click OK. You have the option to not see the additional MOTD pop-ups until the message changes. There is a check box on the panel. From the Celerra Manger Main Page: Select the Licenses tab Select the licenses one at a time. Click Apply, Click OK Celerra Manager Advanced Edition License NFS Licensed CIFS Licensed iSCSI Licensed SnapSure Licensed (CIFS, NFS, iSCSI Snapshots) ReplicatorV2 Licensed (iSCSI and Fibre Channel Only NFS not supported)
Page 31 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager
The TCP/IP address can not be the same as eth0 even though they bind to the same physical interface
Perform these steps again for cge1. (This will map to eth1; your iSCSI interface)
Page 32 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager
Note: This is not explicitly required but most folks want to play with more storage the 25GB available in the Celerra VSA by default. Ive added these steps so you have the option if you choose to add capacity. Of course you must have available physical capacity to support the additional virtual capacity in the Celerra VSA. This storage can be in the form of Direct Attached/Internal Server Storage (DAS) or iSCSI/FibreChannel (SAN) based storage.
This can be accomplished via CLI within the VM or via Virtual Center
Take a snapshot of the Celerra VSA VM from Virtual Center Right Click on the Celerra VSA VM and Select Edit Settings Add a new VMDK to the Celerra VSA VM, make it an appropriate size
Best practice: add many small VMDKs vs. one large one. Each VMDK file will be associated with a physical disk partition. If you every need to run files system check (fsck) on one of these physical partitions the process will be less intrusive and will typically require less time. Note: Maximum size of a physical partition/virtual disk file is 2,048 GB (2 TB)
Power On Celerra VSA VM Page 33 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager
Remember to wait for the Celerra VSA service to finish loading. To check use the following command: [nasadmin@csprod1 ~]$ /nas/sbin/getreason [nasadmin@csprod1 ~]$ su When prompted enter the root users password: nasadmin [root@csprod1 ~]# dmesg | grep sd This command will search device message and provide an indication of the added storage device address.
We need to create a partition on the newly added disk. To do this, enter the following command: [root@csprod1 ~]# fdisk /dev/sdb
Page 34 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager
The device number/letter (/dev/sd#) will change based on the number of attached disks
At the prompt; Command (m for help): Type n for new partition Enter p for primary partition (without the quotes) Enter 1 for partition number Take the defaults for First Cylinder and Last Cylinder Command (m for help): w (write partition table to disk)
The next command assumes the existence of the directory /mount. If /mount doesnt exist issue the following command first; mkdir /mount
[root@csprod1 ~]# mkdir /mount/<insert your desired name here>
This process will take a few minutes to complete. The operating system is formatting the disk Remember this will change based on your device name. In this case it is /dev/sdb1 Repeat the previous three steps for each new disk added to the virtual machine. Subsequent device addressed would be /dev/sdc, /dev/sdd, etc.
Page 35 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager Edit the fstab file: [root@csprod1 ~]# vi /etc/fstab Add the following line to the bottom of the existing file and save the file (<esc> :wq!) /dev/sdb1 /mount/sdb1 auto defaults 0 0 (these are zeros)
These may also be added for other drives added to the virtual machine. These are not needed if you only added one additional disk. Furthermore, only add one entry per added disk /dev/sdc1 /dev/sdd1 /mount/sdc1 /mount/sdd1 auto auto defaults defaults 00 00
[root@csprod1 ~]# mount a [root@csprod1 ~]# df h (verify the newly mounted file system is listed) [root@csprod1 ~]# export NAS_DB=/nas
Page 36 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager
The initial output from this command presents the serial number of the virtual CLARiiON backend storage array. Please note this number: The format starts with BB
Because the Celerra VSA was created from another virtual device we need to ensure the serial number and the associated log .xml files match this number. To accomplish this execute the following commands: Note the BBxxxxxx.xml file used in the example below should match that of the serial number just noted above.
[root@csprod1 ~]# cp /nas/log/backend_status.BBxxxxxxxxxxxx.xml /nas/log/backend_status.<INSERT_YOUR_SERIAL_HERE>.xml [root@csprod1 ~]# cp /nas/log/backend_resume.BBxxxxxxxxxxxx.xml /nas/log/backend_resume.<INSERT_YOUR_SERIAL_HERE>.xml
If the previous two copy commands are not executed the following steps will fail. Tip: Use the up arrow to load the second command from the CLI history and just change the words status to resume. Of course leave the quotes out.
Page 37 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager [root@csprod1 ~]# /nas/sbin/setup_clariion init
Notice the initial configuration defaults include two virtualized disk storage shelves. This graphic shows a third as well. I previously configured the new shelf but forgot to take the screen shot so we actually see three here.
Select Option 1. Add New Enclosure Accept defaults for Enclosure [2_0], Physical Disk Size [146] and Physical Disk Type [FC] Select Option 3, Continue to Diskgroup Template Menu Select Option 1, CX_Standard_Raid_5 Type yes to continue when asked: Do you want to continue and configure as shown? When prompted to enter pathname for disk storage location enter the mount point you defined earlier: i.e., /mount/sdb1 Next prompt, Select 1. Create LUNs using available storage size in this path (xx MB in total) Type no when asked about zeroing all LUNS
Note: there is a slight performance benefit when zeroing the LUNS up front but it takes a really long time to complete. The Celerra VSA will now create the enclosures, hot spares, LUNs and disk groups
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager
Remember to wait for the Celerra VSA service to finish loading. To check, use the following command: [nasadmin@csprod1 ~]$ /nas/sbin/getreason [nasadmin@csprod1 ~]$ server_devconfig server_2 p s a
You will notice a number of LUNs where the val= -99: This indicates an unsigned LUN
[nasadmin@csprod1 ~]$ server_devconfig server_2 c s a
The new LUNs are now ready for use. We see these new LUNs as we configure file systems on top of these in a few moments.
[nasadmin@csprod1 ~]$ logout Page 39 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager
Section 3: Review
At this point you should have two fully configured Celerra VSAs installed and configured with additional disk, iSCSI and NFS LUNs. You may recall from the opening statements that we would not walk thru DR site Celerra VSA implementation. The DR Celerra VSA is a mirror image of the primary site Celerra. The only differences relate to the networking configuration. Using the previous steps in Section1 and Section 2 and replacing all references to csprod1 with csdr1 you will successfully configure the DR Celerra VSA. Remember the DR Celerra VSA is installed on the ESX host designated for DR in your test environment. In the next section, Configuring Celerra Replication, we will connect the two Celerra VSAs and replicate LUNs between them. Once we complete the replication configuration we will install and configure Site Recovery Manager, setup Protection and Recovery Groups and exercise the tool in test and failover scenarios in Section 5.
If you have not already done so, please setup, install and configure the Celerra VSA for the recovery site.
Page 40 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager
Site Recovery Manager (SRM) doesn't support NFS yet, so this section is focused on iSCSI - but it's very easy to see how you would do CIFS/NFS by selecting the appropriate radio button during configuration. Contained within this section are the following items: Configuring NTP Correcting the Celerra VSA Replication Database
The ID associated with the Replication Database needs to be unique. As a result of the import process the database still maintains the original ID it had prior to cloning. Therefore, two Celerra VSAs replicating between each other would potentially have the same ID which will inhibit replication setup. The process of correcting/changing the ID will avoid this problem
Configuring an iSCSI LUN that will be used as a replication target Configuring Celerra Replicator
My configuration was completed in a local datacenter. All connections and communications where limited to that location. If you want to perform these steps over your WAN or to a network protected by a firewall you should be aware of the following:
Port 8888 is used by Celerra Replicator for transferring data between Celerra VSAs The HTTPS connection between the source and destination Data Movers take place on port 5085 The HTTPS connection between the Control Station on the source site and the Control Station on the destination site happens on port 443
Page 41 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager
Click Control Station Properties tab Modify the Current Date and Time, Current Time zone and NTP Servers: fields accordingly
Note: Changing the Timezone may require a reboot of the Celerra VSA
Page 42 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager Open the Data Movers Folder and Right Click, Properties on server_2 Verify the date and time are in sync with the control station If the time skew is too great to sync with the NTP server you specified in the GUI, follow this sequence:
o o o o o o o
Login to the Celerra VSA Console as root [root@csprod1 ~]# date (verify current date and time) [root@csprod1 ~]# pgrep ntpd (ensure the NTP daemon is running) [root@csprod1 ~]# chkconfig ntpd on (ensures the NTP starts with the VSA) [root@csprod1 ~]# service ntpd stop [root@csprod1 ~]# ntpdate u <NTP server IP address> Here are a couple of Internet based NTP server addresses for reference: 129.6.15.28 (time-a.nist.gov) 129.6.15.29 (time-b.nist.gov) 128.138.140.44 (utcnist.colorado.edu)
o o o
[root@csprod1 ~]# service ntpd start [root@csprod1 ~]# date (verify current date and time are correct) [root@csprod1 ~]# logout
Login to the Celerra VSA Console as nasadmin PW: nasadmin [nasadmin@csprod1 ~]$ server_date server_2 timesvc delete ntp [nasadmin@csprod1 ~]$ server_date server_2 timesvc start <NTP server IP address> [nasadmin@csprod1 ~]$ server_date server_2 [nasadmin@csprod1 ~]$ logout
If the times are still out of sync after following these steps reboot the entire Celerra VSA
Repeat these steps for the recovery site Celerra VSA (csdr1)
Page 43 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager
Scroll to bottom of the file and comment the dpinit line (comment character = #) Use the Insert Command (i) from the beginning of the dpinit line and type the hash (#) key. Save the file: (<esc> :wq!)
Page 44 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager
Scroll to middle of the file and comment the dpinit line (comment character = #) Use the Insert Command (I) from the beginning of the dpinit line and type the hash (#) key. Save the file: (<esc> :wq!)
[root@csprod1 slot_2]# server_cpu server_2 reboot now [root@csprod1 slot_2]# /nas/sbin/getreason (will delver status on the data mover reboot)
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager
replication database)
database Dont forget the . at the beginning of this command. It is not a typo)
[root@csprod1 slot_2]# nas_cel list [root@csprod1 slot_2]# nas_cel update id=0 (zero)
(The CMU now correctly correlates to the Celerra VSA serial number)
Page 46 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager
Scroll to bottom of the file and uncomment the dpinit line (remove the #) Save the file: (<esc> :wq!)
Scroll to middle of the file and comment the dpinit line (remove the #) Save the file: (<esc> :wq!)
Page 47 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager
Page 48 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager
Click Next to Select server_2 (default), Click Next to Select Storage Pool (default)
Page 49 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager Select the Storage Pool capable of providing capacity for the iSCSI LUN. In my case, I am using clar_r5_economy. Click Next
Enter a name for the file system. I used iscsi_fs1_replicated as I plan to replicate this LUN to csdr1. Enter the size of the LUN in MB (1024MB = 1GB). Ensure the Slice Volumes check box is selected. Click Next
Page 50 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager For the purposes of this exercise, take the defaults on the Enable Auto Extension Panel and the Default Quota Settings Panel by clicking Next
Page 51 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager When completed. The Panel will look similar to this one. Click Close
Repeat this process for csdr1. This will ensure the Celerra VSAs are configured for bi-directional replication which makes the process of fail back and reconfiguration that much easier. If you choose not to do this than proceed to the next step. One you have completed the file system creation on csdr1 well add an iSCSI LUN and iSCSI target on csprod1. Well repeat the process on csdr1
Page 52 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager
Page 53 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager During this step the Create iSCSI LUN wizard will create the iSCSI target and the iSCSI LUN associated to said target. Click Next to select server_2 (default), on the Select/Create Target panel, Click Create Target
Enter a Target Alias Name (I am using csprod1_rep_target1), Ensure the Auto Generate Target Qualifier Name check box is checked. Click Next
Page 54 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager Select the iSCSI interface created earlier (cge1), Click Add, Click Next
Page 55 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager Select the iSCSI target just created, Click Next
On the Select/Create File System panel; select the file system created for replication earlier. Click Next
Page 56 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager Enter the LUN information. Click Next
Note: When configuring this step on csdr1 the LUN must be defined as read-only
Here is the same screen for csdr1. Notice the LUN sizes are identical.
Page 57 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager Add the iSCSI initiators for the ESX servers that will access the replicated LUN Enable Multiple Access if more than one ESX will access this LUN Click Add New, Add the IQN number from your production ESX server, Click OK Repeat this for each ESX server accessing this LUN
When completed, Click Next on the main Main LUN Masking panel Click Next on the CHAP Access (Optional) Panel
Page 58 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager Assuming you have not added any iSCSI LUNs to this Celerra VSA you will be prompted with the following panel: Ensure the iSCSI service check box is selected, Click Next
Of course this needs to be completed on csdr1. When completed move on to Configuring Replication
Page 59 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager
Configuring Replication
Select New Replication from the Wizards folder
Page 60 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager All of our conversation thus far has centered on iSCSI. To that, we will select the iSCSI LUN radio button and Click Next
Page 61 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager Enter the TCP/IP address and the user credentials of csdr1 The username is nasadmin; password is nasadmin Click Next
Enter a Celerra Network Server Name: Your choice Enter a passphrase: Your choice Click Next
Page 62 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager Click Submit and view the following screen
Page 63 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager Select the Celerra VSA Destination just created. In my case it network name was cns.
Page 64 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager Name the data mover interconnect. I used csprod1-to-csdr1, Click Next
The next panel is asking you to define an interconnect for the peer data mover. This is the interconnect the allows for reverse communication between the Celerra VSAs I used csdr1-to-csprod
Page 65 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager Take the defaults on the next screen, Click Next
Ensure the time on the Celerra VSA control stations is within 10 minutes of each other. Double-check prior to clicking submit. Are you sure? Click Submit
Page 66 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager Review the results of the changes and Click Next
Page 67 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager Select the iSCSI interfaces for each of you Celerra VSAs, Click Next
The next couple of items will name the replication session and select the specific LUNs involved in this session. Since this session will facilitate replication for VMware Site Recovery Manager, I named the session srm_replication. Notice the iSCSI target and iSCSI LUN selections. These were there by default. Double-check the entries for accuracy prior to clicking Next
Page 68 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager If the Target Destination information is correct Click Next. Otherwise select the correct parameters and then Click Next
Page 69 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager Click Finish
Page 70 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager Once completed. Navigate back to the Celerra Manager Main Panel. Click on the Replications folder. Do this from both sides of the replication relationship. csprod:
csdr1:
Page 71 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager
Page 72 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager Click on the Security tab Change Promiscuous Mode to Accept
Note: This is NOT required for the Celerra Simulator as it will run either way. This is a best practice as it related to most simulators
Click OK, Click Close From the main panel: Click on the Security Profile. Ensure that the Software iSCSI Client is enabled
Page 73 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager Move to the Storage Adapters Tab and scroll to the iSCSI Software Adapter.
Note: Two items: 1) the iSCSI Initiator IQN, 2.) Targets: currently 0 (zero)
Click Properties, Click Configure, Select Enabled
Enter the TCP/IP address of the iSCSI target you created earlier. Ensure the port number is 3260 (default) Click OK (This may take a moment or two...) Click Close Page 74 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager When prompted to Rescan the host Click Yes Accept the defaults and Click OK
Page 75 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager
Notice the new Target. Repeat this step for each ESX server masked to the replicated iSCSI Target
To this point all we have done is allow the ESX servers to see the iSCSI LUNs. We must establish a VMFS file system so they are usable within the Virtual Infrastructure.
The steps on the proceeding two pages are for the Production side ESX servers only!
Page 76 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager Click on the Configuration tab, Select Storage, Click Add Storage
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager
Review the current disk layout and notice the message: The hard disk is blank.
Click Next Enter a Name in the Datastore Name field:
Important: Add or migrate at least one virtual machine you want to protect with SRM to the replicated datastore. If not the SRM configuration will not display any summary information in the Display Replicated Datastores later in this configuration. Please add at least one VM to this new datastore. Also only one LUN is replicating between the Celerra VSAs. You can create additional LUNS following the process above. Page 78 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager
Page 79 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager
Within your Active Directory create a new user: I used srmprotected-db. Notice I selected the User cannot change password and Password never expires options. Click Create
Once created launch the SSMSEE application: Start> all Programs> Microsoft SQL Server 2005 > SQL Server Management Studio Express The application may or may not display the following panel:
Page 80 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager From the SQL Server 2005 Connect to Server Panel, Click Connect
Make note of the Server Name: in my case VMWARE-VC\SQLEXP_VIM. You will need this information when you create an ODBC connection to the SRM database.
For some reason it will not be on the drop down list so I copied it to my clipboard.
In the main panel Right Click Databases and Select New Database. Name the Database as you wish. I used srmprotected-db. Click OK
Page 81 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager Create a new login: Right Click on the Logins Folder under Security
Enter the login ID you created for this purpose. Enter the name in domain\user format: i.e., BAKER-ISS\srmprotected-db Select the database you created from the Default Database Drop Down
Page 82 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager Next select User Mappings from the Select a Page panel. Select the database you user will be mapped to (i.e., srmprotected-db), Select db_owner and leave public checked. Click OK
Now that the database has been created and the user mapped to it, we need to create an ODBC connection to the new database. Navigate to the Control Panel on the Virtual Center Server. Double Click on the Data Sources (ODBC) shortcut. Click Add
(If the ODBC setup icon is not in the Control Panel it will be in the Administrative Tools folder. You may need to enable this via the start menu properties.)
Page 83 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager Select SQL Native Client, Click Finish
Name the ODBC Connection to the SRM database a name and enter the server name on the Server: field as noted above i.e., in my case VMWARE-VC\SQLEXP_VIM, Click Next
Select the With Integrated Windows authentication radio button, Click Next
Page 84 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager Ensure Change the default database to: selection is checked Select the SRM database you created earlier from the dropdown box. Keep the rest of the selections at their defaults. Click Next
On the next panel take the defaults and Click Finish Once completed you will be presented with the following panel:
Page 85 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager If everything was configured properly you should see the following panel (a small reward for the work so far!). ClickOK
Page 86 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager Click OK again when presented with the ODBC Data Source Administrator Panel
At this point you should configure SQL on the recovery site Virtual Center Server.
Page 87 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager
Accept the ELUA and Select a Destination Folder, Fill in the appropriate information in the SRM registration panel.
Page 88 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager Acknowledge the Security Warning and accept the certificates thumbprint by Clicking Yes
Select the Automatically generate a certificate radio button and Click Next
Enter the appropriate (anything you like) information in the Organization and Organizational Unit fields and Click Next Fill in the requested information and Click Next. Do not modify the SOAP or HTTP ports
Page 89 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager Enter the database specifics in the next panel and Click Next
Click Install The installation will proceed without any additional input. Once completed Click Finish and proceed to the next step
Page 90 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager
Page 91 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager Click the Download and Install button for Site Recovery Manager
At this point a separate installer will launch to install the plug-in. This is a standard windows install similar to the installs completed in Steps 2 and 3 of this section.
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager Shortly after the completion of the add-in install, you will be taken back to the plug-in manager for Virtual Center. Notice the plug-in has been installed but still needs to be enabled. Click on the Installed tab and Click Enabled in the Site Recovery Manager panel, Click OK
Page 93 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager This panel offers us the ability to start configuring the actual linkages between SRM servers and their associated storage devices. Prior to that however, the SRM installation needs to be completed on the recovery, disaster recovery site. Please complete Section 5 on the recovery site Virtual Center server before proceeding to Section 6.
Page 94 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager
Page 95 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager
Page 96 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager Enter the Administrative Credentials for the Virtual Center Server. You may be asked to authenticate to both the remote and the local servers.
Once Authenticated the system will establish a reciprocal connection between the Virtual Center Servers. Acknowledge any Security Certificate warnings (you may not see any)
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager Check the status of the connection. It should be Connected
Page 98 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager
Page 99 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager Enter the requested information in the Add Array Manger Panel. The display name should be the same as your protection site Celerra VSA. Manager Type is Celerra iSCSI Native, Control Station IP is the csprod1 eth2 IP address, User Name and Password are nasadmin. Click Connect
Once added, the replicated LUNs on csprod1 should be displayed in the Protection Arrays Panel Click OK
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager The Add Array Manager process will return to the Configure Array Manager main panel. Notice he Protection Arrays: panel. It lists the Celerra VSA as NS-Simulator, shows the Peer array at the remote site and the number of LUN(s) configured for replication
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager
Enter the requested information in the Add Array Manger Panel. The display name should be the same as your protection site Celerra VSA. Manager Type is Celerra iSCSI Native, Control Station IP is the csdr1 eth2 IP address, User Name and Password are nasadmin. Click Connect
ClickNext
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager Review the Replicated Datastores, Click Finish
Remember: If you did not add at least one virtual machine to the replicated datastore this panel will be blank That is OK but dont think something is wrong if the datastore summary is blank.
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager
Use this panel to map source site resources to recovery site resources. These are the recovery site resources used during testing and failover. Specifically, you are mapping network, resource pools and data center locations.
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager Example:
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager
Click Create next to Protection Groups in the Protection Setup Panel Give the Protection Groups a Name and Click Next
The name should give some insight to the VMs that will be contained within.
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager Select a Datastore Group and Click Next (as discovered by the Array Manager)
Select a recovery site datastore for the shadow/placeholder Virtual Machines, Click Finish
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager We now see the created Protection Group
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager
Name the Recovery Plan use something indicative of the Plans scope (entire site, windows machines, Linux machines, web servers, etc.), Click Next
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager In the Create Recovery Plan - Protection Groups dialog select the Protection Group(s) created in the previous step. Click Next
In the next panel, Response Times, leave the defaults and Click Next
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager The Configure Test Networks dialog provides the ability to select a test network for use when testing your recovery plan. By default this is set to Auto. Auto will dynamically create a bubble network during a running test and tear it down when the test completes. Note: Any virtual machines dependant on non-protected machines running DNS, AD and DHCP may not be able to communicate with each other once on the bubble network. Make provisions for these services in the bubble or configure an isolated network with these services for testing purposes. Make the appropriate selections. Click Next
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager It is likely, during a test, that you may need to free-up some of the available DR site resources (CPU and Memory) until the test cycle is completed. SRM provide this capability via the recovery plan. If you choose to, SRM will suspend selected virtual machines. Click the check box next to the VMs you wish to suspend. Click Finish
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager The steps listed here have been automatically created via the Create Recovery Plan.
Take a few moments to review the steps in the recovery plan. You have many options to tweak the plan but that is slightly outside the scope of this document. When you are ready to proceed Click on the Green Test Button (The Red Run Button is reserved for the actual failover event!) Acknowledge the following dialog
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager SRM will start by configuring the storage components. Without a long dissertation, SRM will ready the remote storage and present a writeable snapshot to the ESX server in the DR site.
Notice the status in the recent task panel at the bottom of the main Virtual Center window.
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager The next step in the process will suspend any Virtual Machines (if they where selected to do so during the creation of the Recover Plan)
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager Next the plan starts recovering, in order of priority, the virtual machines (these are the place holder VMs created during the Protection Groups configuration)
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager Once the VM have been recovered and all of the recovery plan steps have been completed, the plan will pause for user input. During this pause the virtual machines are running and connected to the network previously defined in the recovery plan. Remember, by default this network is referred to as the bubble network and maintains no connectivity to other physical networks and does not provide for network-based services such as AD, DNS or DHCP. This is unless you have made provisions ahead of time. Testing complete? Great! Click Continue
Site Recovery Manager will clean up the storage, place the VMs back into a powered off state in the placeholder datastore and return to the Recovery Plan main panel. You have just executed a successful Site Recovery Manager test using Celerra VSAs. How did it go? It is a lot of work but this document in conjunction with the Celerra VSAs allow for the creation of an environment fully capable of testing SRM and a very real manner. Please let me know if you found this document useful. I would also welcome feedback on the accuracy of the content. Thanks for taking time to look at the Celerra VSA and testing VMware Site Recovery Manager in this manner. Page 118 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager csdr1: [nasadmin@csdr1 ~]$ nas_fs -name fs1-csdr1 -type uxfs -create size=45G pool=clar_r5_performance -option slice=y [nasadmin@csdr1 ~]$ server_mount server_2 fs1-csdr1 /nas/fs2
Start the iSCSI service csprod1: [nasadmin@csprod1 ~]$ server_iscsi server_2 -service -start csdr1: [nasadmin@csdr1 ~]$ server_iscsi server_2 -service -start
Create an iSCSI target that will be used to serve LUNs to each Celerra VSA csprod1: [nasadmin@csprod1 ~]$ server_iscsi server_2 -target -alias target1-csprod1 -create 1:np=172.16.1.150 (This is the iSCSI interface on you created in Section 1-eth2) csdr1: [nasadmin@csdr1 ~]$ server_iscsi server_2 -target -alias target1-csdr1 -create 1:np=172.16.1.160 (This is the iSCSI interface on you created in Section 1-eth2)
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager
Note: In order to setup reverse replication, from secondary to primary, a second set of LUNs needs to be created. This time the target replication LUN on the Primary must be set to read-only. The source LUN on the secondary is configured normally as a read/write LUN.
Set the iSCSI LUN masking for each ESX host csprod1: [nasadmin@csprod1 ~]$ server_iscsi server_2 -mask -set target1-csprod1 -initiator iqn.1998-01.com.vmware:esx2-29d419e2 -grant 1 csdr1: [nasadmin@csdr1 ~]$ server_iscsi server_2 -mask -set target1-csdr1 -initiator iqn.199801.com.vmware:esx3-017f4fc5 -grant 1
Note: The IQN number is associated with the software iSCSI interface on the ESX host. You will need to retrieve this from the ESX server via Virtual Center as noted earlier.
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager
Configuring Replication
Set up mutual trusts between Celerra VSAs This trust needs to be setup in both directions; primary to secondary and secondary to primary. The passphrase must be the same in both cases: csprod1: [nasadmin@csprod1 ~]$ nas_cel -create cel1 -ip 192.168.1.160 passphrase nasadmin csdr1: [nasadmin@csdr1 ~]$ nas_cel -create cel1 -ip 192.168.1.150 passphrase nasadmin
Configure the Data Movers to converse and share replicated data csprod1: [nasadmin@csprod1 ~]$ nas_cel -interconnect -create cel1-to-cel2 -source_server server_2 -destination_system cel2 -destination_server server_2 -source_interfaces ip=172.16.1.150 -destination_interfaces ip=172.16.1.160 csdr1: [nasadmin@csdr1 ~]$ nas_cel -interconnect -create cel2-to-cel1 -source_server server_2 -destination_system cel1 -destination_server server_2 -source_interfaces ip=172.16.1.160 destination_interfaces ip=172.16.1.150
Note: Perform this interconnect from both directions; primary to secondary and secondary to primary. If you only set it from one side replication will only work in one direction. The interconnect must be the same in both cases.
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager Establish Replication from the primary Celerra VSA to the secondary Celerra VSA csprod1: [nasadmin@csprod1 ~]$ server_iscsi server_2 target l
This will provide the Celerra VSA IQN number for the next command
csdr1: [nasadmin@csdr1 ~]$ server_iscsi server_2 target l
This will provide the Celerra VSA IQN number for the next command
csprod1 (only): [nasadmin@csprod1 ~]$ nas_replicate -create rep1 -source -lun 1 -target iqn.199205.com.emc:bb0050568a59e40000-3 -destination -lun 1 -target iqn.199205.com.emc:bb0050569a3d070000-3 -interconnect cel1-to-cel2 -source_interface ip=172.16.1.150 -destination_interface ip=172.16.1.160 -overwrite_destination csprod1: [nasadmin@csprod1 ~]$ nas_replicate -l csdr1: [nasadmin@csdr1 ~]$ nas_replicate l
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager
Login to your Celerra VSA as nasadmin, Password: nasadmin Navigate to the Wizards Folder Click New iSCSI Target, Click Next to configure an iSCSI target on server_2 Enter the Target Alias Name: csprod1_target1 and select Auto Generate Target Qualified Name
Note: This will generate a Unique IQN number for this target.
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager Assign the target to cge1 (your iSCSI Interface), Click Next Click Finish This graphic represents the complete target configuration as viewed from the iSCSI folder, Targets tab.
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager
Step 2: Configuring iSCSI LUNs and presenting them to ESX as VMFS Datastores
You may not need to complete this step (outlined on pages 118-121) if you already configured your environment for iSCSI replication. These steps have already been completed. If you did not set up replication please continue.
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager Select vSwith, Click Edit
Note: This is NOT required for the Celerra Simulator as it will run either way. This is a best practice as it related to most simulators
Click OK, Click Close Page 127 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager From the main panel: Click on the Security Profile. Ensure that the Software iSCSI Client is enabled
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager Move to the Storage Adapters Tab and scroll to the iSCSI Software Adapter.
Note: Two items: 1) the iSCSI Initiator IQN, 2.) Targets: currently 0 (zero)
Click Properties, Click Configure, Select Enabled Copy the iSCSI Name field to the clipboard. You may want to write it down as well. Just in case you copy something else to the clipboard.
Click OK Click on the Dynamic Discovery tab, Click Add Enter the TCP/IP address of the iSCSI target you created earlier. Ensure the port number is 3260 Click OK (This may take a moment or two...) Click Close When prompted to Rescan the host Click No Repeat this step for each ESX host that will attach to this target.
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager
Click New iSCSI LUN, Click Next to configure an iSCSI target on server_2
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager Pick the iSCSI target created earlier (csprod1_target1), Click Next
Click Create File System (remember, like all filers, iSCSI LUNs are files in file systems)
Select the Storage Pool radio button. On the next screen select the specific Storage Pool you want to use. Two pools are defined by default; one for Performance and one for Economy. Page 131 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager
Click Next, Enter the name for the new file system: iscsifs1 Enter the size of the new file system: enter an appropriate value in MB Click Next, Click Submit wait for the job to complete, review the summary results Click Next
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager Pick the file system just created from the pick list windows (it may be highlighted) Click Next
There are several options presented on this page: execute the options required for your specific use case. For the purpose of this document we'll focus on creating a single LUN
Enter the Number: (This is the LUN number assigned to the newly created LUN) i.e., 10 Enter the size of the new LUN
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager Click Next Click Enable Multiple Access check box so it is selected Click Add New Paste the ESX Host IQN number you copied earlier.
Repeat for all ESX Hosts, Click Next, Click Next on the CHAP Access Panel Click iSCSI Service check box so it is selected. Click Next Click Finish Wait for the job to process, review the summary for errors Click Close
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager
Notice the Targets count has incremented by the number of LUNs associated with the target and masked to this server. In this case, maybe one or two LUNs..
Perform these steps for all ESX hosts in the cluster.
To this point all we have done is allow the ESX servers to see the iSCSI LUNs. We must establish a VMFS file system so they are usable within the Virtual Infrastructure.
Page 135 of 146
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager Click on the Configuration tab, Select Storage, Click Add Storage
Select the Disk/LUN radio button, Click Next Select the LUN just added (i.e., LUN10 which may be display as vmhba32:0:10) Click Next
Review the current disk layout and notice the message: The hard disk is blank.
Click Next Enter a Name in the Datastore Name field: Use something obvious like csprod1_iscsi_lun10 Click Next
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager
Step 3: Configuring NFS LUNs and presenting them to ESX as NFS Datastores
Note: This step is not required for SRM but is included here for completeness of configuration and protocol options.
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager
Select the Storage Pool radio button. On the next screen select the specific Storage Pool you want to use. Two pools are defined by default; one for Performance and one for Economy.
Click Next
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager Enter the name for the new file system: nfsfs1
Enter the size of the new file system: enter an appropriate value in MB Click Next, Next, Next
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager Back on the main panel: Click NFS Exports, Click New
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager Select the appropriate file system; i.e., nfsfs1 (/nfsfs1) Enter TCP/IP Addresses for each ESX host that will access this export in the Root Hosts: field
Note: this field accepts wild cards and CIDR formatted entries
Click OK
Celerra Manager will finish the task and present a list of the NFS exports as confirmation.
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager In the Properties panel enter the Server: name or IP and the Folder: that server is exporting: Example: Server: 192.168.1.247
Review the properties, Click Finish Verify the LUN was added to the ESX host.
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager
To start vi, all you need to do is type "vi" (minus the quotes, always minus the quotes from here on through) Lets say I wanted to open a file named foo.txt that resides in the same directory I am currently in. I would type "vi foo.txt" Now lets say i wanted to open inetd.conf which lives in the /etc directory. I would type "vi /etc/inetd.conf". The directory path to inetd.conf is always the same, no matter what directory you are in.
Quitting VI
First lets make sure you are in what is called "command mode". We do this by pressing the Esc key. Now we press " : ". You should see a little prompt at the bottom of your page. Now press "q" and hit enter. If it will not let you quit, it is because you have made some changes to the file and you must press ":", then "q!", then enter to force vi to quit (warning, this will disregard any changes you have currently made). If you would like to save the changes you have made within you vi session press ":", then "wq", now press enter. If vi ever complains that you're trying to make changes to a read-only file, try "w!" or "wq!". If you ever have any problems with doing this or other "command mode" commands hit the Esc key once or twice to make sure you are in command mode and try again.
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager Basic command mode commands for inserting text
First make sure you are in command mode by pressing the Esc key and then make sure you press ":" before you try any of the commands bellow. Please note that after you enter one of these commands in command mode you will be put into insert mode, which will enable you to enter new text into the file. In order to get back to command mode you will have to press the Esc key again. c - Add changes to file. This is a commonly used command for editing a file with vi. When this option is invoked, you rewrite over the text you want to change by backing up over it using the del key. Simply re-type over the text you have backed up over. When you use the left arrow key to back up over text you can not rewrite over the text you backed up over, but just insert new text at the point where you stopped backing up. Make sense? If not, no worries, you'll catch on. a - Append text after the cursor. A - Append text at the end of the line. d - delete text i - Insert text before the cursor. I - Insert text at the beginning of the line. R - Allows you to overwrite over text.
EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager