IBM SVC Advanced Copyservices

Front cover
Draft Document for Review February 10, 2008 5:17 pm SG24-7574-00
SVC 4.2.1 Advanced Copy Services

Learn about the new Advanced Copy Services functionality Server integration examples
Scripting examples
Jon Tate Pall Thorir Beck Tom Jahn Dan Rumney
ibm.com/redbooks
Draft Document for Review February 10, 2008 5:17 pm
7574edno.fm
International Technical Support Organization SVC 4.2.1 Advanced Copy Services November 2007
SG24-7574-00
7574edno.fm
Note: Before using this information and the product it supports, read the information in Notices on page v.
First Edition (November 2007) This edition applies to the IBM System Storage SAN Volume Controller v4.2.1. This document created or updated on February 10, 2008.
Copyright International Business Machines Corporation 2007. All rights reserved. Note to U.S. Government Users Restricted Rights -- Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp.
7574spec.fm
Notices
This information was developed for products and services offered in the U.S.A. IBM may not offer the products, services, or features discussed in this document in other countries. Consult your local IBM representative for information on the products and services currently available in your area. Any reference to an IBM product, program, or service is not intended to state or imply that only that IBM product, program, or service may be used. Any functionally equivalent product, program, or service that does not infringe any IBM intellectual property right may be used instead. However, it is the user's responsibility to evaluate and verify the operation of any non-IBM product, program, or service. IBM may have patents or pending patent applications covering subject matter described in this document. The furnishing of this document does not give you any license to these patents. You can send license inquiries, in writing, to: IBM Director of Licensing, IBM Corporation, North Castle Drive, Armonk, NY 10504-1785 U.S.A. The following paragraph does not apply to the United Kingdom or any other country where such provisions are inconsistent with local law: INTERNATIONAL BUSINESS MACHINES CORPORATION PROVIDES THIS PUBLICATION "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Some states do not allow disclaimer of express or implied warranties in certain transactions, therefore, this statement may not apply to you. This information could include technical inaccuracies or typographical errors. Changes are periodically made to the information herein; these changes will be incorporated in new editions of the publication. IBM may make improvements and/or changes in the product(s) and/or the program(s) described in this publication at any time without notice. Any references in this information to non-IBM Web sites are provided for convenience only and do not in any manner serve as an endorsement of those Web sites. The materials at those Web sites are not part of the materials for this IBM product and use of those Web sites is at your own risk. IBM may use or distribute any of the information you supply in any way it believes appropriate without incurring any obligation to you. Information concerning non-IBM products was obtained from the suppliers of those products, their published announcements or other publicly available sources. IBM has not tested those products and cannot confirm the accuracy of performance, compatibility or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products. This information contains examples of data and reports used in daily business operations. To illustrate them as completely as possible, the examples include the names of individuals, companies, brands, and products. All of these names are fictitious and any similarity to the names and addresses used by an actual business enterprise is entirely coincidental. COPYRIGHT LICENSE: This information contains sample application programs in source language, which illustrate programming techniques on various operating platforms. You may copy, modify, and distribute these sample programs in any form without payment to IBM, for the purposes of developing, using, marketing or distributing application programs conforming to the application programming interface for the operating platform for which the sample programs are written. These examples have not been thoroughly tested under all conditions. IBM, therefore, cannot guarantee or imply reliability, serviceability, or function of these programs.
Copyright IBM Corp. 2007. All rights reserved.
7574spec.fm
Trademarks
The following terms are trademarks of the International Business Machines Corporation in the United States, other countries, or both:
Redbooks (logo) AIX DS4000 DS8000 FlashCopy IBM Redbooks System Storage Tivoli
The following terms are trademarks of other companies: QLogic, and the QLogic logo are registered trademarks of QLogic Corporation. SANblade is a registered trademark in the United States. Snapshot, and the Network Appliance logo are trademarks or registered trademarks of Network Appliance, Inc. in the U.S. and other countries. Java, and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both. Windows Server, Windows, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both. Linux is a trademark of Linus Torvalds in the United States, other countries, or both. Other company, product, or service names may be trademarks or service marks of others.
vi
7574TOC.fm
Contents
Notices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .v Trademarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vi Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii The team that wrote this book . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii Become a published author . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv Comments welcome. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv Chapter 1. Introduction to SVC 4.2.1 Advanced Copy Services . . . . . . . . . . . . . . . . . . . 1.1 What is new in FlashCopy. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.1.1 Incremental FlashCopy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.1.2 Cascaded FlashCopy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.1.3 Dynamically configurable replication services space . . . . . . . . . . . . . . . . . . . . . . . 1.1.4 Grain size . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.1.5 Bitmap space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.1.6 Cleaning mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 What is new in Remote Copy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.3 Other new functionality in 4.2.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.3.1 Increased storage addressing. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.3.2 Volume management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.3.3 Managed disks test before added to MDisk Groups . . . . . . . . . . . . . . . . . . . . . . . . 1.3.4 Cache . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 2 2 2 2 2 3 3 3 3 3 3 3 4
Chapter 2. Planning for Advanced Copy Services implementation . . . . . . . . . . . . . . . . 5 2.1 First questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 2.2 Reasons for copying data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 2.3 Deciding what data to copy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 2.4 Ensuring copied data sets are usable . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 2.4.1 Understanding caching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 2.4.2 Understanding consistency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 2.5 Accessing the copied data sets. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 2.6 Determining how much data can be replicated . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 2.6.1 VDisk capacity limit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 2.7 Understanding your current environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 2.7.1 Storage controller information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 2.7.2 Server information. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 2.7.3 Rate of transfer of data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 2.7.4 Intersite latency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 2.8 Administration of Copy Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 2.8.1 SSH Keys . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 2.8.2 Command Auditing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 2.8.3 Authorization Roles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 2.9 Removing workarounds with 4.2.1 new functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 Chapter 3. Remote Copy: Metro Mirror and Global Mirror . . . . . . . . . . . . . . . . . . . . . . 3.1 Remote Copy services overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1.1 Metro Mirror . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1.2 Cluster partnership . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1.3 Intercluster communication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 24 24 25 26 vii
7574TOC.fm
3.1.4 Maintenance of the intercluster link. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1.5 I/O channel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1.6 Background copy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1.7 Remote Mirror relationship . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1.8 Copy relationship naming convention . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1.9 Supported methods for synchronizing. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1.10 Remote mirror consistency groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2 State overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2.1 Connected or disconnected . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2.2 Consistent or inconsistent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2.3 Consistent or synchronized. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2.4 Detailed Information about copy states. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3 Metro Mirror and Global Mirror configuration limits . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4 Remote Copy internals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4.1 Read stability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4.2 Write ordering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4.3 Write consistency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4.4 Write completion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4.5 Freeze/Run on consistency groups. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4.6 Management of difference map . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4.7 Non-volatile journal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4.8 Quick Recovery Copy or Reconstruct . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chapter 4. Implementing Remote Copy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.1 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2 Implementation using the GUI. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.1 Creation of cluster partnership . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.2 Changing a cluster partnership . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.3 Deleting a cluster partnership . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.4 Creating a copy relationship . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.5 Changing copy relationship. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.6 Starting a copy relationship. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.7 Stopping a copy relationship. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.8 Deleting a copy relationship . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.9 Creating consistency groups. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.10 Starting a consistency group. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.11 Stopping a consistency group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.12 Renaming a consistency group. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.13 Reversing copy directions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3 Implementation using the CLI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3.1 Listing cluster candidates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3.2 Create cluster partnership. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3.3 Change cluster partnership . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3.4 Deleting a cluster partnership . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3.5 Creating a copy relationship . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3.6 Starting a copy relationship. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3.7 Stopping a copy relationship . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3.8 Change a copy relationship . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3.9 Delete copy relationship . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3.10 Creating a consistency group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3.11 Start consistency group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3.12 Stopping the consistency group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3.13 Changing a consistency group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
26 27 27 28 28 29 30 31 32 33 34 34 38 38 38 39 39 39 40 41 42 43 45 46 47 47 50 51 52 56 57 59 60 61 63 64 66 66 68 69 69 70 70 71 72 74 74 75 75 77 78 79
viii
7574TOC.fm
4.3.14 Deleting a consistency group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3.15 Reversing copy directions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.4 Example of site failover . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.4.1 Site failover when production site loses back-end storage . . . . . . . . . . . . . . . . . . 4.4.2 Site failover using the GUI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.4.3 Site failover using the CLI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.5 Copy commands using the CLI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
79 81 82 83 84 87 89
Chapter 5. FlashCopy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91 5.1 Introduction to FlashCopy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 5.1.1 What is FlashCopy and what is it useful for? . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 5.1.2 Usage cases of FlashCopy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 5.2 FlashCopy functions and features. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93 5.2.1 FlashCopy Mappings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93 5.2.2 FlashCopy Consistency Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94 5.2.3 FlashCopy background copy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 5.2.4 FlashCopy autodelete option . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96 5.2.5 Multiple Target FlashCopy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96 5.2.6 Cascaded FlashCopy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97 5.2.7 Combined Multiple Target FlashCopy and Cascaded FlashCopy . . . . . . . . . . . . 98 5.2.8 Incremental FlashCopy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99 5.3 FlashCopy internals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100 5.3.1 What is the indirection layer and what is it for?. . . . . . . . . . . . . . . . . . . . . . . . . . 101 5.3.2 What is a grain? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 5.3.3 What is a bitmap? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102 5.3.4 Usable bitmap space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102 5.3.5 Number of FlashCopy Mappings in a cluster . . . . . . . . . . . . . . . . . . . . . . . . . . . 103 5.3.6 How can we specify and view the bitmap space on an I/O group. . . . . . . . . . . . 104 5.3.7 Where is the bitmap stored? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104 5.3.8 FlashCopy characteristics listed . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 5.3.9 FlashCopy maximum configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 5.4 FlashCopy events and states . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106 5.4.1 FlashCopy Mapping states . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106 5.4.2 FlashCopy events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107 5.4.3 FlashCopy consistency group states . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109 5.5 Dependencies between FlashCopy Mappings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110 5.5.1 Linked lists of mappings and resulting dependencies. . . . . . . . . . . . . . . . . . . . . 110 5.5.2 Stopping state and cleaning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115 5.5.3 Command Line commands for information on dependencies. . . . . . . . . . . . . . . 116 Chapter 6. Implementing FlashCopy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.1 Example 1: basic FlashCopy using the GUI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.1.1 Creating a FlashCopy mapping. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.1.2 Starting a FlashCopy mapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.1.3 Modify a mapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.1.4 Stopping a mapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.1.5 Creating a FlashCopy Consistency Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.1.6 Creating a FlashCopy mapping and adding it to the Consistency Group . . . . . . 6.1.7 Preparing a FlashCopy Consistency Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.1.8 Starting a FlashCopy Consistency Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.1.9 Stopping a Consistency Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.1.10 Renaming a Consistency Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.1.11 Issuing a command to a mapping within a Consistency Group . . . . . . . . . . . . 117 118 118 124 125 126 127 130 134 136 139 140 141
Contents
ix
7574TOC.fm
6.2 Example 2: Multiple Target FlashCopy using the GUI . . . . . . . . . . . . . . . . . . . . . . . . 6.2.1 Creating FlashCopy Mappings and a Consistency Group . . . . . . . . . . . . . . . . . 6.2.2 Starting a Consistency Group in an MTFC tree . . . . . . . . . . . . . . . . . . . . . . . . . 6.3 Example 3: Cascaded and Incremental FC using the GUI . . . . . . . . . . . . . . . . . . . . . 6.3.1 Creating FlashCopy Consistency Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.3.2 Create Incremental FlashCopy Mappings. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.4 Example 4: FlashCopy of Metro Mirror VDisks using the CLI . . . . . . . . . . . . . . . . . . . 6.4.1 Checking VDisks and Metro Mirror relationship to be used . . . . . . . . . . . . . . . . 6.4.2 Create FlashCopy mappings. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.4.3 Creating a FlashCopy Consistency Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.4.4 Adding FlashCopy mappings to the Consistency Group. . . . . . . . . . . . . . . . . . . 6.4.5 Preparing a FlashCopy mapping. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.4.6 Starting a FlashCopy Consistency Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.4.7 Modify a mapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.4.8 Stopping a mapping and a Consistency Group. . . . . . . . . . . . . . . . . . . . . . . . . . 6.4.9 Deleting a mapping and a Consistency Group . . . . . . . . . . . . . . . . . . . . . . . . . . Chapter 7. Server integration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.1 What data is copied?. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.2 What data is on the copied VDisk? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.3 AIX specifics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.3.1 Creating useful copies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.3.2 Accessing useful copies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.4 Windows 2000/2003 and Copy Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.4.1 Creating useful copies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.4.2 Accessing useful copies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.5 Linux Specifics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.5.1 Creating useful copies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.5.2 Accessing useful copies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.6 Other Operating Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chapter 8. Automation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.1 Running commands on the cluster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.1.1 Connection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.1.2 Authentication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.1.3 Submission . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.1.4 Authorization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.1.5 Execution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.2 Creating connections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.2.1 Transient connections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.2.2 Persistent connections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.3 SVC command line scripting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.3.1 Submitting command line scripts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.3.2 An example SVC Command Line script . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.4 Server side scripting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.4.1 An example script for creating persistent SVC connections . . . . . . . . . . . . . . . . 8.4.2 Handling the SVC response . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.5 SVC CIMOM programming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.5.1 CIM concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.5.2 How SVC concepts map to CIM concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.5.3 Example script to create and start a FlashCopy mapping. . . . . . . . . . . . . . . . . .
142 143 146 148 149 149 157 158 161 163 164 166 166 169 170 170 173 174 174 175 175 178 188 188 194 200 200 203 205 207 208 208 209 210 210 210 211 211 214 214 215 216 217 217 223 224 225 226 227
7574TOC.fm
8.6 Logging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233 8.7 Auditing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234 Related publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . IBM Redbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Other resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Referenced Web sites. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . How to get Redbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Help from IBM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237 237 237 238 238 238
Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241
Contents
xi
7574TOC.fm
xii
7574pref.fm
Preface
This book describes the new features that have been added with the release of the IBM System Storage SAN Volume Controller 4.2.1 code. Readers are expected to have an advanced knowledge of the SVC and SAN environment, and we recommend these books as background reading: IBM System Storage SAN Volume Controller, SG24-6423 Introduction to Storage Area Networks, SG24-5470 SAN Volume Controller: Best Practices and Performance Guidelines, SG24-7521 Using the SVC for Business Continuity, SG24-7371
The team that wrote this book

This book was produced by a team of specialists from around the world working at the International Technical Support Organization, San Jose Center. The team is pictured in Figure 0-1.
Figure 0-1 L-R, Dan, Pall, Tom , and Jon
Jon Tate is a Project Manager for IBM System Storage SAN Solutions at the International Technical Support Organization, San Jose Center. Before joining the ITSO in 1999, he worked in the IBM Technical Support Center, providing Level 2 support for IBM storage
xiii
7574pref.fm
products. Jon has 22 years of experience in storage software and management, services, and support, and is both an IBM Certified IT Specialist and an IBM SAN Certified Specialist. He is also the UK Chair of the Storage Networking Industry Association (SNIA). Pall Thorir Beck is a Technical Team Lead in IBM ITD Denmark. He has 11 years of experience working with storage, and joined the IBM Denmark team in 2005. Prior to working for IBM in Denmark he worked as an IBM CE performing hardware installaions and repairs for IBM iSeries, pSeries and zSeries in Iceland. His current position involves the administration, design, and support of large SAN environments in Denmark. He is also part of a Nordic storage team that covers storage for open systems in Sweden, Norway and Finland. Pall has a diploma as an Electronics Technician from Odensen Tekniske Skole in Denmark, and IR in Reykjavik, Iceland. Tom Jahn is a Senior I/T Specialist for IBM TSS Storage Systems Sales in Germany. He has 10 years of experience providing technical sales support in IBM. Tom provided technical sales support for networking and server consolidation on OS/390 UNIX for IBM and its customers before he joined the storage brand eight years ago. He is engaged in providing technical support for open systems storage solutions across multiple platforms and a wide customer base, currently with a focus on storage virtualization. He holds a Dipl. Ing. degree in Computer Science from the Staatliche Studienakademie Sachsen. Dan Rumney is the IBM Global Support Manager for the JPMC Account. Prior to this role, he has worked in development and support of storage products since joining IBM in 2002. Dan worked in System Verification Test for IBM before moving to support. He has been providing Level 2 and Level 3 support of SAN Volume Controller for the past 2 years.. We extend our thanks to the following people for their contributions to this project. There are many people that contributed to this book. In particular, we thank the development and PFE teams in Hursley. In particular Matt Smith was also instrumental in moving any issues along and ensuring that they maintained a high profile. We would also like to thank the following people for their contributions: John Agombar Chris Beeken Iain Bethune Trevor Boardman Carlos Fuente Gary Jarman Colin Jewell Andrew Martin Paul Merrison Steve Randle Bill Scales Matt Smith Barry Whyte IBM Hursley Bill Wiegand IBM Advanced Technical Support Evelyn Wick IBM Beaverton Dorothy Faurot IBM Raleigh xiv
7574pref.fm
Marci Nagel IBM Rochester Chris Saul IBM San Jose Sharon Wang IBM Chicago Tom Cady Deanna Polm Sangam Racherla Bill Trimble IBM ITSO Tom and Jenny Chang Garden Inn Hotel, Los Gatos, California
Become a published author

Join us for a two- to six-week residency program! Help write a book dealing with specific products or solutions, while getting hands-on experience with leading-edge technologies. You will have the opportunity to team with IBM technical professionals, Business Partners, and Clients. Your efforts will help increase product acceptance and customer satisfaction. As a bonus, you will develop a network of contacts in IBM development labs, and increase your productivity and marketability. Find out more about the residency program, browse the residency index, and apply online at: ibm.com/redbooks/residencies.html
Comments welcome
Your comments are important to us! We want our books to be as helpful as possible. Send us your comments about this book or other IBM Redbooks in one of the following ways: Use the online Contact us review Redbooks form found at: ibm.com/redbooks Send your comments in an e-mail to: redbooks@us.ibm.com Mail your comments to: IBM Corporation, International Technical Support Organization Dept. HYTD Mail Station P099 2455 South Road Poughkeepsie, NY 12601-5400
Preface
xv
7574pref.fm
xvi
7574 Introduction.fm
Chapter 1.
Introduction to SVC 4.2.1 Advanced Copy Services

In this book we will discuss what is new in v4.2.1 code, how to plan for it, focus on the new functions and features, including implementation examples. We also describe server integration and finally some some useful scripts to simplify some of the SVC tasks. Note: IBM System Storage SAN Volume Controller Software v4.2.1 can only be installed on the IBM 2145-8G4, IBM 2145-8F4, 2145-8F2, and 2145-4F2 SAN Volume Controller Storage Engines. We do not intend to go into any great depth about Business Continuity and Disaster Recovery theories and recommend these books as sources for information: IBM System Storage Business Continuity: Part 1 Planning Guide, SG24-6547 IBM System Storage Business Continuity: Part 2 Solutions Guide, SG24-6548 Using the SVC for Business Continuity, SG24-7371
1.1 What is new in FlashCopy

There are some considerable changes made to FlashCopy functionality in this release. The major changes are to FlashCopy as follows: Incremental FlashCopy Cascaded FlashCopy Dynamically configurable replication services space Capability to address storage up to eight petabytes Grain size Bitmap Space Cleaning mode
1.1.1 Incremental FlashCopy

This feature allows us to copy only the portions of the source VDisk that have been updated at either the source or target since the last time that the mapping was started, instead of copying all the content of that disk. The quantity of data that needs copying also depends on the grain size, the smaller the grain size the less data there may be to copy. This will lead to shorter copy time and reduce the load on the SVC. To be able to see the difference between source and target a difference value has been added.
1.1.2 Cascaded FlashCopy

This feature allows us to use a target VDisk of a FlashCopy mapping as the source VDisk of another FlashCopy mapping. The total number of targets from the original source can be a maximum of 16.
1.1.3 Dynamically configurable replication services space

This feature allows us to configure up to 1 PB of address space per I/O group. The default bitmap space is 40 MB per I/O group, which is evenly divided by FlashCopy and Metro/Global Mirror. SVC 4.2.1 introduces a new feature which can borrow some read/write cache to serve as copy service bitmap space. The bitmap space per I/O group can be up to 512 MB, which can support up to 1 PB replication service capacity. We can dynamically configure this space between FlashCopy, Metro Mirror and Global Mirror relationships in customized sizes. This new feature has the option to allocate the cache memory address space when needed to achieve copy services requirements. The exact amount of copy services space available to users will depend upon the copy services relationships used, and the resulting copy services space allocated to each VDisk.
1.1.4 Grain size

Now we can specify a smaller grain size of 64KB. This smaller grain size is not restricted to incremental mappings. If a mapping is being created which will be added to a tree of cascaded and/or multiple target mappings, the mapping will use the grain size used by the other mappings in the tree. The cascading of mappings provides the possibility of joining two separate trees of mappings together. If these two trees do not use the same grains size then it will not be possible to create the mapping which will join them.
1.1.5 Bitmap space

Bitmap space for mappings is allocated on a per I/O group basis. For a single mapping, the default I/O group from which to take bitmap space is that of the source VDisk for the mapping. For a mapping which is being added to an existing tree of mappings, the mapping will use the I/O group being used by the other mappings in the tree. The I/O group can be set as a parameter when creating the mapping to allow the user to force the I/O group from which to consume bitmap space. This ability can be useful if the user knows that trees of mappings may be joined in the future, or for reconstructing a tree of mappings in a different order to their original creation during configuration restore.
1.1.6 Cleaning mode

Cleaning mode is a method of copying grains from the mapping that is to be stopped when it is in copying state (and when its target VDisk is still accessible to the host). This is achieved by adding a new cleaning rate parameter for a mapping which, if combined with a zero background copy rate, causes the mapping to copy grains to a downstream mapping. A new cleaning progress field in the query of a mapping informs the user of progress. When cleaning is complete, the mapping can be stopped and will transition to stopped state immediately. If the mapping is stopped before cleaning is complete, the remainder of the cleaning will occur whilst in the existing stopping state.
1.2 What is new in Remote Copy

Metro Mirror now uses the same inter-node communication technology as Global Mirror. That means only one round trip delay for the remote I/O traffic instead of two. This improves the robustness, and could increase performance on the exciting applications, and could even increase the distance where Metro Mirror latency impact does not play a significant role.
1.3 Other new functionality in 4.2.1

This is an overview of other new functions in SVC 4.2.1.
1.3.1 Increased storage addressing

This feature has increased the storage addressing capability from two petabytes to eight petabytes per cluster. This capability is based on an increased extent size (up to 2 GB), and applies regardless of the number of nodes in the cluster. MDisk and VDisk relations and use are unaffected by this increase.
1.3.2 Volume management

This feature offers the possibility to assign or change the preferred node upon VDisk reallocation to new I/O group.
1.3.3 Managed disks test before added to MDisk Groups

This feature offers a built-in test to check MDisks before they are added to an MDisk Group.
Chapter 1. Introduction to SVC 4.2.1 Advanced Copy Services
1.3.4 Cache
In this version of software the cache space is partitioned by managed disk group. Each MDisk group that qualifies is limited to a certain amount of the cache, depending on how many MDisk groups there are running in the cluster.
7574 Planning for ACS implementation.fm
Chapter 2.
Planning for Advanced Copy Services implementation

Implementing Advanced Copy Services requires considered planning in order to be effective. In this chapter, we discuss the issues that should be considered during your planning phase and provide guidance on how to resolve some of the questions.
2.1 First questions

Copy Services come in two types: Remote Copy, comprising: Metro Mirror Global Mirror FlashCopy, comprising: Single Target FlashCopy (FC) Multiple Target FlashCopy (MTFC) Cascaded FlashCopy (CFC) Incremental FlashCopy (IFC) In the early phases of planning, it is tempting to select one of these Copy Services functions as the very first step; we strongly recommend that you wait until later in the process, By identifying the business aspects first, you will be better positioned to select the most appropriate technical solution. Prior to implementing Copy Services, there are a number of questions to ask yourself. Why do I need multiple copies of my data? What data do I want to copy? How do I ensure that my copies are usable? Under what circumstances will I need to access this data? How much data can I copy? How are my current systems workloads characterized? Will Copy Services be controlled by server or storage administrators?
2.2 Reasons for copying data

There are many reasons why you may want to copy your data. Several business cases are discussed in the later chapters, but at a high level, these may include: Disaster Recovery Data are copied to prevent loss of information due to a disaster and to allow the continuation of business operations until the disaster has passed
Testing and Assurance Data are copied to provide test and assurance systems with a copy of real production data to allow for meaningful testing Business Operations Data are copied prior to system configuration changes to provide snapshots in case rollback is required
The reason for copying data dictates which data will be copied. For instance, the data copied for a Disaster Recovery solution may differ significantly from the data copied for Testing, even if the same servers are involved.
2.3 Deciding what data to copy

Complex systems consume and produce vast amounts of data during day to day operations. If the decision was made to mirror every byte of data in an environment then storage 6
utilization would never exceed 50%. Its clear that decisions need to be made to select precisely which data need copying to balance the cost of copying with its benefits. There are costs associated with Copy Services beyond the cost of the technology. These include: The capital and maintenance cost of the additional storage needed The management cost of administering the Copy Services The opportunity cost of the bandwidth used to mirror the data These costs are offset by the benefit of having multiple copies of your data. Business cases for which these Copy Services can be used are listed in later chapters. The benefits of these copies are often calculated by determining the cost of having to recreate this data or the cost of losing it forever. The benefit is considered to be the fact that you do not have to meet this cost, if you implement the Copy Services solution. In extreme cases, the data may be critical to your businesss core function and the cost of losing it is your business. We call groupings of files and other information data sets. A data set could be multiple VDisks which contain the information that you are concerned with. It could be that a data set is smaller than a VDisk, for instance, a single partition on a larger VDisk. However, the smallest entity that SVC can replicate is a VDisk and we will not discuss file or filesystem replication in this Redbook. Some data sets have no business value, but is generated during day to day operations. Here are some examples of data sets that you may not wish to copy: Temporary file space Temporary database tables Non-business data, such as employees personal files Some data sets are easy to reproduce or, more relevantly, cheaper to reproduce than they are to copy. As such, you may decide that some data should be left uncopied. Examples of such data sets may include: Development sandboxes Test systems SAN Boot volumes (reinstallation may be simpler and cheaper than replication) Undoubtedly, even after removing data sets such as those above, you will be left with data sets that are too expensive to lose or recreate. It is the VDisks upon which these data sets reside that you will want to copy.
2.4 Ensuring copied data sets are usable

Copying the contents of a VDisk to another VDisk is all very well, but for the copy to be worth something, you must be able to access and use it at a later date. In order to design a system that ensures this, you must understand how write-caching comes into play and what consistency is.
2.4.1 Understanding caching

Caching has been used in computing for over three decades to increase the performance of access requests to storage devices. In many computer systems, there are multiple layers of cache and it is important to understand some of these layers when implementing Copy Services.
Chapter 2. Planning for Advanced Copy Services implementation
While information is cached on both reads and writes, we are only concerned with write caching, when we consider Copy Services
Timeline of a write cache

Figure 2-1 on page 8 shows a basic diagram, outlining the timeline of a write to a storage device when a write cache is in use. Using a cache speeds up write operations in the following way. Without the cache, the host will send a write request directly to the storage device. The storage device will respond and data will travel from client to device. Once all of the data is in place on the storage device it will signal that the process is complete. The transfer is limited by the speed at which the disks can provide data. For a storage subsystem. this operation could take 10s of milliseconds to complete, in addition to the transit time over the network. With a cache in place, the storage client need only transfer the data to the cache. The cache works much more quickly than the disks themselves, so this transfer takes less time to complete. This corresponds to the time interval between T0 and T1 on Figure 2-1. At some later time (T2), the cache performs a destaging operation and saves the cached data to the physical disk. This completes at time T3. The host, however, is completely unaware of this operation. For a storage subsystem with cache, the network transit time becomes the limiting factor and the operation time could be less than a millisecond.
Figure 2-1 Simple write operation with caching
Figure 2-1 is a simplification. In a production environment, you will have cache above cache above cache. Consider a system with SAN Volume Controller (SVC) installed, as displayed in Figure 2-2 on page 10. This shows how a production environment can have many caches between the application that end users see and the disks in a storage subsystem. The numbering in the list below refers to the numbering of the caches in Figure 2-2 on page 10: 1. Database cache: Queries to the database may well be cached by the database process to allow for rapid completion of application processes. This may be done in the form of storing queries for later execution. 2. Filesystem cache: IO commands to the filesystem may be cached in memory to allow faster completion to processes running on the server. These commands will be passed to underlying storage at a later time or when the cache needs to make space for subsequent IO commands.
3. SVC cache: SVC has a redundant write-cache which allows for fast completion of IO commands to the server. The data from these commands are destaged at a later time. 4. Storage controller cache: Most modern storage controllers have write cache of their own. Once data is stored in these caches, the system will report a successful conclusion. At a later date, this information will be written to the physical disks themselves. The diagram shows 4 layers representing a function in the system. As data flows from layer to layer, it passes through a cache. Each cache makes a commitment to the layer above that any data that it accepts will eventually be destaged. However, there is no commitment as to when the data will be committed. This is important to remember when planning a solution that uses Copy Services. It should also be understood that each layer has no idea that there is data in the caches owned by higher layers. Remember: A cache makes a commitment that completed writes will be written to the layer below, but with no commitment as to when A cache has no idea about the existence of caches above it.
Figure 2-2 Layers of caching in a production environment
Impact of write caches to Copy Services

Copy Services are provided by SAN Volume Controller at the storage layer. However, they are used to solve business challenges at the application or hosting layer. As a result, it is important to be aware of the data flow between the layers when planning your Copy Services solution. Consider the FlashCopy function. This provides a Point In Time (PIT) copy of your data. When using the FlashCopy function, it is important to ensure that the VDisk that FlashCopy is
10
copying is identical to the volume that your application is seeing. Because of write-caching, this is not always the case. In Figure 2-3 you can see where FlashCopy exists with respect to caches. The server sees a SCSI LUN presented by the SVC, corresponding to a VDisk. When an application performs certain actions, data is written to this SCSI LUN. Because of the caching, data written at some time t0 may not actually hit the VDisk until a later time t1 . Normally this is not a problem, because: The writes will eventually get there All reads go through the same cache, so there is never any noticeable inconsistency between what is on the SCSI LUN that the server sees and what is on the VDisk that SVC presents.
Figure 2-3 Position of FlashCopy with respect to caches
When using Copy Services, this does become a problem. The business problem that is being solved demands that the data on the LUN be copied. As a result, every block of data that the system has written to its LUNs must be captured by the FlashCopy. However, as commented above, SVC works with VDisks and has no idea that there is data in the caches above it. In order to ensure that the FlashCopy Target VDisk is an identical match to the SCSI LUN that the server sees, data must be purged from the caches at all layers and the system must be configured to place no more data in the caches until the FlashCopy has been started. This can be done by Quiescing IO operations
11
Setting caches to a write-through mode Vendor-specific processes, such as setting applications to special backup modes that allow for consistent copies to be made. The same phenomenon exists for Global and Metro Mirror. In these cases, the Primary and Secondary VDisks remain in synch with one another, but, as with FlashCopy VDisks, there will be a time lag between the SVC VDisks and the SCSI LUN that the server sees. Thus, when the relationship is broken or stopped, the data on the VDisk will not be identical to the data that application sees on the LUN.
Dealing with write caches

All of the above is a property of write caches and not a limitation of SVC. Once data is in the SVC write-cache, it will be handled appropriately; write commands that have been reported as having completed successfully will be completed. You must be aware of all the write caches on systems that you implement Copy Services on. This may involve discussions with vendors or systems experts to understand what data is cached, when it is cached and how long it is cached for. In addition, you should find out if there are methods for purging the cache, disabling the cache and checking the cache state. This information will be critical for planning your end to end solution.
2.4.2 Understanding consistency

Consistency is a critical concept when it comes to Copy Services. By ensuring consistency in its Copy Services, SVC allows systems running on attached hosts to access copied VDisks effectively. It does this by ensuring Read Stability If two reads are submitted for the same area and those reads complete successfully and there was no intervening (in time) or overlapping (in time) write activity then the two reads must return the same data This may seem obvious, but its useful to state it explicitly. In addition to this, the following behavior is also displayed: If a write to a certain area completes successfully then any future reads from that area will return the data that was written (until a further write to the same area is completed successfully) Again, this may seem obvious, but these two characteristics define how the ordering of reads and writes behave. They allow us to define our expectations for what is on a disk at any single point in time. Take note that writes must complete successfully before you can be certain of what a read operation will return. If you send a read (R1) to an area of disk, then send a write (W1) to the same area of disk and then send another read (R2) to the same area of disk, before W1 completes, the only guarantee is that R2 will match R1 or W1. Finally, we discuss write ordering. Many applications rely on write ordering in order to survive failures. Write ordering means: If a write (W1) is sent and then reported as complete, and then a second write (W2) is sent and then reported as complete, then the cache will submit these writes to the next level in the
same order.
12
However, if W2 is sent before W1 completes, then there is no guarantee as to the order in which the cache will submit these to the layer below. In fact, the cache may well batch these and submit them at the same time.
Consistency for a single copy relationship

Like all SCSI Storage Devices, an SVC VDisk is made up of blocks. Consider a VDisk at a time t0 . A process starts writing data to numerous blocks on the VDisk until time t1 , at which point it stops. If you now read from all of the blocks, you will have a set of data the represents the VDisks state at time t1 . If at some later point in time, t2 , you read from all of the blocks again, because of Read Stability, you will get an identical set of data. Therefore, you can say that the data set from t2 is the same data set as that from t1 . Since each block from t2 matches each block from t1 we say that it is a consistent copy. To demonstrate this concept further, take a look at Figure 2-4 on page 14. This figure demonstrates the idea of consistency in the context of Copy Services. Consider two brand new VDisks: a Source and a Target (or a Primary and a Secondary in the case of Metro/Global Mirror)
13
Figure 2-4 Explaining Copy Services Consistency
At t0 , we have 2 VDisks which are about to be put into a copy relationship. They are very small VDisks with only 4 blocks each. The precise type of copy relationship is not relevant as the concept is the same. At this point in time, the relationship is inconsistent because the Target VDisk data set does not match any version of the Source VDisk data set. The subsequent points in time are described in the following list: 1. At this point, block A is copied from the Source to the Target. The relationship is still inconsistent. 2. At this point, block B is copied from the Source to the Target. The relationship is still inconsistent. 3. At this point, block C is copied from the Source to the Target. The relationship is still inconsistent. 4. At this point, block A is overwritten by the server with block E. The relationship is still inconsistent.
14
5. At this point, block D is copied from the Source to the Target. The relationship is now consistent as the Target VDisk data set represents the same data set as the Source VDisk at t3 . 6. At this point, block E is copied from the Source to the Target. The relationship is still consistent and is now synchronized The difference in time between t3 and t5 represents the Recovery Point Objective1 (RPO) for this VDisk.
Consistency for a group of copy relationships

Having described the concept of consistency for a single copy relationship, we go on to define the concept of consistency for a group of copy relationships. A single relationship is consistent if the data set of the Target VDisk matches the data set of the Source VDisk at some point in time in the past. In the same way, a group of relationships is consistent if each of the relationships in the group are themselves consistent and each relationship represents the same point in time as all of the others. Once consistency has been attained, it is maintained until specific user actions are taken to disrupt it. Actions that result in consistency being broken are explained later in this Redbook.
Consistency for FlashCopy

Once a FlashCopy mapping started, the Source and Target VDisks are, by definition, consistent. The mapping represents a single point in time and every FlashCopy mapping in a consistency group represents the same point in time. Bear in mind that the Target VDisks are consistent with the Source VDisks that SVC is presenting. Because of write caches above the Storage Layer, it is possible that these differ from the Logical Unit that the application sees. This is why the caches should be purged prior to starting FlashCopy mappings.
Consistency for Metro and Global Mirror

When Metro/Global Mirror relationships are started, they are inconsistent. They must go through a synchronization stage before they become consistent (much like the process outlined in Figure 2-4 on page 14). Once this process is complete, the relationships will stay consistent throughout normal running. At this point, we introduce the concept of synchronized relationships. As outlined above, a relationship is consistent if the data set on the Secondary VDisks represent a data set on the Primary VDisks at some point in time. A relationship is synchronized if it is consistent and the point in time that the Secondary VDisks represent is the current point in time. Put another way, the Secondary VDisks contain exactly the same data as the Primary VDisks. This is represented by t6 on Figure 2-4 on page 14. This definition is stretched a little when it comes to Global Mirror due to the asynchronous nature of the function. The Global Mirror design is such that the lag is kept as low as possible. If the lag grows too long for an extended period of time, then SVC will stop the relationship to prevent performance on the Primary VDisk from getting too poor.
See Using the SVC for Business Continuity, SG24-7371 to learn more about the language of Business Continuity
15
2.5 Accessing the copied data sets

The decision of when to access the data sets that have been copied is very much a business decision. It stems from the rationale behind copying the data in the first place. We do not discuss that aspect here. Then technical practicalities of presenting replicated VDisks from the SVC is straightforward and are discussed in Chapter 6, Implementing FlashCopy and Chapter 4, Implementing Remote Copy. Accessing these data sets from your servers is more complex and will be discussed in Chapter 7, Server integration.
2.6 Determining how much data can be replicated

A major part of Copy Services planning is determining how much data you actually want to replicate. As discussed in Deciding what data to copy on page 6, there are many costs associated with implementing Copy Services and some of these may act to limit the amount of data that you wish to replicate. In addition to these business limitations, there are some technical limitations that you should be aware of when planning your solution. Copy Services require the use of SVC resources and so are not limitless. However, Table 2-1 shows that these limitations are enough to allow all but the most expansive Copy Services solutions. As new versions of SVC have been released, these limitations have become less restraining and we list here the limitations as they have changed over past releases. In each column, we highlight the item that has changed since the previous release
Table 2-1 SVC Copy Services Limitations Objects FlashCopy Mappings per cluster FlashCopy Mappings per consistency group FlashCopy consistency groups FlashCopy VDisk per IO Group FlashCopy targets per source Metro/Global Mirror relationships per cluster Metro/Global Mirror consistency groups Metro/Global Mirror VDisk per IO Group Metro/Global Mirror round-trip latency (ms) 3.1.0.xa 2048 512 128 16 TB 1 1024 256 16 TB 68c/20d 4.1.0.xa 2048 512 128 16 TB 1 1024 256 16 TB 68c /20d 4.1.1.x 2048 512 128 16 TB 1 1024 256 16 TB 80 4.2.0.x 3855 512 128 40 TB 16 1024 256 40 TB 80 4.2.1.x 3855 512 128 1024 TBb 16 1024 256 1024 TBb 80
a. Global Mirror not supported until SVC 4.1.1.x b. Default is 40 TB. The bitmap space that is used to control Copy Services is shared between FlashCopy and Metro/Global Mirror. c. Fibre Channel Extenders d. SAN Routers
16
You should refer to the latest SVC documentation to find out what limitations apply to the level of SVC that you are running with.
2.6.1 VDisk capacity limit

All Copy Services use bitmaps to track which VDisk data need to be copied from the Source/Primary to the Target/Secondary. The amount of space made available to these bitmaps is variable. This feature allows the user to trade off memory usage between FlashCopy, Metro Mirror and Global Mirror. Copy Services use grains to track what needs to be copied. Each grain represents 256 KB worth of data that may need copying for Metro/Global Mirror. For FlashCopy, a grain is either 64KB or 256KB at the users choice. Each bit in the bitmap represents 1 grain. Bitmap space is owned by an IO Group. When a mapping or relationship is created, you can choose which IO Group will provide the bitmap space. This can be a different IO Group to the ones providing the associated VDisks. The amount of VDisk capacity that can be copied depends on the following: The amount of the memory allocated to hold the bitmap (bitmap space) The grain size If FlashCopy is being used, then, whether it is normal or incremental The maximum size of the bitmap space for one IO group is 512MB, with 20MB being the default. The bitmap space for an IO Group must be shared between FlashCopy and Metro/Global Mirror. If all of the space is used for FlashCopy, for instance, then there would be no space available for Metro/Global Mirror. Bitmap space is allocated to FlashCopy mappings and Metro/Global Mirror relationships in increments of 4KB. This corresponds to 32,768 bits of bitmap space and so 32,768 grains. Thus, the smallest unit of VDisk capacity that can be copied is 8GB (for 256KB grains) or 2GB (for 64KB grains). Because bitmap space is assigned in fixed increments, it takes the same amount of bitmap space to copy a 1GB VDisk as it takes to copy an 8GB VDisk. With Incremental FlashCopy, there are two bitmaps per FlashCopy mapping to be stored, which makes it consume twice the bitmap space of a normal FlashCopy mapping. Table 2-2 on page 17 shows the relationship between VDisk capacity and bitmap space depending on the size of the grain and the kind of copy service being used.
Table 2-2 Bitmap space required for Copy Services Copy Service Grain size (KB) 256 256 64 256 64 VDisk capacity provided by bitmap space 1 Byte 2MB 2MB 512KB 1MB 256KB 4KB 8GB 8GB 2GB 4GB 1GB 1MB 2TB 2TB 512GB 1TB 256GB 20MB 40TB 40TB 10TB 20TB 5TB 512MB 1024TB 1024TB 256TB 512TB 128TB
Metro/Global Mirror FlashCopy FlashCopy Incremental FlashCopy Incremental FlashCopy
17
Note: For the FlashCopy mapping bitmap space, the amount of mappings has to be taken into account. Each mapping target for a FlashCopy uses up bitmap space. This means for Multiple Target FlashCopy, even though there is only one source vdisk involved, multiple targets will consume the same amount of bitmap space as the same number of targets of independent FlashCopy mappings would use up.
2.7 Understanding your current environment

Prior to implementing Copy Services, you should have a good understanding of your storage environment. Implementing Copy Services leads to increased data transfers, so you must understand your current configuration before implementing a solution that will affect it. Copy Services operations are driven by two things: Background copy processes Foreground application processes Both of these processes will generate traffic on your SAN and so its important to understand what your SAN can take, without disruption and how much traffic your application processes are generating.
2.7.1 Storage controller information

Controllers are designed to take as much IO as possible, before latency starts to creep up. Naturally, as a system with finite resources, you cannot increase the workload on a controller indefinitely without consequences. Figure 2-5 shows an illustration of how latency is affected as you increase the workload on a storage controller. All commands will have some latency due to transit time to the storage controller and service time of the commands. As the number of commands sent to a controller each second increases, there will be an increased wait due to queuing, which will be of the order of the service time. The dotted line on Figure 2-5 shows the maximum workload that should be drive to this storage controller.
18
Figure 2-5 Latency of commands to a storage controller as workload increases
If you have a well equipped test lab, you may be in a position to create this kind of graph on the same equipment as that in your production environment. Failing that, your storage controller vendor should be able to advise the maximum workload that your storage controllers are capable of without significant increases in latency. Once you have done this, you should investigate your storage controllers to see what kind of workload they are currently handling. The method by which you do this depends on the controller that you are using. If you use Tivoli Productivity Center (TPC) for Disk and it is configured to monitor your storage controllers, then you can gather this information that way. Alternatively, if you have TPC installed and it is monitoring your SVC, you can gather performance information about your MDisks and work it out that way. Bear in mind that any performance statistics gathered about your MDisks only tell part of the storage if your storage controller is also presenting volumes directly to other servers.
2.7.2 Server information

It is important to know what kind of workload your servers are driving to the system. This include information about Data rate (MB/s) read and written Workload patterns, e.g. sequential, random Response requirements, i.e. maximum acceptable latency It is important to get specific information and not settle for requirements such as As fast as possible and No latency. There is no way to meet these requirements and so they are of no
19
use to anyone. Thats not to say that the servers dont have demanding requirements, but you will need real figures in order to plan your Copy Services system. When a VDisk is being copied (either by FlashCopy or Metro/Global Mirror) writes to the VDisks will drive data copies. In the case of FlashCopy, the first write to a block on the source VDisk will result in data being written to the target VDisk; further writes to the same block will generate no extra IO. In the case of Metro/Global Mirror, each write to a block on the primary VDisk will be replicated to the secondary VDisk. You must ensure that the storage controllers that the target/secondary VDisks are on are up to the workload that will be driven to them. This is particularly important for Metro/Global Mirror relationships. For instance, you may have a set of VDisks that are residing on DS8000 storage controllers; these VDisks are comfortably handling a high write workload. You decided to replicate these VDisks to a remote site, using Metro Mirror. If the remote VDisks are residing on DS4000 storage controllers, then you could expect to see problems. The remote VDisks must be able to handle the same write workload as the local storage or else you will start to see performance problems. Under most workload patterns, Metro/Global Mirror does not cause serialization of writes that are submitted at the same time. Consider a server that sends 5 write operations to 5 different locations on a VDisk, all at the same time. Obviously, this server is not concerned about the write order as it has not waited for completion from the first before sending the second, third, fourth or fifth. Thus for Metro Mirror, these 5 writes will be sent to the secondary disk in any order and completion will be reported to the server as and when the operations are complete. For Global Mirror, these writes will be completed once they are in the cache and then written to the remote cluster as soon as possible. There is one workload pattern which can lead to performance issues with Global Mirror. In order to maintain write ordering, the Global Mirror code only allows a single write to be active on any given block of a VDisk. Thus, if a write comes in from a host to a block, and that block still has data waiting to be written to the remote cluster, the new write will be delayed until the copy has been completed. Applications that deliver this kind of workload will not achieve the performance that Global Mirror is intended to support. The Per VDisk Cumulative total number of overlapping writes (gwo) statistic provided by the SVC gives a count of colliding writes. If bad performance is being seen, checking this statistic can be used to determine if you are hitting this limitation.
2.7.3 Rate of transfer of data

As mention at the start of this section, Copy Services are driven, in part, by foreground application processes. The following operations will lead to data being copied: Every write to a grain on the primary VDisk of a Metro/Global Mirror relationship The first write to a grain on the source or target VDisk in a FlashCopy mapping. The first read from a grain on the target VDisk in a FlashCopy mapping. The rate at which your applications change their data will give you an indication of the rate at which data will be transferred as part of Copy Services mapping/relationship. Copy Services are also driven by background copy. The rate at which data is transferred is configured on a per-mapping basis for FlashCopy. For Metro/Global Mirror, the rate of data transfer is controlled by the intercluster bandwidth setting.
20
2.7.4 Intersite latency

Table 2-1 on page 16 shows the latency requirements for an intercluster connection. This latency requirement should be met in all situations. At the very least, the peak workload on the intersite link should not result in the latency requirement being missed. If the intersite link is made up of multiple links trunked together, then the link should be architected such that the failure of a single link does not result in the latency requirement being missed.
2.8 Administration of Copy Services

In order to administer Copy Services, users needs access to the SVC Cluster. SVC provides multiple user roles to limit the actions that can be taken via the GUI and the command line. By doing this, system administrators can access the SVC Cluster in a limited capacity to administer the copy relationships associated with their servers storage. These commands are all logged in the Audit Log which can be used for problem resolution and to satisfy auditing requirements
2.8.1 SSH Keys

Access to the SVC Command Line is managed through SSH Keys. To provide a user with access to the SVC Command Line you should create a public-private key pair. The private key should be given to the user and kept safe. The public key should be uploaded to the SVC Cluster and given a descriptive label that uniquely describes that user. This label is used in the audit log (see 2.8.2, Command Auditing) Access to the SVC GUI is managed through ICAT users.
2.8.2 Command Auditing

SVC keeps an audit log of all successfully executed commands. With appropriate configuration of SSH keys and ICAT users, you can use the audit logs to track which users made what changes to the cluster and when they made the changes.
2.8.3 Authorization Roles

The following roles are available Service Monitor CopyOperator Administrator These roles are applied to the public keys which have been uploaded to the cluster. When a user connects to the cluster, they do so with a specific private key. This corresponds to a public key on the cluster. Each command that they submit is checked against that public keys authorization role before being executed and may fail if it does not have the required level of authority. The commands that each role is authorized to execute are give in Table 2-3.
21
Table 2-3 Commands available to each Role Role Service Allowed commands The following commands: svcinfo: all commnads svcservicetask: all commands The following commands: svcinfo: all commands. svctask : finderr, dumperrlog, dumpinternallog svcservicetask : finderr, dumperrlog svcconfig: backup All commands allowed for Monitor role plus: svctask : prestartfcconsistgrp startfcconsistgrp, stopfcconsistgrp chfcconsistgrp prestartfcmap startfcmap, stopfcmap chfcmap startrcconsistgrp, stoprcconsistgrp switchrcconsistgrp chrcconsistgrp startrcrelationship, stoprcrelationship switchrcrelationship chrcrelationship chpartnership All commands can be submitted by users of this role
Monitor
CopyOperator
Administrator
Up to 100 different keys can be stored on a cluster, so there is ample scope for assigning individual keys (and so roles) to systems or system administrators. Currently, there is no way to limit which copy relationships a user can affect. A CopyOperator can execute commands against any FlashCopy mapping or Metro/Global Mirror relationship. However, the audit log will track all commands that are executed so rogue operators cannot act with impunity. Chapter 8, Automation goes into more details about authorization and auditing.
2.9 Removing workarounds with 4.2.1 new functions

If you are already running SVC Copy Services then its quite possible that you implemented processes to work around the limitations of the FlashCopy and Metro/Global Mirror limitations. For instance, you may have a requirement for FlashCopies to be taken of the same VDisk in a short period of time. Before Incremental FlashCopy and Multiple Target FlashCopy, this may have been impossible as the FlashCopy mappings may have needed a too large period of time to complete. With the new functions that SVC brings, you have a lot more flexibility as to when you can start new mappings involving VDisks already in a FlashCopy mapping. Chapter 5, FlashCopy and Chapter 6, Implementing FlashCopy explain the new functionality. The changes to Metro/Global Mirror in 4.2.1 are mainly in the restrictions; they have become less restrictive. With fewer constraints, Copy Services should become simpler to manage.
22
7574 Remote Copy.fm
Chapter 3.
Remote Copy: Metro Mirror and Global Mirror

In this chapter we will introduce Remote Copy services, Remote Copy comprises of two methods of copying: one is Metro Mirror and the other is Global Mirror . Metro Mirror is designed used for shorter distances and is a synchronous copy; Global Mirror is used for long distances and is asynchronous. We will discuss the functionality and how we can use Remote Copy services in our daily operations and for business continuity.
23
7574 Remote Copy.fm
3.1 Remote Copy services overview

The terminology Metro Mirror and Global Mirror are IBM branded terms for the functions Synchronous Remote Copy and Asynchronous Remote Copy respectively. Throughout this book the term Remote Copy is used to refer to both functions where the text applies to either equally. In the topics that follow we discuss some of the many features and functionality of Metro Mirror and Global Mirror.
3.1.1 Metro Mirror

Metro Mirror works by establishing a synchronous copy relationship between two VDisks of equal size. This can be an intracluster relationship, that is within the same I/O group of one SVC cluster, or intercluster, which means a relationship between two SVC clusters that are separated by distance. Those relationships can be standalone or in a Consistency Group. Figure 3-1 shows the function of Metro Mirror relationship.
(1)Write
(4) Ack (2) Mirror write
Remote copy CACHE
Remote copy CACHE
(3) Ack write
Vdisk (Primary)
Vdisk (Secondary)
Metro Mirror Relationship
Figure 3-1 Shows the functionality of Metro Mirror write operation.
The Metro Mirror functionality ensures that I/Os are committed to both the primary and secondary VDisks before sending confirmation of the completion to the server. This ensures that the secondary VDisk is synchronized with the primary VDisk in case of a failure. The secondary VDisk is in a read-only state and manual intervention is required to change that access to read/write state. The server administrator will also need to mount the secondary disk so the application can start to use that VDisk.
Global Mirror
Global Mirror copy relationships work in a similar way as Metro Mirror does but by establishing an asynchronous copy relationship between two VDisks of equal size. This is mostly intended for intercluster relationships over long distances. 24
SVC Advanced Copy Services for Business Continuity
7574 Remote Copy.fm
Global Mirror functionality is that a confirmation is sent to the server before it has received good completion at the secondary VDisk. When a write is sent to a primary VDisk it is given a sequence number. Mirror writes sent to the secondary VDisk are committed in sequence number order. If a write is issued, while another write is outstanding, it may be given the same sequence number. Figure 3-2 shows the sequence.
(1)Write
(2) Ack (3) Mirror write
Remote copy CACHE
Remote copy CACHE
(4) Ack write
Vdisk (Primary)
Vdisk (Secondary)
Global Mirror Relationship
Figure 3-2 Global mirror write operation
Therefore, if a host has submitted concurrent writes, the SVC will batch them together (because the host does not care in what order they are written). The secondary VDisk is therefore consistent with the primary. If a host is writing to the same block, before the previous write is committed, the next write is held until the previous write is completed.
3.1.2 Cluster partnership

When creating an SVC cluster copy partnership, we are connecting two SVC clusters together which are separated by distance. After the SVC cluster partnership creation has been configured on both clusters, further communication between the nodes in each of the clusters is established and maintained via the SAN network. The intercluster link, see 3.1.3, Intercluster communication on page 26, controls the traffic between the nodes and helps to co-ordinate activity between the cluster. All intercluster communication goes through the Fibre Channel network. The SVC does not require a control network or fabric to be installed to manage Remote Copy services. The control link controls the copy states and co-ordinates updates at either end. The control link is implemented on top of the same FC connection as the SVC uses for Remote Copy I/Os. If communication between the SVC clusters is interrupted or lost, an error is logged and all copy relationships will stop automatically.
Chapter 3. Remote Copy: Metro Mirror and Global Mirror
25
7574 Remote Copy.fm
3.1.3 Intercluster communication

Each SVC cluster can be configured with one other SVC cluster as its partner. When both clusters are correctly configured and can communicate with each other over the fabric, they establish further communication facilities between the nodes on each of the clusters.This is called the intercluster link. The intercluster link keeps information about the total bandwidth available from the local cluster to the remote cluster that is to be assigned to background copy process. This is used to govern the rate at which the background copy is performed. The intercluster link can be changed during normal operation.The default value for this parameter is 50 MBps.
3.1.4 Maintenance of the intercluster link

All SVC nodes maintain a database of the other devices that are visible on the fabric. This is updated as devices appear and disappear. Devices that are identified as SVC nodes are categorized according to the SVC cluster to which they belong. SVC nodes that belong to the same cluster establish communication channels between themselves and begin to exchange messages to implement the clustering and functional protocols of the SVC. Nodes that are in different clusters do not exchange messages after the initial discovery is complete, unless they have been configured in a Remote Copy relationship. The intercluster link carries the control traffic to coordinate activity between the two clusters. Through this intercluster link there is a message sent between the nodes that just checks if a connection is established, and that is referred to as a heartbeat between the clusters. It is formed between one node in each cluster which is termed the focal point. The traffic between the focal point nodes is distributed among the logins that exist between those nodes. If the focal point node should fail (or all its logins to the remote cluster fail), then a new focal point is chosen to run the control traffic. Note: If we change the focal point node, we would cause I/O to pause, but copy relationships will not enter the consistent stopped state. The software that controls the intercluster link monitors for link errors and will exclude the link if the error rate gets too high: An intermittent hardware problem on a non-redundant link that leads to a link reset for the SVC cluster. Lost frames or delay exceeding 1.5 seconds affecting the heartbeats between the two clusters. A problem leading to excessive data frames being dropped (even where non-data frames are passed successfully). A series of failures that mean that progress is not made, such that a message is not delivered for more than 15 seconds. If there are errors that lead to the heartbeat timeout being detected on one of the clusters, that cluster will close all process logins between the focal point nodes. The SVC cluster monitors for errors on the intercluster link and if an error reoccurs within 30 seconds, it is ignored but if there are more than three 30 second periods, that experience an 26
7574 Remote Copy.fm
error event, the intercluster link is excluded. Then, the focal point node will deliberately prevent an intercluster link being created between the local cluster and the remote cluster. The other cluster will report partially_configured in its configuration state for the remote cluster. It is likely the other cluster will also have detected the same set of errors and excluded the link on the same basis; in which case both clusters will report partially_configured. If only one cluster detects an error and excludes the link, the cluster that does not detect the error will continue to report fully_configured.
3.1.5 I/O channel

The I/O channel is the term used to describe the connections between clusters which are used to deliver mirror writes between the clusters when using Remote Copy (intercluster). Intracluster Remote Copy uses connectivity within a node to deliver I/O to secondary virtual disks. I/Os are never routed between nodes in different I/O groups. The I/O channel uses the same protocols for communication between the nodes in different clusters as is used between the nodes within a local cluster, This requires only on round trip to deliver write and receive its completion. The I/O channel traffic is monitored using the same scheme as described in 3.1.4, Maintenance of the intercluster link on page 26.
3.1.6 Background copy

The bandwidth that background copy can use is configured on the intercluster link as stated in 3.1.3, Intercluster communication on page 26. That bandwidth is divided evenly between the nodes that are performing background copy for active copy relationships. This bandwidth allocation is independent from the number of disks that a node is responsible for. Each node, in turn, divides its bandwidth evenly between the multiple relationships performing background copy. The default value for this feature is 50MBps and this value should be set to less or equal to the bandwidth that can be sustained by the intercluster link. For intracluster relationships each node is assigned 25 MBps for background copy. Note: Background copy will proceed from low LBA to high LBA in sequence.
Bandwidth impact on foreground I/O latency

The background copy bandwidth determines the rate at which the background copy will be attempted. The background copy bandwidth can affect foreground I/O latency in one of three ways: The following results can occur if the background copy bandwidth is set too high compared to the intercluster link capacity: There is a delay in the synchronous secondary writes of foreground I/Os. The foreground I/O latency will increase as perceived by applications. If the background copy bandwidth is set too high for the storage at the primary site, background copy read I/Os overload the primary storage and delay foreground I/Os. If the background copy bandwidth is set too high for the storage at the secondary site, background copy writes at the secondary overload the secondary storage and again delay the synchronous secondary writes of foreground I/Os.
27
7574 Remote Copy.fm
In order to set the background copy bandwidth optimally, make sure that you take into consideration all aspects of your environments. The three biggest contributing resources are the primary storage, the intercluster link bandwidth, and the secondary storage. Provision the most restrictive of these three resources between the background copy bandwidth and the peak foreground I/O workload.
3.1.7 Remote Mirror relationship

A Remote Mirror relationship is a relationship between two individual VDisks of the same size and if both VDisks are in the same cluster they must both be within the same I/O group. These disks are called a master VDisk and an auxiliary VDisk, for more details see 3.1.8, Copy relationship naming convention on page 28. A primary VDisk contains a valid copy of host data and is accessible for host write I/Os. A secondary VDisk, if consistent, contains a valid copy of host data but is not available for host write I/Os. In an ordinary relationship, (not idling) with one primary VDisk and one secondary VDisk, Remote Copy will try to make the data on the secondary VDisk match the data on the primary VDisk by performing writes to the secondary virtual disk. The following types of VDisk make a copy relationship. Primary VDisk - enabled for read and write I/Os Secondary VDisk - enabled only for read I/Os Note: When the copy relationship is in the IDLE state, both primary and secondary are enabled for read and write I/Os. When a Remote Copy relationship is first created, the master VDisk is always assigned the role of primary, and the auxiliary VDisk is assigned the role of secondary. Hence, the copying direction is from master to auxiliary. These roles might be reversed and the copying direction will then be from auxiliary to master. The direction of copying is reflected in the state of the system. A Global Mirror and Metro Mirror copy relationship have the same characteristics and use the same set of commands to create and control the relationship. The relationship between the VDisks persists until it is deleted.
3.1.8 Copy relationship naming convention

The SVC introduces the following terms in Remote Copy. An explanation of those commonly encountered terms follows. Master VDisk is the one that is created as master and it will always be the master. When creating a master VDisk by default it will become the master/primary VDisk open for write I/Os from the application. Auxiliary VDisk is a partner vdisk to the master vdisk. When the copy relationship is created all previous data on that VDisk is destroyed. This disk will always be the auxiliary VDisk in this relationship. Primary VDisk is the production VDisk and allows read/writes. Updates to this VDisk are mirrored to the secondary VDisk. This state can be changed when changing copy directions. When changing the copy direction we are enabling write I/Os to the secondary VDisk and the primary will be set to read-only.
28
7574 Remote Copy.fm
Secondary VDisk in an active relationship is always in read-only mode when it contains consistent image of the primary VDisk. This role changes when you change the copy direction. Figure 3-3 shows how the role name follows the copy direction.
Master VDisk
Auxiliary VDisk
Role: Primary Role: Secondary
Copy direction
Role: Secondary Role: Primary
Copy direction
Figure 3-3 Naming conventions for VDisks in copy relationship.
When you create a copy relationship you need to state which VDisk is the master VDisk, and the other VDisk in that relationship will become the auxiliary. This relationship does not change even if you change copy directions. Note: When creating the SVC cluster partnership neither cluster is master/primary or auxiliary/secondary. They are both equal in this partnership and when you use the mkpartnership command you are using the only Remote Copy command that needs to be issued to both clusters, all other Remote Copy commands need be issued from only one of the clusters in the relationship.
3.1.9 Supported methods for synchronizing

This section describes three methods that can be used to establish a relationship.
Full synchronization after Create

This is the default method. It provides full copy of the primary VDisk to the secondary VDisk. It is the simplest in that it requires no administrative activity apart from issuing the necessary commands. However, in some environments the bandwidth available will make this method unsuitable. The sequence for a single relationship is as follows: A mkrcrelationship is issued without specifying the -sync flag. A startrcrelationship is issued without the -clean flag.
29
7574 Remote Copy.fm
Synchronized before Create

In this method, the administrator must ensure that the primary and secondary virtual disks contain identical data before creating the relationship. There are two ways in which this might be done: Both disks are created with the -fmtdisk feature so as to make all data zero. A complete tape image (or other method of moving data) is copied from one disk to the other. In either technique, no write I/O must take place to either primary or secondary before the relationship is established. Then, the administrator must ensure that: A mkrcrelationship is issued with the -sync flag. A startrcrelationship is issued without the -clean flag. If these steps are not performed correctly, the relationship will be reported as being consistent, when it is not. This is likely to make any secondary disk useless. This method has an advantage over the full synchronization, in that it does not require all the data to be copied over a constrained link. However, if the data needs to be copied, the master and auxiliary disks cannot be used until the copy is complete, which might be unacceptable.
Quick synchronization after Create

In this method, the administrator must still copy data from primary to secondary. But it can be used without stopping the application at the primary. The administrator must ensure that: A mkrcrelationship is issued with the -sync flag. A stoprcrelationship is issued with the -access flag. A tape image (or other method of transferring data) is used to copy the entire primary disk to the secondary disk. Once the copy is complete, the administrator must ensure that: A startrcrelationship is issued with the -clean flag. With this technique, only the data that has changed since the relationship was created is copied from the primary and secondary. As with Synchronized before Create on page 30, the copy step must be performed correctly, or else the secondary will be useless, though the copy will report it as being synchronized. Note: In an SVC cluster relationship it is a good rule to use the same SVC cluster for your master VDisks, even if the master SVC cluster contains the secondary VDisk.
3.1.10 Remote mirror consistency groups

A consistency group consist of one or more copy relationships. By grouping the relationships we ensure that these relationship are managed so that data within the group is in a consistent state. If the consistency group is empty, it acquires the type of the first relationship that is added to it, Metro Mirror or Global Mirror. Therefore, each relationship that you add to an already in-use consistency group must be of the same type. Consistency groups address the issue where the objective is to preserve data consistency across multiple Remote Mirrored VDisks because the applications have related data which spans multiple VDisks. A requirement for preserving the integrity of data being written is to ensure that dependent writes are executed in the application's intended sequence.
30
7574 Remote Copy.fm
Copy commands can be issued to either Metro Mirror or Global Mirror consistency groups, which affects all Remote Mirror relationships in that consistency group, or to a single standalone Remote Mirror relationship, if it is not part of a consistency group. In summary: Remote Mirror relationships can be part of a consistency group, or be stand-alone and therefore handled as single instances. A consistency group can contain zero or more relationships. An empty consistency group, with zero relationships in it, has little purpose until it is assigned its first relationship, except that it has a name. All the relationships in a consistency group must have matching master and auxiliary SVC clusters. All relationships must be of the same type. The rules behind consistency mean that certain configuration commands are prohibited, where this would not be the case if the relationship was not part of a consistency group.
3.2 State overview

After creation of a Remote Copy relationship, that relationship has a state at any given time. Stand- alone relationships and consistency groups share a command configuration and state modes. We will discuss those states and start with the three most common ones as shown in Figure 3-4 on page 32 :connected versus disconnected, consistent versus inconsistent and consistent versus synchronized.
31
7574 Remote Copy.fm
inconsistent
INCONSISTENT S TOPPED
Start
INCONSISTENT C OPYING
Stop create_unsync
Forced Start out of sync
IDLING
Forced Start
Background copy complete consistent
Stop enable access
create_sync
C ONSISTENT S TOPPED
Start
C ONSISTENT S YNCHRONIZED
Start in sync
Stop
Legend
Stop enable access
R EMOTE C OPY S TATE
Remote Copy Event
Figure 3-4 State overview.
3.2.1 Connected or disconnected

This distinction can arise when a Remote Copy relationship is created with two virtual disks in different clusters. It is possible that the communication line between those clusters gets broken, and therefore communication between the two clusters is lost. When the two clusters can communicate, the clusters and the relationships spanning them are described as connected. When they cannot communicate, the clusters and the relationships spanning them are described as disconnected. If the cluster cannot communicate, each of the clusters is left with half the relationship and has only a portion of the information that was available to it before. Some limited configuration activity is possible, but not all. When the clusters can communicate again the relationships become connected again. Remote Copy automatically reconciles the two state fragments, taking into account any configuration or other event that took place while the relationship was disconnected.
32
7574 Remote Copy.fm
As a result, the relationship can either return to the state it was in when it became disconnected, or it can enter a different connected state. Note: Relationships that are configured between virtual disks in the same cluster will never be described as being in a disconnected state since only one cluster is involved.
3.2.2 Consistent or inconsistent

A copy relationship between primary VDisk and a secondary VDisk can be described as being consistent or inconsistent. Consistency Groups containing copy relationships can also be described as being consistent or inconsistent. The Consistent/Inconsistent attribute describes the relationship of the data which exists on the secondary VDisk to that which exists on the primary VDisk. It can be considered a property of the secondary VDisk itself. A secondary VDisk is described as consistent if it contains data that could have been read by a host system from the primary if, for example, power had failed at some point in time while I/O was in progress and the power was later restored. This imaginary point in time is defined as the recovery point. The requirements for consistency are expressed with respect to activity at the primary up to the recovery point: The secondary VDisk contains the data from all writes to the primary for which the host had received good completion and that data had not been overwritten by a subsequent write (before the recovery point) If we look at it from the host site then consistency means that a secondary VDisk contains the same data as the primary VDisk at the recovery point (the time at which the example power failure occurred). If an application is designed to cope with unexpected power failure, this guarantee of consistency means that the application will be able to use the secondary and begin operation just as if it had been restarted after the hypothetical power failure. Again, the application is dependent on the key properties of consistency: Write ordering Read stability for correct operation at the secondary If a relationship, or set of relationships, is inconsistent and an attempt is made to start an application using the data in the secondaries, a number of outcomes are possible: The application might decide that the data is corrupt and crash or exit with an error code. The application might fail to detect the data is corrupt and return erroneous data. The application might work without a problem. Because of the risk of data corruption, and in particular undetected data corruption, Metro Mirror strongly enforces the concept of consistency and prohibits access to inconsistent data. Consistency as a concept can be applied to a single relationship or a set of relationships in a consistency group. Write ordering is a concept that an application can maintain across a number of disks accessed through multiple systems and therefore consistency must operate across all those disks. When deciding how to use consistency groups, the administrator must consider the scope of an applications data, taking into account all the interdependent systems which communicate and exchange information.
33
7574 Remote Copy.fm
If two programs or systems communicate and store details as a result of the information exchanged, then either of the following actions might occur: All the data accessed by the group of systems must be placed into a single consistency group. The systems must be recovered independently (each within its own consistency group). Then, each system must perform recovery with the other applications to become consistent with them.
3.2.3 Consistent or synchronized

A copy which is consistent and up-to-date is described as synchronized. In a synchronized relationship, the primary and secondary VDisks are only different in regions where writes are outstanding from the host. Consistency does not mean that the data is up-to-date. A copy can be consistent and yet contain data that was frozen at some point in time in the past. Write I/O might have continued to a primary and not have been copied to the secondary. This state arises when it becomes impossible to keep up-to-date and maintain consistency. An example is a loss of communication between clusters when writing to the secondary. When communication is lost for an extended period of time, Metro Mirror tracks the changes that happen at the primary, but not the order of such changes, nor the details of such changes (write data). When communication is restored, it is impossible to make the secondary synchronized without sending write data to the secondary out-of-order, and therefore losing consistency. Two policies can be used to cope with this: Take a point-in-time copy of the consistent secondary before allowing the secondary to become inconsistent. In the event of a disaster before consistency is achieved again, the point-in-time copy target provides a consistent, though out-of-date, image. Accept the loss of consistency during resync, and loss of useful secondary, while making it synchronized.
3.2.4 Detailed Information about copy states.

The available states are described in detail in this section, and they are: Consistent Stopped Consistent Synchronized Inconsistent Stopped Inconsistent Copying Idling Idling disconnected Inconsistent disconnected Consistent disconnected Empty (applies only to Consistency Groups)
Consistent Stopped
This is a connected state. In this state, the secondary contains a consistent image, but it might be out-of-date with respect to the primary. This state can arise when a relationship was in consistent synchronized state and suffers an error which forces a consistency freeze. It can also arise when a relationship is created with a sync parameter set. 34
7574 Remote Copy.fm
Normally, following an I/O error, subsequent write activity cause updates to the primary and the secondary is no longer synchronized. In this case, to re-establish synchronization, consistency must be given up for a period. A Start command with the Force option must be used to acknowledge this, and the relationship or consistency group transitions to inconsistent copying. Do this only after all outstanding errors are repaired. In the unusual case where the primary and secondary are still synchronized (perhaps following a user stop, and no further write I/O was received), a Start command takes the relationship to consistent synchronized. The No Force option is required. Also in this unusual case, a Switch command is permitted which moves the relationship or consistency group to consistent synchronized and reverses the roles of the primary and secondary (switches copy directions). If the relationship or consistency group becomes disconnected, then the secondary side transitions to consistent disconnected. The primary side transitions to idling disconnected. An informational status log is generated every time a relationship or consistency group enters the consistent stopped with a status of Online state. Note: It is possible to create some form of automation that monitors the status log. The SVC cluster has the possibility to send an SNMP trap that triggers an automation script that issues the start command in a case of loss of synchronization when certain log entries appear.
Consistent Synchronized
This is a connected state. In this state, the primary VDisk is accessible for read and write I/O and the secondary VDisk is accessible for read-only I/O. This state is also called normal copy state and indicates that now your relationship is both consistent and synchronized. Writes that are sent to the primary VDisk are sent to both primary and secondary VDisks. Either good completion must be received for both writes, or the write must be failed to the host, or a state transition out of consistent synchronized must take place before a write is completed to the host. A stop command takes the relationship to consistent stopped state. A stoprcrelationship command with the -access argument takes the relationship to the Idling state. A switch copy direction command leaves the relationship in the consistent synchronized state, but reverses the primary and secondary roles and therefore the copy direction. A Startrcrelationship or startrcconsistgrp command is accepted, but has no effect. If the relationship or consistency group becomes disconnected, the same transitions are made as for consistent stopped.
Inconsistent Stopped
This is a connected state. In this state, the primary is accessible for read and write I/O but the secondary is not accessible for either. The command start copy process needs to be given to make the secondary consistent. This state is entered when the relationship or consistency group was inconsistent copying and has either suffered a persistent error or received a stop command which has caused the copy process to stop.
35
7574 Remote Copy.fm
A Start command causes the relationship or consistency group to move to the inconsistent copying state again. A stop command is accepted, but has no effect. If the relationship or consistency group becomes disconnected, the secondary side transitions to inconsistent disconnected. The primary side transitions to idling disconnected.
Inconsistent Copying
This is a connected state. In this state, the primary is accessible for read and write I/O but the secondary is not accessible for either read or write I/O. Inconsistent copying runs as a background copy process, which copies data from the primary to the secondary VDisk. This state is entered after a start command is issued to an inconsistent stopped relationship or consistency group. It is also entered when a forced start is issued to an idling or consistent stopped relationship or consistency group. In the absence of errors, an inconsistent copying relationship is active, and the copy progress increases until the copy process completes. In some error situations, the copy progress might freeze or even return to a known state. A persistent error or stop copy process command places the relationship or consistency group into an inconsistent stopped state. A start command is accepted, but has no effect. If the background copy process completes on a standalone relationship, or on all relationships for a consistency group, the relationship or consistency group transitions to consistent synchronized. If the relationship or consistency group becomes disconnected, then the secondary side transitions to inconsistent disconnected. The primary side transitions to idling disconnected.
Idling
This is a connected state. Both master and auxiliary disks are operating in the primary role. Consequently both are accessible for write I/O. In this state, the relationship or consistency group accepts a start command. Metro Mirror maintains a record of regions on each disk which received write I/O while Idling. This is used to determine what areas need to be copied following a start command. The Start command must specify the new copy direction. A Start command can cause a loss of consistency if either VDisk in any relationship has received write I/O. This is indicated by the synchronized status. If the start command would lead to a loss of consistency, then the Force parameter must be specified. Following a Start command, the relationship or consistency group transitions to consistent synchronized if there is no loss of consistency, or to inconsistent copying if there is such a loss. Also, while in this state, the relationship or consistency group accepts a Clean option on the start command. If the relationship or consistency group becomes disconnected, then both sides change their state to idling disconnected.
IdlingDisconnected
This is a disconnected state. The VDisk or disks in this half of the relationship or consistency group are all in the primary role and accept read and write I/O.
36
7574 Remote Copy.fm
The main priority in this state is to recover the link and make the relationship or consistency group connected once more. No configuration activity is possible (except for deletes or stops) until the relationship becomes connected again. At that point, the relationship transition to a connected state. The exact connected state which is entered depends on the state of the other half of the relationship or consistency group, which depends on: The state when it became disconnected The write activity since it was disconnected The configuration activity since it was disconnected If both halves are idling disconnected, then the relationship becomes idling when reconnected. While idling disconnected, if a write I/O is received which causes loss of synchronization and the relationship was not already stopped (either through user stop or a persistent error), then an error log entry is raised to notify this. This error log is the same as that raised when the same situation arises when consistent synchronized.
Inconsistent Disconnected
This is a disconnected state. The VDisks in this half of the relationship or consistency group are all in the secondary role and do not accept read or write I/O. No configuration activity except for deletes is permitted until the relationship becomes connected again. When the relationship or consistency group becomes connected again, the relationship becomes inconsistent copying automatically unless either: The relationship was inconsistent stopped when it became disconnected The user issued a stop command while disconnected In either case, the relationship or consistency group becomes inconsistent stopped.
Consistent Disconnected
This is a disconnected state. The VDisks in this half of the relationship or consistency group are all in the secondary role and accept read I/O but not write I/O. This state is entered from consistent synchronized or consistent stopped when the secondary side of a relationship becomes disconnected. In this state, the relationship or consistency group displays an attribute of FreezeTime ( see 3.4.5, Freeze/Run on consistency groups on page 40) which is the point in time that consistency was frozen. When entered from consistent stopped, it retains the time it had in that state. When entered from consistent synchronized, the FreezeTime shows the last time at which the relationship or consistency group was known to be consistent. This corresponds to the time of the last successful heartbeat to the other cluster. A stop command with enable write access to secondary transitions the relationship or consistency group to an idling disconnected state. This allows write I/O to be performed to the VDisks and is used as part of a disaster recovery scenario. When the relationship or consistency group becomes connected again, the relationship or consistency group becomes consistent synchronized only if this does not lead to a loss of consistency under these criteria: The relationship was consistent synchronized when it became disconnected.
37
7574 Remote Copy.fm
No writes received successful completion at the primary while disconnected. Otherwise the relationship become consistent stopped. The FreezeTime setting is retained.
Empty
This state only applies to consistency groups. It is the state of a consistency group which has no relationships and no other state information to show. It is entered when a consistency group is first created or a all relationship are removed from the consistency group.It is exited when the first relationship is added to the consistency group at which point the state of the relationship becomes the state of the consistency group.
3.3 Metro Mirror and Global Mirror configuration limits

Table 3-1 shows the Metro Mirror and Global Mirror configuration limits. All the limits in this table apply to the aggregate of the whole Remote Copy configuration, regardless of whether an individual relationship, or group, is Metro or Global Mirror.
Table 3-1 Metro Mirror and Global Mirror configuration limits. Parameter Number of consistency groups Number of relationships Total VDisk size per I/O group on both clusters Value 256 per SVC cluster 1024 per SVC cluster 1024 TB is the per I/O group limit on the quantity of primary and secondary VDisk address spaces that can participate in relationships, the default is 40TB
3.4 Remote Copy internals

The following describe some aspects of the internal functions of Metro Mirror and/or Global Mirror.
3.4.1 Read stability

As mentioned in Chapter 2, Planning for Advanced Copy Services implementation on page 5, read stability can by defined as: If two reads are submitted for the same area and those reads complete successfully and there was no intervening (in time) or overlapping (in time) write activity then the two reads must return the same data. A number of applications that perform recoveries following power failure or crashes depend on this property for correct behavior. Many implementations of mirroring systems fail to maintain read stability. Following a power loss or reset scenario this non-volatile record is used to ensure that both mirrors are made consistent by copying data from one disk to the other. Reads are only allowed when stability is assured.
38
7574 Remote Copy.fm
Note: Read stability is also known as Mirror Write Consistency. This definition of read stability makes no guarantee of data returned to reads while writes are outstanding, only where reads are issued after writes have completed. It is possible to conceive of an application that issues writes and without waiting for write completion, issues reads to the same region. Such an application, on seeing new data being returned for its read, might conclude that the Write data is indeed committed. Such an application is not satisfied by this definition of Read Stability. Remote Copy only guarantees Read Stability following a Write Completion.
3.4.2 Write ordering

Many applications that use block storage have a requirement to survive failures such as loss of power, or a software crash, and not lose data that existed prior to the failure. Since many applications need to perform large numbers of update operations in parallel to that storage, maintaining write ordering is key to ensuring the correct operation of applications following a disruption. If an application is performing a large set of updates, it needs to be designed with the concept of allowing dependent writes. These are writes where it is important to ensure that an earlier write has completed before a later write is started. Reversing the order of dependent writes can undermine the applications algorithms and can lead to problems such as detected, or undetected, data corruption.
3.4.3 Write consistency

Remote Copy maintains write consistency where it ensures that, while primary VDisk and secondary VDisk are synchronized, the VDisks stays in sync even in the case of failure in the primary cluster or other failures which cause the results of writes to be uncertain. After such a failure it is important to ensure that any data reads at the primary VDisk will also be read at the secondary VDisk, since subsequent writes will be dependent on the results of the read data for correctness. It has been argued that Remote Copy does not require this support since during normal operation as only the primary disk is active, and the secondary is in a read only state. Only if an unusual event, forces us to use the secondary VDisk and the content of that disk is examined. SVC Remote Copy Services uses mirrored non-volatile record marks to protect against this. This will lead to a very small increase in processor work, since extra protocol is involved, but should have no impact on I/O latency (unless I/O throughput is constrained by SVC), or bandwidth.
3.4.4 Write completion

To ensure read stability (see 3.4.1, Read stability on page 38) Remote Copy must not allow reads to complete successfully to areas which are out of sync except where there is currently an outstanding write to that region. During normal operation, it is only regions that have outstanding writes that are out of sync and so reads do not need to be delayed and do not need to be tracked. When an I/O receives an error, then processing of new read I/Os is suspended, and this is achieved before the failed
39
7574 Remote Copy.fm
write is completed. This situation is resolved before any read is released, either by forcing the relationship ConsistentStopped or by failing new and pending I/O.
3.4.5 Freeze/Run on consistency groups

The algorithm that consistency is based on is called Freeze/Run. This algorithm freezes I/O in the consistency group before allowing the next I/O to commit, and when restarted it will only allow I/Os to the primary VDisk. A Freeze/Run cycle is required when the following has occurred: A Stop command received from the User An I/O command to either Primary or Secondary fails such that a host write is not known to be copied to both disks and the Primary disk has not become path offline. (In such circumstances a Freeze is performed on the primary of the affected relationships) New host write I/O is suspended without issuing writes to either Primary or Secondary Host writes where the data has not been written, with successful completion, to both Primary and Secondary are Stalled. That is, completion to the host is delayed All ongoing write activity is allowed to complete until it has either become Stalled or completed to the host only if both Primary and Secondary writes have completed New Read I/O is suspended
FreezeTime
When the freeze has been achieved across all relationships in a consistency group and any system transients (such as path offline status associated with connectivity to the controllers) have completed, then a decision is made as to how to proceed. An attempt might be made to perform a recovery copy which, if successful, allows I/O to resume without loss of synchronization. Otherwise, assuming the primary is still online , then a Run is performed with the relationship consistentstopped. As a result of a run I/O is resumed which means: Host I/Os are allowed to complete New I/Os are allowed to proceed issuing writes only to the primary Read I/Os are allowed to proceed When a consistency group is in the state of Consistent Stopped, there is a parameter called freeze_time, see Example 3-1. It shows the time in YY/MM/DD/HH/MM format, and the time it shows is the last known time when the state between primary and secondary was consistent synchronized.
Example 3-1 The freeze time parameter.
IBM_2145:ITSOCL2:admin>svcinfo lsrcconsistgrp 254 id 254 name ITSO_SERVER_GRP master_cluster_id 000002006B603A38 master_cluster_name ITSOCL2 aux_cluster_id 0000020068403A42 aux_cluster_name ITSOCL3 primary master state consistent_stopped relationship_count 2 freeze_time 2007/11/08/20/14/54 status online sync in_sync copy_type metro RC_rel_id 2 40
7574 Remote Copy.fm
RC_rel_name ITSO_SERVER_01 RC_rel_id 9 RC_rel_name ITSO_SERVER_02 The most useful value of the FreezeTime is on the secondary VDisk since the primary sets its value when it first finds out that the relationship is consistent
3.4.6 Management of difference map

The difference map performs change recording. This records which areas of the disk were not copied from primary to secondary or were changed since they were copied, as well as recording which areas have writes in flight when synchronized or copying.
Space management
A difference map is maintained on all online nodes that are members of the I/O group of a primary VDisk. Space for a different map is permanently reserved for the secondary VDisk for use when it needs to act as a primary The difference map is implemented as a non-volatile bitmap, one bit corresponding to a grain of data, each grain corresponding to 256kB of aligned disk space. This is the unit of background copy (as well as recovery copy). The difference map space is allocated in units corresponding to 8GB of disk space. (4KB * 8 bits * 256kB). Whole units are allocated to each VDisk, with any excess being lost. The limit is expressed in terms of VDisk space in each I/O group. Note: When creating a copy relationship with VDisks that are less than 8GB we are still reserving bitmap space for 8GB, so when making a copy relationship it is a good practice to scale the size of the VDisks in increments of 8GB so as not to waste any bitmap space. Within each I/O group it includes both primary and secondary VDisks
Difference map state

The difference map may be in one of three global states, recorded in cluster state. The grain state at a specific moment can be clean, out of sync, or dirty. Clean means that there is no I/O in flight and the primary and secondary are synchronized. Out of sync means that the primary and secondary are synchronized, but I/O is in flight and must complete before being marked clean. It could also mean that after a disruption or failure, RecoveryCopy is required to complete successfully before declaring it clean. If a grain is dirty, the primary and secondary are not synchronized. Each grain may be in a state no worse than the difference map (that is to say, dirty difference maps can have grains that are clean or dirty, but a clean difference map has all grains clean, and hence no I/O in flight). Both OutOfSync and Dirty are recorded the same in the non-volatile bitmap. They are distinguished by the global state.
Messaging
When two nodes are present in an I/O group they will exchange messages to ensure that each is kept up-to-date with respect to changes required in the difference map:
41
7574 Remote Copy.fm
When a write takes place to a grain that has been synchronized the map must be marked out of sync on both nodes before the I/O is allowed to proceed. The Unlock message is not sent until both primary and secondary I/Os have completed successfully. If either write fails an UnlockDoneFailed message is sent. When a background copy or recovery copy operation has successfully synchronized a grain, the grain must be marked clean on both nodes. The node that owns the grain sends a message to the partner node to mark the grain Clean. This is replied to by using a CleanDone message. When an I/O touches a grain that is synchronized but it is not going to perform a write to the secondary (e.g. it is in ConsistentStopped state) then the grain must be marked Dirty on both nodes. A node sends a DirtyGrain message and receives a DirtyDone reply from the partner. When multiple write I/Os are active in the same synchronized grain each requires its own Lock/LockDone/Unlock exchange with the partner node. Only when the last I/O completes within a grain and no I/O that was active in that grain suffered a write failure is the grain marked Clean in each node.
Effect of node failure

When nodes appear and disappear the difference maps must be resynchronized so that both nodes start (or restart) I/O operation with identical maps. When a node disappears for a long period of time and is survived by its partner node, the partner node must delete all the failed nodes maps, because they are invalid. This invalidation must complete before allowing any I/O to progress on the assumption that it has an assured difference map mark in place. An I/O only proceeds when all up-to-date difference maps have been marked as required. That is just the local difference map if only one node is operational, but both difference maps if two nodes are operational. For instance, an I/O that dirties a grain must mark both maps dirty before allowing any write I/O to take place. It is possible that through a sequence of node appearance and disappearance, an up-to-date copy of the difference map is not accessible though there is a node in the I/O group which has an online path to the VDisk. In this case, Remote Copy will hold the path offline and reject host I/O. Either the missing node must be recovered or the Remote Copy relationship must be deleted.
3.4.7 Non-volatile journal

The non-volatile journal data structure is used to maintain details of all writes that are active on consistent synchronized relationships. These details include the VDisk, LBA and count of sectors, and for Global Mirror the sequence number that was assigned. These details are used by the Quick Recovery Copy and Reconstruct processes, see 3.4.8, Quick Recovery Copy or Reconstruct on page 43. The contents of the non-volatile journal are maintained on the nodes in the I/O group of the primary VDisk for the relationship, and are updated at the same time as the bitmap bit in the difference map. The same messages that maintain the locks are used to drive the maintenance of the non-volatile journal. The non-volatile journal has the same availability as any of the difference maps associated with the virtual disks in the same I/O group. In the event that the nodes of the I/O group are offline, then all relationships using the non-volatile journal will change from consistent_synchronized to consistent_stopped, at which point the non-volatile journal is deallocated and will be re-established when the nodes reappear. No administrative intervention is required.
42
7574 Remote Copy.fm
3.4.8 Quick Recovery Copy or Reconstruct

Reconstruct and Quick Recovery Copy are used to attempt to maintain synchronization where some disruption has taken place and it is not certain whether I/Os have completed successfully to both primary and secondary. They operate during the freeze state described previously. If successful, then I/O can be resumed without having to change to ConsistentStopped state. These processes are essential to making the ConsistentSynchronized state fault tolerant. Quick Recovery Copy is used for Metro Mirror. Reconstruct is used for Global Mirror. Each uses a record of outstanding I/O, the record of I/Os that must be copied to perform QuickRecoveryCopy or Reconstruct are maintained in the non-volatile journal. Each entry details the range of sectors that were subject to a write I/O that was in flight at the point that the disruption happened. This record is used to drive a process to copy data from the primary to secondary disks. This copy is similar to the process that is used for background copy. While either Reconstruct or Quick Recovery Copy is in progress, I/O is suspended for the Relationship. Essentially Recovery Copy forms part of the Freeze described above. Recovery Copy completes when either all areas in doubt are copied or either Primary or Secondary are Offline or a persistent error occurs during a read. Quick Recovery Copy or Reconstruct is performed by the owner node if two nodes within the I/O Group have online paths to the VDisk, or the one node with an online path within the IO Group if there is only one. Recovery Copy uses the non-volatile difference map to record which grains were subject to an outstanding write I/O. Each grain is copied completely from primary to secondary. This process will only be performed by the current software if a recovery is required in the short period after a software upgrade completes, but before the conversion to use the non-volatile journal is complete.
43
7574 Remote Copy.fm
44
7574 Implementing Metro Mirror and Global Mirror.fm
Chapter 4.
Implementing Remote Copy

In this chapter we will show you how to implement Remote Copy using the GUI and the command line interface. We will demonstrate basic usage and some of the advanced functionality of Remote Copy services.
45
4.1 Implementation
In this chapter we will show how we implemented Remote Copy in our environment. We will use both the GUI and the command line interface to assist us in creating Metro and Global Mirror relationships and consistency groups. Our environment consists of the equipment shown in Table 4-1
Table 4-1 SVC and Subsystem Production site Devices Disaster site Devices SVC ITSOCL2 SVC ITSOCL3 Subsystem DS4500 Subsystem DS4700
We will be using Metro Mirror intercluster relationship between the clusters as shown in Figure 4-1.
Figure 4-1 Our demo setup
Our SVC clusters are ITSOCL2 and ITSOCL3, and we are going to create a copy relationship between those two clusters. Since Metro Mirror and Global mirror are using the same set of commands we will show you in detail how we implement Metro Mirror and we will add additional comments when Global Mirror differs from Metro Mirror. To be able to start the copy relationship there should been established connection between the two clusters through the SAN network and in this chapter we assume that this has been done. For information on how to install the SVC refer to the IBM System Storage SAN Volume Controller, SG24-6423.
46
4.2 Implementation using the GUI

When using the GUI to establish a copy relationship between two VDisks, we will have to logon to both SVC clusters (as stated, we will be using intercluster copy relationship). The only copy relationship command that has to be submitted from both clusters, is the create partnership command. After submitting the create partnership command we can perform all our copy functions from one cluster since this is an active/active partnership and both clusters can control the copy process.
4.2.1 Creation of cluster partnership

We start by logging on to the cluster called ITSOCL2, and press the Create button to create the new partnership, as in Figure 4-2.
Figure 4-2 Creating the partnership between the clusters.
After choosing to create our new partnership we will see our cluster candidates as in Figure 4-3 on page 48.
Chapter 4. Implementing Remote Copy
47
Figure 4-3 Selecting ITSOCL3 as our cluster partner
The candidates are the clusters that are already configured on the SAN and are zoned to our production cluster ITSOCL2. When we are creating the partnership we need to configure the link between the clusters. If we are creating a Metro Mirror Partnership we only need to think of the bandwidth for the link. If we are creating a Global Mirror partnership we need to be aware of the link tolerance between the two SVC clusters. These parameters are described as follows: Bandwidth Specifies the bandwidth that is used by the background copy process between the clusters. It adjusts the bandwidth that is used by Metro Mirror or Global Mirror for the initial background copy process. The bandwidth is set to default to 50 MBps (megabytes per second) if you do not specify it. Set the bandwidth to a value that is less than or equal to the bandwidth that can be sustained by the intercluster link. Link Tolerance ( Use for Global mirror only) Defines how sensitive the cluster is to inter-link overload tolerance. The value specified are the seconds that the cluster will tolerate continued difficulties before the cluster will suspend the copy function to prevent host I/O impact. The parameter accepts values from 60 to 86400 seconds in steps of 10 seconds. The default is 300 seconds (5 minutes). You can disable the link tolerance by entering a value of zero (0) for this parameter. Inter cluster delay ( Use for Global mirror only) An optional feature for Global Mirror permits a delay simulation to be applied on writes sent to secondary virtual disks. This allows testing to be performed and can be used to
48
test an application before full deployment of the feature. A value of 0 disables this feature. Intra cluster delay ( Use for Global mirror only) An optional feature for Global Mirror permits a delay simulation to be applied on writes sent to secondary virtual disks. This allows testing to be performed and can be used to test an application before full deployment of the feature. A value of 0 disables this feature. We will use the default values in our creation of the partnership and in Figure 4-4 on page 49, we have established a partial partnership between ITSOCL2 and ITSOCL3. Note: When creating a partnership be aware of setting the copy rate settings to a value appropriate to the link and secondary back-end storage.
Figure 4-4 Partially configured partnership
We logon to ITSOCL3 to complete the creation of the partnership. See Figure 4-5 on page 50.
49
Figure 4-5 Completing the creation of the partnership on the remote cluster.
After selecting OK, we can see that the partnership is fully configured as in Figure 4-6.
Figure 4-6 Fully configured cluster partnership.
We close that window and now we have a fully configured partnership between the clusters and all copy commands from now on only need to be entered from either of those clusters, not both. We will continue using ITSOCL2 as our production SVC cluster.
4.2.2 Changing a cluster partnership

When changing a cluster partnership we select our partnership from the main menu as shown in Figure 4-7 on page 51.
50
Figure 4-7 Modifying a cluster partnership
If any changes are made on the partnership link it will not stop any active mirroring. In Figure 4-8 we can change the same settings that we inserted when we created the partnership, as in Figure 1-2 on page 5.
Figure 4-8 Parameter that can be changed for the cluster partnership.
4.2.3 Deleting a cluster partnership

To delete a partnership we will have to run the delete command on both clusters, similar to what we did when we created the partnership. See Figure 4-9 on page 52.
51
Figure 4-9 Deleting a cluster partnership
Figure 4-10 shows the partnership delete.
Figure 4-10 Confirming deletion of an partnership
After we have clicked the delete button the cluster partnership will be partially configured as shown in Figure 4-11. We can only see that state on the remote cluster and on that cluster we complete the deletion of the partnership.
Figure 4-11 After deleting the local partner the deletion is partially done
As we did when we created the partnership we will have to logon to the remote cluster to finish the deletion. And as shown in Figure 4-11 we have a partially configured link that we will permanently delete by clicking delete.
4.2.4 Creating a copy relationship

We have a VDisk on our production SVC cluster and we are going to mirror its contents to the remote SVC cluster. After creating the VDisks on both clusters we will create the relationship 52
between those VDisks as shown in Figure 4-12, then we will import that relationship into the consistency group.
Figure 4-12 Creating copy relationship.
After we have selected the creation of the relationship and pressed Go, we will get an overview of the steps ahead, see Figure 4-13 on page 53.
Figure 4-13 Overview of selections for creating the relationship
We name the relationship with a name that gives a meaning to what it is used for as shown in Figure 4-14.
53
Figure 4-14 Name the relationship
We choose the copy type to be Metro Mirror, since we want the copy to be synchronous. We can also select the relationship to be inter or intra cluster. We select intercluster since we are mirroring from production to disaster/recovery clusters that are separated by distance.
Figure 4-15 Search for Master VDisk candidates
If we know the name of the VDisk we are going to user as our Master VDisk, we can filter for that name, or to see all VDisk candidates on the Master cluster we can use an asterisk to see all the available VDisk candidates as in Figure 4-15 on page 54. After sorting our available candidates, we choose ITSO_MM_S001 as our Master Vdisk as in Figure 4-16 on page 55.
54
Figure 4-16 Selecting our Master VDisk
Now we need to choose our Auxiliary VDisk from the other SVC cluster. The SVC clusters are already in a partnership and therefore it provides the cluster a list of candidates for the auxiliary VDisk, without requiring us to login to the remote cluster. The GUI will only show VDisks that are of the same size as the Master Vdisk and are available for a copy relationship as in Figure 4-17 on page 55.
Figure 4-17 Selecting VDisk as our auxiliary VDisk
We are creating a copy relationship with unused VDisks, we know the status of the disks, and that is both VDisks exist and have not been mapped to any host. From an operating system point of view they are not initialized and empty. Therefore we can chose to select them as synchronized. This is the most common way to perform this action if you are creating a new relationship between unused VDisks. as shown in Figure 4-18 on page 56.
55
Figure 4-18 Select synchronized so the primary and secondary are in sync when the copy process starts.
We will not add this relationship to any consistency group at this time, we will do that later in this chapter. But this feature to be able to add a relationship directly to a consistency group is excellent to use when adding disks to a host that already has VDisks in an consistency group. Our relationship is called ITSO_SERVER_01 and it is now created as shown in Figure 4-19 on page 56.
Figure 4-19 Verifying the relationship.
4.2.5 Changing copy relationship

If we need to make any modifications to our relationship we can do that by selecting the relationship and from the drop down menu select modify relationship as shown in Figure 4-20 on page 57
56
Figure 4-20 Modifying a relationship.
When we are modifying a relationship we can change the name of the relationship, add/remove from a consistency group as shown in Figure 4.2.6.
Figure 4-21 Modifying a relationship.
4.2.6 Starting a copy relationship

After the successful creation of the relationship, the relationship is in a consistent stopped state as shown in Figure 4-22.
Figure 4-22 The relationship is created and ready to be started.
In Figure 4-22, notice that even though we have created the relationship, and that although we have not started any copy process the relationship is in a consistent stopped state. It is consistent because we selected the synchronized parameter when creating the relationship.
57
The final step towards a fully configured relationship is to start the copy process between the VDisks. We select to start the copy process from the drop down menu as shown in Figure 4-23.
Figure 4-23 Start the copy process
In Figure 4-24 on page 58 we do not have to mark forced start or mark as clean since we are starting a new relationship and we have already selected it to be synchronous from the start.
Figure 4-24 Starting the Copy process
And now we see the final state of our relationship in Figure 4-25. Our relationship is in a consistent synchronized state so both primary and secondary VDisks should have the same content.
Figure 4-25 The final state of our relationship creation
58
Now we have created a relationship between two VDisks on two SVC clusters that are in an intercluster partnership. Our Primary VDisk is the Master VDisk and is therefore our production disk.
4.2.7 Stopping a copy relationship.

We can only stop a copy relationship when it is a stand alone relationship. If the relationship is in a consistency group we must stop the consistency group and therefore stop all relationships member in that group. As shown in Figure 4-27 we select our relationship and from the drop down menu we select stop copy process.
Figure 4-26 Stopping copy relationship.
In Figure 4-27 we can select that we want to enable write access to the secondary VDisk. Since we are only stopping the copy process and do not want to enable write access to the secondary, we select OK without setting a check mark to enable write access.
Figure 4-27 Stopping copy process
In Figure 4-28 on page 60 we can see the result of stopping the copy process. Our copy state is now consistent stopped.
59
Figure 4-28 Copy state is now consistent stopped
4.2.8 Deleting a copy relationship

When we need to delete a relationship we select the relationship that we want to delete, and from the drop down menu we select Delete a Relationship as shown in Figure 4-29.
Figure 4-29 Deleting relationship
In Figure 4-30 we can choose to remove the relationship from the consistency group before we delete it. This will still delete the relationship but it removes the relationship from the consistency group before deleting it.
Figure 4-30 Removing from consistency group before it is deleted
60
4.2.9 Creating consistency groups

When we are creating our relationship we have the opportunity to add the relationship to a consistency group at the same time. If we have not already created the consistency group we need to create one and import the relationship to that consistency group. Many administrators will start with making the consistency group, and then add the relationship to it during creation. We will also show you how you create a consistency group and how to perform different tasks with it, such as adding/removing the relationship, and stopping/starting the copy process. In Figure 4-31 we select Manage Copy Services from the main menu, and from the main menu we select Metro & Global Mirror Consistency Groups.
Figure 4-31 Select Create a Consistency Group
After we have selected to create a new consistency group we click Go.. In Figure 4-32 we name our consistency group and select if we want to have it as an intracluster or intercluster consistency group.
Figure 4-32 Naming and configuring Consistency Group
61
We have given our consistency group the name ITSO_SERVER_GRP and it will be an intercluster relationship, since we know the relationship is between two SVC clusters. When we have configured the consistency group we also have the opportunity to add already configured relationships to it. The relationship and the consistency group need to have the same type of configuration and that is intercluster or intracluster. In Figure 4-33 on page 62 we already have a relationship that is not in a consistency group and we select that we want to add it as a member in our ITSO_SERVER_GRP consistency group.
Figure 4-33 Selecting relationship to the consistency group
If we do not select any relationship for our consistency group it will be defined as empty, we can then add relationship(s) to it at our convenience. In Figure 4-34 we see a summary of our consistency group.
Figure 4-34 Summary over the consistency group
After clicking Finish our consistency group is created, but as we can see in Figure 4-35 on page 63, the state of the consistency group is consistent stopped. This is because the relationship we added to the consistency group was in a consistent stopped state. Consistency groups enter the state of the first relationship that is added to the 62
group, and after that you cannot add a relationship of a different state than the consistency group already has.
Figure 4-35 Consistency group created in consistent stopped state
4.2.10 Starting a consistency group

As the nature of consistency groups is to maintain consistency in all member relationships, we are therefore affecting all relationship in the given group we are working on. Figure 4-36 shows where we can select to start the copy process from the drop down menu.
Figure 4-36 Starting a copy process
We start by selecting the consistency group that we want to start the copy process on and the options as in Figure 4-37 on page 64.
63
Figure 4-37 Select options on how we will start the copy process
As before, this copy relationship and the VDisks in that relationship are in sync. Figure 4-38 shows successful start of copy process on the ITSO_SERVER_GRP consistency group.
Figure 4-38 Consistency group is now successfully started
4.2.11 Stopping a consistency group

When we stop a copy process, we are not effecting the primary VDisk in any way. There is a possibility to enable write access to the secondary VDisk when we select the stop copy process (we discuss this in 4.4.2, Site failover using the GUI on page 84). In this scenario we are only stopping the copy process on all relationships in the consistency group leaving the secondary in read only mode. In Figure 4-39 on page 65 from the Metro and Global Mirror consistency groups main menu we select the consistency group that we would like to stop.
64
Figure 4-39 Selecting consistency group to stop copy process on
We select stop the copy process, see Figure 4-40.
Figure 4-40 Stopping the copy process
In this case we do not enable write access to the secondary, but we select Go to stop the copy process.
Figure 4-41 Our consistency group is now consistent stopped
65
4.2.12 Renaming a consistency group

When we need to rename a consistency group it will not effect in any way the operation of the relationships that are within that group. In Figure 4-42 we start by selecting the group we would like to rename and then from the drop down menu we select rename a consistency group.
Figure 4-42 Renaming a consistency group
After we have clicked on Go we enter the new name into the new name field, as shown in Figure 4-43.
Figure 4-43 New name selected for the consistency group
4.2.13 Reversing copy directions

If we have the consistency group in a consistent synchronized state we can switch copy directions. This is a full reverse of copy direction, that is primary will become secondary and vice versa. In Figure 4-44 we choose the Metro and Global mirror consistency groups, to see what consistency groups are available and in what state they are.
66
Figure 4-44 Metro & global mirror consistency groups
We verify that the status is consistent synchronized and we can also see that the Primary is the Master VDisk, so the copy direction is from master to auxiliary. We select the consistency group that we want to change the copy direction on and highlight Switch Copy Direction and click on Go as shown in Figure 4-45 on page 67.
Figure 4-45 Selecting switch copy directions
The next screen (Figure 4-46) shows us that this is the Primary VDisk and asks us if we would like to change that copy direction. If we choose the same current copy direction this command has no effect on the copy state. We select to change the Primary VDisk from Master to auxiliary and therefore select the Primary VDisk to be the auxiliary under new copy direction as shown in Figure 4-46.
67
Figure 4-46 Making Auxiliary the Primary VDisk
We have now changed our Remote copy direction. Primary is now auxiliary as in Figure 4-47 on page 68. This command does not check if the secondary VDisk is mapped to a host or the host is actually sending I/Os to the primary VDisk when we make the change. So it is recommended to have the system administrator verify that all I/O is stopped when we issue the switch copy direction command.
Figure 4-47 Primary VDisk is now auxiliary.
4.3 Implementation using the CLI

When we are implementing Remote Copy using the CLI we do not have to use all the parameters of the command. Many of the parameters are optional and only if it is in our interest will we need to use them. In Chapter 8, Automation on page 207 in this book we discuss some of the commands to use when creating a relationship and importing to a consistency group. More detailed descriptions on each command can be found in IBM Total Storage SAN Volume Controller Command Line interface Users Guide SC26-7903-02. As we did using the GUI our demo environment consists of two SVC clusters that are separated by distance. We have already done the installation of both SVCs, following instructions in IBM System Storage SAN Volume Controller, SG24-6423. We start by checking for available candidates, and we are using SVC clusters ITSOCL2 and ITSOCL3. ITSOCL2 will be our production cluster and ITSOCL3 is our disaster/recovery. 68
4.3.1 Listing cluster candidates

We start by checking for if we have a cluster candidate for our partnership as shown in Example 4-1.
Example 4-1 List the cluster candidate.
IBM_2145:ITSOCL2:admin>svcinfo lsclustercandidate id configured cluster_name 0000020068203A42 no ITSOCL3 We can see that this SVC cluster is not configured so our SVC cluster candidate is not already configured as our partner. Only one cluster partnership is available for each cluster.
4.3.2 Create cluster partnership

In Example 4-2 we make the partnership.
Example 4-2 The mkpartnership command.
>>- svctask -- -- mkpartnership -- -----------------------------> >--+-----------------------------------+-- ---------------------> '- -bandwidth -- bandwidth_in_mbps -' >--+- remote_cluster_id ---+----------------------------------->< '- remote_cluster_name -' This creates a one-way partnership between our production cluster and the disaster/recovery cluster. To complete the two-way partnership, the equivalent mkpartnership command must be issued on the disaster/recovery cluster. We create the partnership using the default bandwidth , that is 50 MBps, and our remote cluster is called ITSOCL3. Example 4-3 shows the command used on the production SVC cluster.
Example 4-3 Creation of partnership with ITSOCL3.
IBM_2145:ITSOCL2:admin>svctask mkpartnership ITSOCL3 Now we have partially created the partnership. We can see that this is partially created by issuing the command lscluster -delim : ITSOCL3 . See Example 4-4.
Example 4-4 Partially configured partnership.
IBM_2145:ITSOCL2:admin>svcinfo lscluster -delim : ITSOCL3 id:0000020068403A42 name:ITSOCL3 location:remote partnership:partially_configured_local bandwidth:50 We logon to ITSOCL3 ( the remote cluster) and complete the creation. We issue the same command as before svctask mkpartnership ITSOCL2 and then the svcinfo lscluster -delim : ITSOCL2 to see the status of the partnership as in Example 4-5.
69
Example 4-5 Creation of partnership with ITSOCL2
IBM_2145:ITSOCL3:admin>svctask mkpartnership ITSOCL2 IBM_2145:ITSOCL3:admin>svcinfo lscluster -delim : ITSOCL2 id:000002006B603A38 name:ITSOCL2 location:remote partnership:fully_configured bandwidth:50 Now we have created our fully configured partnership between ITSOCL2 and ITSOCL3 with a bandwidth of 50 MBps.
4.3.3 Change cluster partnership

In Example 4-6 we change the cluster partnership.
Example 4-6 The chpartnership command.
>>- svctask -- -- chpartnership -- -----------------------------> >-- -bandwidth -- bandwidth_in_mbps -- -------------------------> >--+- remote_cluster_id ---+----------------------------------->< '- remote_cluster_name -' We can change the parameters of the cluster link without stopping the active copy process. We might want to increase the bandwidth that background copy is allowed to use from the default value of 50 Mbps to 75 MBps. This is shown in Example 4-7.
Example 4-7 Changing bandwidth between clusters from 50 to 75 MB/s.
IBM_2145:ITSOCL2:admin>svctask chpartnership -bandwidth 75 ITSOCL3 IBM_2145:ITSOCL2:admin>svcinfo lscluster ITSOCL3 id 0000020068403A42 name ITSOCL3 location remote partnership fully_configured bandwidth 75
4.3.4 Deleting a cluster partnership

In Example 4-8 we show how to delete the cluster partnership.
Example 4-8 The rmpartnership command.
>>- svctask -- -- rmpartnership -- -----------------------------> >--+- remote_cluster_id ---+----------------------------------->< '- remote_cluster_name -' When deleting a copy partnership we need to delete the partnership from both the local and the remote clusters. From the cluster we are logged on we issue the rmparntership
70
command, we use the remote clusters name in the command as shown in Example 4-9 on page 71. After issuing the rmpartnership command we logon to the remote cluster to complete the deletion of the partnership
Example 4-9 Removing partnership.
IBM_2145:ITSOCL2:admin>svctask rmpartnership ITSOCL3 IBM_2145:ITSOCL3:admin>svctask rmpartnership ITSOCL2
4.3.5 Creating a copy relationship

After we have created the intercluster partnership between our clusters, we need to start a copy relationship between two VDisks. We have already created VDisks on both clusters of the same size, which is required for a copy relationship to be created. The disks we have created are as follows: On our production cluster ITSOCL2 ITSO_MM_S01 ITSO_MM_S02 On our Disaster/Recovery cluster ITSOCL3 ITSO_MM_T01 ITSO_MM_T02 In Example 4-10 we create the relationship.
Example 4-10 The mkrcrelationship command.
>>- svctask -- -- mkrcrelationship -- --------------------------> >-- -master --+- master_vdisk_id ---+-- ------------------------> '- master_vdisk_name -' >-- -aux --+- aux_vdisk_id ---+-- ------------------------------> '- aux_vdisk_name -' >-- -cluster --+- cluster_id ---+-- ----------------------------> '- cluster_name -' >--+------------------------+-- --------------------------------> '- -name -- new_name_id -' >--+-----------------------------------------+-- ---------------> '- -consistgrp --+- consist_group_id ---+-' '- consist_group_name -' >--+---------+-- --+-----------+------------------------------->< '- -sync -' '- -global -'
71
Note: If there is a consistency group already created we can add a relationship to a consistency group in the same step, using the -consistgrp parameter.
When using the mkrelationship command we need to have names and/or ids of the VDisks and clusters we are using. For more detailed information about certain aspects of the command it can be found in the CLI command guide1. Our command will look like that shown in Example 4-11.
Example 4-11 Creating the relationship.
IBM_2145:ITSOCL3:admin>svctask mkrcrelationship -master ITSO_MM_S01 -aux ITSO_MM_T01 -cluster ITSOCL3 -name ITSO_SERVER_01 -sync RC Relationship, id [0], successfully created Now we have created a fully configured relationship between ITSO_MM_S01 and ITSO_MM_T01, but the copy process is not started. The VDisks are in sync and are only waiting for the copy process to be started, see Example 4-12.
Example 4-12 State of the relationship.
IBM_2145:ITSOCL3:admin>svcinfo lsrcrelationship 0 id 0 name ITSO_SERVER_02 master_cluster_id 000002006B603A38 master_cluster_name ITSOCL2 master_vdisk_id 1 master_vdisk_name ITSO_MM_S02 aux_cluster_id 0000020068403A42 aux_cluster_name ITSOCL3 aux_vdisk_id 0 aux_vdisk_name ITSO_MM_T02 primary master consistency_group_id consistency_group_name state consistent_stopped bg_copy_priority 50 progress freeze_time status online sync in_sync copy_type metro
4.3.6 Starting a copy relationship

In Example 4-13 we start the copy relationship.
Example 4-13 The startrcrelationship command.
>>- svctask -- -- startrcrelationship -- -----------------------> >--+--------------------------+-- --+----------+-- ------------->

1
IBM Total Storage SAN Volume Controller Command Line Interface Users Guide, SC26-7903-02
72
'- -primary --+- master -+-' '- aux ----'
'- -force -'
>--+----------+-- --+- rc_rel_id ---+-------------------------->< '- -clean -' '- rc_rel_name -' To start the copy process for our relationship, containing the VDisks ITSO_MM_S01 and ITSO_MM_T01, we will use the startrcrelationship command. If we have already put the relationship into a consistency group we would have to start the consistency group with all the relationships in that group by using the startrcconsistgrp command. We have created the relationship and now we want to start copying, but we need to apply the -force parameter, because this parameter is required if consistency would be lost by starting a copy operation. Since we do not know if an I/O has occurred to the primary VDisk after we created the relationship we need to issue the command with the -force parameter. In general, the -force parameter is required if the relationship is in one of the following states: ConsistentStopped but not synchronized Idling but not synchronized The -force parameter is not required if the relationship is in one of the following states: InconsistentStopped InconsistentCopying ConsistentSynchronized Example 4-14 shows where our relationship ITSO_SERVER_01 is started.
Example 4-14 Starting the copy process within the relationship.
IBM_2145:ITSOCL2:admin>svctask startrcrelationship -force
ITSO_SERVER_01
After we have started the relationship it will become consistent synchronized as shown in Example 4-15.
Example 4-15 State of the relation ship is now consistent synchronized.
IBM_2145:ITSOCL2:admin>svcinfo lsrcrelationship ITSO_SERVER_01 id 2 name ITSO_SERVER_01 master_cluster_id 000002006B603A38 master_cluster_name ITSOCL2 master_vdisk_id 2 master_vdisk_name ITSO_MM_S001 aux_cluster_id 0000020068403A42 aux_cluster_name ITSOCL3 aux_vdisk_id 4 aux_vdisk_name ITSO_MM_T001 primary master consistency_group_id consistency_group_name state consistent_synchronized bg_copy_priority 50 progress freeze_time status online sync
73
copy_type metro
4.3.7 Stopping a copy relationship

In Example 4-16 we show how to stop a copy relationship.
Example 4-16 The stoprcrelationship command.
>>- svctask -- -- stoprcrelationship -- --+-----------+-- ------> '- -access -' >--+- rc_rel_id ---+------------------------------------------->< '- rc_rel_name -' The stoprcrelationship command applies only to a stand-alone relationship. If the relationship is part of a consistency group, the group needs to be stopped and therefore all relationships within that group too. We can issue this command to stop a relationship that is copying from primary to secondary VDisks. If the relationship is in a consistent state, that is; consistent stopped, consistent synchronized, or consistent disconnected state, we can use the -access parameter to enable write access to the secondary VDisk.
4.3.8 Change a copy relationship

In Example 4-17 on page 74 we show how to change a copy relationship.
Example 4-17 The chrcrelationship command
>>- svctask -- -- chrcrelationship -- --------------------------> >--+- -name -- new_name_arg -----------------+------------------> +- -force --------------------------------+ '- -consistgrp --+- consist_group_id ---+-' '- consist_group_name -' >--+- rc_rel_id ---+------------------------------------------->< '- rc_rel_name -' When we need to change relationship we use the command svctask chrcrelationship We can change name and add/remove relationships from the consistency group using this command. We can remove a relationship from a consistency group by specifying the -force parameter and the name or ID of the relationship. We start by adding a relationship to an existing consistency group with the name of ITSO_SERVER_GRP as in Example 4-18.
Example 4-18 adding a stand alone relationship to consistency group.
IBM_2145:ITSOCL2:admin>svctask chrcrelationship -consistgrp ITSO_SERVER_GRP ITSO_SERVER_01 Then we remove the relationship from the consistency group again as in Example 4-19.
74
Example 4-19 Removing a relationship from a consistency group.
IBM_2145:ITSOCL2:admin>svctask chrcrelationship -force ITSO_SERVER_01 This form of the modify relationship command succeeds in the connected or disconnected states. If the clusters are disconnected the relationship is only removed from the consistency group on the local cluster at the time the command is issued. When the clusters are reconnected the relationship is automatically removed from the consistency group on the other cluster. Alternatively, you can issue an explicit modify (chrcrelationship) command to remove the relationship from the group on the other cluster while it is still disconnected.
4.3.9 Delete copy relationship

In Example 4-20 we show how to delete the copy relationship.
Example 4-20 The rmrcrelationship command.
>>- svctask -- -- rmrcrelationship -- --+- rc_rel_id ---+------>< '- rc_rel_name -' When we delete a relationship we are only deleting the relationship between those two VDisks, it does not affect the VDisks themselves. It is not possible to delete a relationship if it is a member of a consistency group. We would have to remove it from the consistency group first and then delete it as shown in Example 4-21.
Example 4-21 Deleting a copy relationship
IBM_2145:ITSOCL2:admin>svctask chrcrelationship -force ITSO_SERVER_01 IBM_2145:ITSOCL2:admin>svctask rmrcrelationship ITSO_SERVER_01
4.3.10 Creating a consistency group

In Example 4-22 we show how to create a consistency group.
Example 4-22 The mkrcconsistgrp command
>>- svctask -- -- mkrcconsistgrp -- --+---------------------+---> '- -name -- new_name -' >-- --+--------------------------------+----------------------->< '- -cluster --+- cluster_id ---+-' '- cluster_name -'
Note: If we make a Metro Mirror or Global Mirror consistency group not using the -cluster (name or ID of the remote cluster) the consistency group will only be created on the local cluster. We will create a consistency group between ITSOCL2 and ITSOCL3 and implement the copy relationship (that we have already created) to that consistency group. We created the consistency group as shown in Example 4-23.
75
Example 4-23 Creating the consistency group
IBM_2145:ITSOCL2:admin>svctask mkrcconsistgrp -cluster ITSOCL3 -name ITSO_SERVER_GRP RC Consistency Group, id [255], successfully created This consistency group is now created but it has no relationship in it, and we can see that by looking closer at the relationship in Example 4-24.
Example 4-24 Showing the content of the consistency group
IBM_2145:ITSOCL2:admin>svcinfo lsrcconsistgrp ITSO_SERVER_GRP id 255 name ITSO_SERVER_GRP master_cluster_id 000002006B603A38 master_cluster_name ITSOCL2 aux_cluster_id 0000020068403A42 aux_cluster_name ITSOCL3 primary state empty relationship_count 0 freeze_time status sync copy_type empty_group To move a relationship between two consistency groups, you must issue the chrcrelationship command twice. Use the -force parameter to remove the relationship from its current group, and then use the -consistgrp parameter with the name of the new consistency group. Since our relationship was created as a standalone relationship it is not a member of a consistency group, so we only need to use the -consistgrp parameter as in Example 4-25.
Example 4-25 Adding relationship to consistency group.
IBM_2145:ITSOCL2:admin>svctask chrcrelationship -consistgrp ITSO_SERVER_GRP ITSO_SERVER_01 The content of our consistency group has changed as shown in Example 4-26.
Example 4-26 The consistency group now has a relationships member.
IBM_2145:ITSOCL2:admin>svcinfo lsrcconsistgrp ITSO_SERVER_GRP id 255 name ITSO_SERVER_GRP master_cluster_id 000002006B603A38 master_cluster_name ITSOCL2 aux_cluster_id 0000020068403A42 aux_cluster_name ITSOCL3 primary master state consistent_stopped relationship_count 1 freeze_time status online sync in_sync copy_type metro 76
RC_rel_id 0 RC_rel_name ITSO_SERVER_01 Note: If we try to add a relationship to a consistency group which has a different copy state, such as the relationship is stopped and the consistency group is synchronized, we will get an error message saying: The relationship cannot be added because the states of the relationship and the consistency group do not match.
4.3.11 Start consistency group

In Example 4-27 we show how to start the consistency group.
Example 4-27 The startrcconsistgrp command.
>>- svctask -- -- startrcconsistgrp -- -------------------------> >--+--------------------------+-- --+----------+-- -------------> '- -primary --+- master -+-' '- -force -' '- aux ----' >--+----------+-- --+- rc_consist_group_id ---+---------------->< '- -clean -' '- rc_consist_group_name -'
When we start or stop a consistency group we are effecting all relationships in that group, hence the term consistency. It is possible to make changes to the consistency group within the start and stop commands, and this will be explained later. By issuing the startrcconsisgrp command we are given the following possibilities as parameters: -primary Which VDisk will become the primary VDisk in this relationship? Is it the master or the auxiliary that we want to be the active disk. -force Should we force the copy process to start? Do we know the state of the secondary VDisk, and can we start the copy process even if we know that we will loose consistency while the secondary is catching up with primary? -clean Is it a clean start? Clean in this sense means that any changes that have been made at the secondary are ignored, and only changes made at the primary are considered when synchronizing the primary and secondary VDisks. In our example we are starting the copy process by issuing the startrcconsistgrp command and stating that the primary will be the master VDisk. Example 4-28.
Example 4-28 Starting the copy process in the consistency group with master as primary disk.
IBM_2145:ITSOCL2:admin>svctask startrcconsistgrp -primary master ITSO_SERVER_GRP IBM_2145:ITSOCL2:admin>svcinfo lsrcconsistgrp ITSO_SERVER_GRP id 255 name ITSO_SERVER_GRP master_cluster_id 000002006B603A38
77
master_cluster_name ITSOCL2 aux_cluster_id 0000020068403A42 aux_cluster_name ITSOCL3 primary master state consistent_synchronized relationship_count 1 freeze_time status online sync copy_type metro RC_rel_id 0 RC_rel_name ITSO_SERVER_01 Now we have started the ITSO_SERVER_01 copy relationship in the ITSO_SERVER_GRP consistency group and the VDisks are in the state consistent synchronized.
4.3.12 Stopping the consistency group

In Example 4-29 we show how to stop the relationship.
Example 4-29 The stoprcrelationship command.
>>- svctask -- -- stoprcrelationship -- --+-----------+-- ------> '- -access -' >--+- rc_rel_id ---+------------------------------------------->< '- rc_rel_name -' By issuing the stoprcconsistgrp command with the -access parameter, we will stop the copy process and enable write access to the secondary, This command will not change the copy directions, but the state of the consistency group will be idling, and that means that both VDisks have read/write access enabled, and no copy process is active. If we only issue the command stoprcconsistgrp the consistency group will stop mirroring between VDisks, but the secondary will still be in a read-only state. Example 4-30 is an example of how we used the stoprcconsistgrp with the -access command to stop the copy process and enable write access to the secondary VDisk. IBM_2145:ITSOCL2:admin>svctask stoprcconsistgrp -access ITSO_SERVER_GRP IBM_2145:ITSOCL2:admin>svcinfo lsrcconsistgrp ITSO_SERVER_GRP id 255 name ITSO_SERVER_GRP master_cluster_id 000002006B603A38 master_cluster_name ITSOCL2 aux_cluster_id 0000020068403A42 aux_cluster_name ITSOCL3 primary state idling relationship_count 1 freeze_time status sync in_sync copy_type metro RC_rel_id 0 78
RC_rel_name ITSO_SERVER_01 To change the copy direction follow the process in 4.3.15, Reversing copy directions on page 81.
4.3.13 Changing a consistency group

Example 4-30 shows how we change the consistency group.
Example 4-30 The chrcconsistgrp command
>>- svctask -- -- chrcconsistgrp -- -- -name -- new_name_arg ---> >-- --+- rc_consist_group_name -+------------------------------>< '- rc_consist_group_id ---' When changing the consistency group, all we are changing is the name. A consistency group is only a name until it has got a relationships member. In Example 4-31 we are changing the consistency group name from ITSO_SERVER_GRP to ITSO_SILENT.
Example 4-31 Changing the name of the consistency group
IBM_2145:ITSOCL2:admin>svctask chrcconsistgrp -name ITSO_SILENT ITSO_SERVER_GRP Now our consistency group has the name ITSO_SILENT.
4.3.14 Deleting a consistency group

In Example 4-32 we show how to delete the consistency group with active relationships in it, and how the relationships are not effected by this deletion.
Example 4-32 The rmrcconsistgrp command.
>>- svctask -- -- rmrcconsistgrp -- --+----------+-- -----------> '- -force -' >--+- rc_consist_group_id ---+--------------------------------->< '- rc_consist_group_name -' If we are deleting a consistency group that is not empty we can use the -force parameter to delete the consistency group. This will change all relationships in that consistency group to a standalone relationship. The state of the relationship is not changed by deleting the consistency group. In the examples that follow we delete a consistency group with two consistent synchronized relationships in it. First we check the status of the consistency group called ITSO_SILENT, and in this group we have two relationships called ITSO_SERVER_01 and ITSO_SERVER_02. As shown in Example 4-33 the copy status is consistent synchronized.
Example 4-33 Checking status of the relationships within the consistency group.
IBM_2145:ITSOCL2:admin>svcinfo lsrcconsistgrp ITSO_SILENT id 254

79
name ITSO_SILENT master_cluster_id 000002006B603A38 master_cluster_name ITSOCL2 aux_cluster_id 0000020068403A42 aux_cluster_name ITSOCL3 primary master state consistent_synchronized relationship_count 2 freeze_time status online sync copy_type metro RC_rel_id 2 RC_rel_name ITSO_SERVER_01 RC_rel_id 9 RC_rel_name ITSO_SERVER_02 Now we delete the consistency group as in Example 4-34.
Example 4-34 Deleting the consistency group.
IBM_2145:ITSOCL2:admin>svctask rmrcconsistgrp -force ITSO_SILENT And by checking the status of the relationships we now see that they have not changed state, both are consistent synchronized. Example 4-35 and Example 4-36.
Example 4-35 Status of ITSO_SERVER_01 copy relationship.
IBM_2145:ITSOCL2:admin>svcinfo lsrcrelationship ITSO_SERVER_01 id 2 name ITSO_SERVER_01 master_cluster_id 000002006B603A38 master_cluster_name ITSOCL2 master_vdisk_id 2 master_vdisk_name ITSO_MM_S001 aux_cluster_id 0000020068403A42 aux_cluster_name ITSOCL3 aux_vdisk_id 4 aux_vdisk_name ITSO_MM_T001 primary master consistency_group_id consistency_group_name state consistent_synchronized bg_copy_priority 50 progress freeze_time status online sync copy_type metro
Example 4-36 Status of ITSO_SERVER_02 copy relationship.
IBM_2145:ITSOCL2:admin>svcinfo lsrcrelationship ITSO_SERVER_02 id 9 name ITSO_SERVER_02 master_cluster_id 000002006B603A38 80
master_cluster_name ITSOCL2 master_vdisk_id 9 master_vdisk_name ITSO_GLOBAL aux_cluster_id 0000020068403A42 aux_cluster_name ITSOCL3 aux_vdisk_id 5 aux_vdisk_name ITSO_GLOBAL primary master consistency_group_id consistency_group_name state consistent_synchronized bg_copy_priority 50 progress freeze_time status online sync copy_type metro
4.3.15 Reversing copy directions

When a consistency group is in a consistent synchronized state, we can reverse the copy direction for that group by using the switchrcconsistgrp command by using the -primary parameter.See Example 4-37.
Example 4-37 The state of the consistency group before switching copy directions
IBM_2145:ITSOCL2:admin>svcinfo lsrcconsistgrp ITSO_SERVER_GRP id 255 name ITSO_SERVER_GRP master_cluster_id 000002006B603A38 master_cluster_name ITSOCL2 aux_cluster_id 0000020068403A42 aux_cluster_name ITSOCL3 primary master state consistent_synchronized relationship_count 1 freeze_time status online sync copy_type metro RC_rel_id 0 RC_rel_name ITSO_SERVER_01 And now we issue the switch command (Example 4-38).
Example 4-38 Issuing the switch copy directions command
IBM_2145:ITSOCL2:admin>svctask switchrcconsistgrp -primary aux ITSO_SERVER_GRP And then check the status after issuing the command, see Example 4-39.
Example 4-39 The state of the consistency group after the switch command.
IBM_2145:ITSOCL2:admin>svcinfo lsrcconsistgrp ITSO_SERVER_GRP id 255
81
name ITSO_SERVER_GRP master_cluster_id 000002006B603A38 master_cluster_name ITSOCL2 aux_cluster_id 0000020068403A42 aux_cluster_name ITSOCL3 primary aux state consistent_synchronized relationship_count 1 freeze_time status online sync copy_type metro RC_rel_id 0 RC_rel_name ITSO_SERVER_01 Now we have switched copy directions and the primary disk is now the auxiliary Vdisk.
4.4 Example of site failover

When a disaster occurs it is often the case that we are not very well prepared for it. It is always good practice to have a detailed plan of what needs to be done, how, and by whom. In a storage environment we might need to take decisions on whether and when to declare a disaster or not, and when/if we do so it is important to have a detailed action plan, Often it is the same people that are trying to fix the failure and getting the customer up and running again. The detailed action plan should be written in such a way that a storage administrator could carry out those tasks without having the person who wrote the plan explaining each line. When controlling the copy functionality, either with Metro Mirror or Global Mirror, it can be controlled from either cluster in the copy relationship, and that means if one SVC cluster is taken down, then the other SVC cluster can stop the mirroring process and can enable read/write access to the secondary VDisks. Note: Using Metro Mirror or Global Mirror the secondary VDisks are always in a read only mode and should not be mapped to a server until the copy direction is changed or the copy process is stopped and the VDisk is set to read/write mode. If we need to perform a site failover there are some things that will help us to minimize downtime of servers caused by the disaster. Make sure that all VDisks that are in a copy relationship in consistency groups. Make sure that you have created a script that will help you perform a site failover. Chapter 8, Automation on page 207 shows script examples. All servers on the recovery site should be zoned and created, but not mapped to the target SVC cluster VDisks. If disaster is declared and you are going to perform a site failover stop all copy processes if not already stopped. This is because the secondary VDisk cannot by set to read/write without stopping the copy process.
82
4.4.1 Site failover when production site loses back-end storage

If we take an example where two SVC clusters are doing Metro Mirror copy. In Figure 4-48 we have server A and server B. Both servers are connected to the SAN and defined on both SVC clusters, but only primary server A has allocated VDisks from SVC cluster 1. SVC cluster 2 is the remote SVC and therefore we create the disks and Metro Mirror relationship, but we do not allocate the VDisks to server B. The target VDisks are left un allocated.
Figure 4-48 Overview of environment
The server loses access to its LUNs and the SVC cannot see its MDisks from that subsystem as shown in Figure 4-49.
83
Figure 4-49 Subsystems on production site fails
If the disks on Server 1 become unavailable, we can perform a failover to the standby server at the disaster/recovery site and start using the secondary VDisks. Here are some guidelines which we can follow: 1. Terminate all connections from Server 1 to SVC cluster 1 VDisks If we can access SVC cluster 1 it is best to terminate all connections between server 1 and the SVC cluster. When the repair is complete we do not have to worry about conflicts with the old disks. 2. Stop all copy process that involve this server a. Log on to the remote SVC cluster and stop all copy services. i. By doing that we are able to grant write access to the secondary VDisk. b. If we lose connection to our MDisks the copy services automatically stops but that is because the SVC sees this as a hardware failure. i. You could also reverse the mirror and by that give read/write access to the target LUNs, but if the source Managed Disk Group (MDG) is offline, or has a hardware failure, you cannot change the copy direction. 3. Map VDisks from SVC cluster 2 to server B 4. Mount VDisks and restart the application. Note: The automation using CLI is mention in chapter 9 in this Redbook.
4.4.2 Site failover using the GUI

This is our scenario: we have a dual site environment, production and disaster/recovery, the production site consists of server, subsystem and SVC. The disaster/recovery site has the identical hardware, and all disks for the production server are Metro Mirrored across to the disaster/recovery site. The production site we call site A, and the disaster/recovery site we 84
call site B. In site B we have standby server B and a Metro Mirror partner to our production SVC. In Figure 4-50 we can see a Metro Mirror consistency group called ITSO_SERVER_GRP, that is running on our production SVC. In this consistency group we have two active relationships and the state is consistent synchronized.
Figure 4-50 A consistency group in healthy state.
In our scenario the subsystem on site A went offline so all our MDisks were unavailable at that site. Our copy process will then automatically stop as shown in Figure 4-51 on page 85.
Figure 4-51 Consistency groups in status consistent stopped.
At this point the server cannot see its disks at the production site and we want to perform a failover to the disaster/recovery site. We will now log on to the SVC cluster located at the disaster/recovery site and make the secondary VDisks available for read/write access. Even though the status of the relationship is consistent stopped we need to stop the copy process to be able to enable read/write access to the VDisks in the consistency group. The state of the consistency group prevents us from using the Switch Copy Directions command. In Figure 4-52 we have clicked the Metro & Global Mirror consistency groups from the main menu and there we can see the current consistency groups this cluster partnership has. From the drop down menu we can select to stop the copy process on the selected consistency group.
85
Figure 4-52 Selecting to stop copy process.
When we have selected to stop the copy process and clicked Go, we will get the chance to change the mode of the secondary VDisks to enable write access, as shown in Figure 4-53 on page 86.
Figure 4-53 Stopping the copy process and enabling write access to secondary.
By setting a checkmark in the enable write access checkbox, and selecting OK, we are enabling write access to all VDisks in this consistency group. All relationships in the consistency group will become idle and therefore the consistency group itself. Figure 4-54 shows the status of the consistency group after we have enabled write access.
86
Figure 4-54 The consistency group is now in Idling state.
Now we can map the secondary VDisks to our disaster/backup server. After mapping the disk a server administrator needs to check if his SDD driver can see the disks and then restart the application. After fixing the problem on the production site all errors regarding the copy relationship need to be cleared before you can start the copy process again. When we have enabled write access to the secondary VDisk, and all errors are fixed, we can copy from the disaster/recovery site to the production site, and have the auxiliary as our primary disk. When the production environment is stabilized there is a need for a service window for the server to switch the copy directions and run production on the production site.
4.4.3 Site failover using the CLI

When we are performing a site failover using the CLI we can implement those commands in a script, see the examples in Chapter 8, Automation on page 207 to make it easier. We will show the same steps as we did in 4.4.2, Site failover using the GUI on page 84 but using the CLI. The CLI sometimes provides more detailed information about what is going on in our environment. Our production cluster is called ITSOCL3 and our consistency group is called senegal. By checking the status of the consistency group we can see that the state is consistent synchronized and the status is online, we can also see that our primary is our master VDisks. See Example 4-40 for status.
Example 4-40 Status of consistency group called Senegal.
IBM_2145:ITSOCL3:admin>svcinfo lsrcconsistgrp 255 id 255 name senegal master_cluster_id 0000020068203A42 master_cluster_name ITSOCL3 aux_cluster_id 000002006B403A38 aux_cluster_name ITSOCL2 primary master state consistent_synchronized relationship_count 2
87
freeze_time status online sync copy_type metro RC_rel_id 0 RC_rel_name senegal_r01 RC_rel_id 1 RC_rel_name senegal_r02 On the server everything is normal but now something happens to our subsystem and we can see that state and status has changed. The CLI even shows us which VDisk (primary or the secondary) has a problem. And as soon as something happens to either disk the copy process stops. See Example 4-41.
Example 4-41 Copy stops
IBM_2145:ITSOCL3:admin>svcinfo lsrcconsistgrp 255 id 255 name senegal master_cluster_id 0000020068203A42 master_cluster_name ITSOCL3 aux_cluster_id 000002006B403A38 aux_cluster_name ITSOCL2 primary master state consistent_stopped relationship_count 2 freeze_time 2007/10/23/14/03/02 status primary_offline sync in_sync copy_type metro RC_rel_id 0 RC_rel_name senegal_r01 RC_rel_id 1 RC_rel_name senegal_r02 When this happens we need to enable write access to the secondary VDisk in this relationship, so we logon to the disaster/recovery cluster, in our case ITSOCL2. The main command to use for that is stoprcconsistgrp with the -access parameter. Note: stoprcconsistgrp with -access option is the command to be used to enable write access to the secondary VDisks in the group, if the group is in a consistent state. When a consistency group is in a consistent state (for example, in the consistent stopped, consistent synchronized, or consistent disconnected state) you can issue the -access parameter with the stoprcconsistgrp command to enable write access to the secondary virtual disks within that group. See Table 4-2 for examples when to use the -access option.
Table 4-2 Show us the criteria needed to use the -access option. Initial state InconsistentStopped InconsistentCopying ConsistentStopped Final state InconsistentStopped InconsistentStopped ConsistentStopped Notes -access permitted
88
Initial state ConsistentSynchronized Idling IdlingDisconnected
Final state ConsistentStopped ConsistentStopped Unchanged
Notes -access permitted -access permitted A relationship may move to stopped state when reconnected. On the cluster issuing the svctask stoprcconsistgrp command On the disconnected cluster On the cluster issuing the svctask stoprcconsistgrp command, -access permitted On the disconnected cluster, -access permitted.
InconsistentDisconnected
InconsistentStopped
InconsistentDisconnected ConsistentDisconnected
Unchanged ConsistentStopped
ConsistentDisconnected
Unchanged
in our case we use svctask stoprcconsistgrp -access senegal and the outcome is shown in Example 4-42. Notice that it is in idling state.
Example 4-42 Idling state
IBM_2145:ITSOCL2:admin>svcinfo lsrcconsistgrp 255 id 255 name senegal master_cluster_id 0000020068203A42 master_cluster_name ITSOCL3 aux_cluster_id 000002006B403A38 aux_cluster_name ITSOCL2 primary state idling relationship_count 2 freeze_time status sync in_sync copy_type metro RC_rel_id 4 RC_rel_name senegal_r01 RC_rel_id 22 RC_rel_name senegal_r02
Now we have enabled write access to the secondary VDisk so we can map the VDisk to the disaster/recovery server with the svctask mkvdiskhostmap and the server administrator can check if he can se the disks via the SDD driver.
4.5 Copy commands using the CLI

When we are using the CLI there are several commands you can use and we will list them here.
89
All CLI commands can be found, along with a detailed description in the IBM System Storage SAN Volume Controller Command-Line Interface Users Guide, SC26-7903-02. chpartnership chrcconsistgrp chrcrelationship mkpartnership mkrcconsistgrp mkrcrelationship rmpartnership rmrcconsistgrp rmrcrelationship startrcconsistgrp startrcrelationship stoprcconsistgrp stoprcrelationship switchrcconsistgrp switchrcrelationship
90
7574 Discussing FlashCopy.fm
Chapter 5.
FlashCopy
In this chapter we discuss SVC FlashCopy, which is the Point-in-Time copy capability of the SVC. FlashCopy is used to create instant copies of VDisks. We cover the characteristics of FlashCopy, and how to implement it to solve different problems to ensure we meet our Business Continuity objectives.
91
5.1 Introduction to FlashCopy

FlashCopy creates a rapid, complete, and consistent copy from a source VDisk to a target VDisk. Often this functionality is called Time-Zero copy, Point-In-Time copy or Snapshot copy. Its main characteristic is to present a clone of a VDisk for a specific point in time, very fast. That is, it is being presented as complete before the copy is really created.
5.1.1 What is FlashCopy and what is it useful for?

Without a function such as FlashCopy, for a self-consistent copy of data, the I/O of the application which manipulates the data has to be quiesced for the whole time the physical copy process takes place. In such a case the time that the copy process needs is defined by the amount of data to be copied, and the capabilities of the infrastructure to copy the data. Only after the copy process is finished can the application which manipulates the data start to access the VDisk that was involved. Only then is it ensured that the data on the copy target is self-consistent and identical to the data on the source for a given point in time. With FlashCopy this is different. FlashCopy enables the creation of a copy of a source VDisk to a target VDisk in a very short time. This way, the application only has to be prevented from changing the data for a short period of time. After the FlashCopy has been started, the target VDisk represents the contents of the source VDisk for the point-in-time when the FlashCopy was invoked. The target VDisk does not yet contain all the data of the source VDisk physically. It can be seen as a virtual copy, created using bitmaps. After the FlashCopy has been has been started but has not yet finished physically copying the data to the target, the copy can be accessed in read/write mode. From now on, data which gets changed on the source VDisk (by the applications now again manipulating the source VDisk) will be written to the target VDisk, thus ensuring that the representation of the data on the target VDisk for that point-in-time is still valid. It is also possible to copy all the data of the source VDisk to the target VDisk through a background copy process. While not copied fully, the target VDisk represents the clone of the source VDisk as long as the relationship between the source and the target exists. Once all data has been copied to the target VDisk, the relationship between the VDisks can be removed and the both VDisks become normal VDisks again. The former target VDisk is now a physical clone of the source VDisk for the point-in-time that the FlashCopy was invoked. To create consistent copies of data which spans multiple VDisks, consistency groups can be used. Consistency groups are sets of FlashCopy Mappings, which get copied at the same point-in-time, thus creating a consistent snapshot of the data across all VDisks involved. FlashCopy is very flexible. It is possible to have a VDisk be a source VDisk in one FlashCopy Mapping, and to be the target VDisk in another FlashCopy Mapping. Also, one VDisk can have the role as source VDisk in multiple FlashCopy Mappings with different target VDisks. Another feature is that it is possible to incrementally update a fully copied target VDisk with only the changes that have been made to the source VDisk of the same mapping.
5.1.2 Usage cases of FlashCopy

It is possible to create multiple target VDisks from one source VDisk. While it is the same source VDisk and different target VDisks, for every source-target VDisk combination, a dedicated mapping for each combination is required. 92
FlashCopy can have many uses. One obvious use is the backup of a consistent set of data without the need for a long backup window. The application manipulating the data only needs to make sure the data is consistent; and only needs to be suspended for a short period of time. Once the copy is started the backup application can access the target while the applications can resume manipulating the live data. No full volume copy is needed. One very useful case for FlashCopy in our Business Continuity strategy is to create a full consistent copy of production data for a given point-in-time at a remote location. Here, we combine Remote Copy and FlashCopy, where we take a FlashCopy from the Remote Copy secondary VDisks. The usage case could be either to take a consistent backup of our production data on the second location or to have a clone of the data for a given point-in-time available in case our production data gets invalidated due to a logical error. FlashCopy can also be used as a safety net for operations which could make copies of data inconsistent for a longer than normal period of time. For example, if Global Mirror gets out-of-synchronization for some reason, the auxiliary VDisk is still consistent in itself, but the process of resynchronizing renders the auxiliary VDisk inconsistent as long as it is not finished. To retain a consistent copy of the data of the auxiliary VDisk while it is being synchronized and thus is inconsistent, a FlashCopy of this VDisk can be created. Another use is the creation of clones of data for application development testing, or application integration testing. Here, full copies of data would be useful. One other use would be if a set of data has to be used for different purposes, for example a FlashCopy database could be used for data mining.
5.2 FlashCopy functions and features

There are characteristics and features of the FlashCopy function of the SVC which are important to understand to be able to successfully use it. Some features available increase the flexibility of FlashCopy. These are FlashCopy Consistency Groups, Multiple Target FlashCopy (MTFC), Cascaded FlashCopy (CFC) and Incremental FlashCopy (IFC).
5.2.1 FlashCopy Mappings

A FlashCopy Mapping is created when two ordinary VDisks get mapped together for the purpose of creating a point-in-time copy, shown in Figure 5-1 on page 94.
Chapter 5. FlashCopy
93
Source Vdisk m1
Mapping m1
Target Vdisk m1
time T0
Figure 5-1 Single VDisk mapping
Once associated with the mapping, the VDisks get a new responsibility, one VDisk becomes a source VDisk and one VDisk becomes a target VDisk. This mapping of VDisks gets a FlashCopy Mapping Name, which can be seen as an alias that we use to manage the source and target VDisk pairs for FlashCopy tasks, instead of using the names of the VDisks. The source VDisk provides the data to be copied to the target VDisk. The target VDisk is used to present the data of the source VDisk for the point-in-time when the FlashCopy was started, as long as the mapping exists and not all data of the source VDisk got copied to the target VDisk. Additionally, the entire source VDisk can get copied to the target VDisk using a background copy process. Once this is finished the mapping is no longer needed for the target VDisk to present the data. For a single VDisk there is a limit of 16 mappings to be shared between the use of this VDisk in mappings for MTFC and CFC. Any combination of mappings representing MTFC and CFC in a cascading tree may not exceed 16.
5.2.2 FlashCopy Consistency Groups

A consistency group consists of a set of FlashCopy Mappings and has a FlashCopy Consistency Group Name. This name can be seen as an alias as well. As such, FlashCopy actions can now be directed at the consistency group which enables the execution of commands on multiple mappings while ensuring consistency across data which spans multiple source VDisk for all mappings the consistency group contains. A Consistency Group consisting of 2 mappings is illustrated in Figure 5-2 on page 95.
94
Source Vdisk m1
Source Vdisk m2
Mapping m1
Mapping m2
Consistency Group cg1
Target Vdisk m1
Target Vdisk m2
Start Consistency Group cg1
time T0
Figure 5-2 FlashCopy Consistency Group
Data which has to be consistent can span multiple VDisks. For instance, database management system (DBMS) logs reside on a different VDisk than the database itself. The problem to take care of is called dependent writes. These occur, for instance, when database logs are written before (to indicate a database update is about to take place) and after (to log a successful write operation to the database) the database itself gets updated. If we were to start a FlashCopy of the involved VDisks without ensuring consistency across VDisks, it could happen that the FlashCopy target contains both written logs but not the database update. The logs in the FlashCopy target would indicate that the write I/O has occurred, but the actual write I/O would be missing from the target VDisk which holds the database, thus rendering the FlashCopy target data inconsistent from a DBMS point of view. Mappings which are not put into a consistency group are reported to be members of the pseudo consistency group 0.
5.2.3 FlashCopy background copy

FlashCopy background copy is a feature which enables you to copy all the data of a source VDisk to the corresponding target VDisk. Without it, only data which changed on the source VDisk would be copied to the target VDisk. The result of a FlashCopy Mapping with a
95
background copy enabled would be that the target VDisk would become a real (independent from the source VDisk) clone of the FlashCopy Mapping source VDisk. The background copy rate is a property of a FlashCopy Mapping which is expressed as a value between 0 and 100 percent. It can be changed in any FlashCopy Mapping state and can be different in the Mappings of one Consistency Group. With a value of 0, background copy is disabled. Table 5-1 shows the relationship of the background copy rate value to the attempted number of grains to be split per second. Grains are discussed in 5.3.2, What is a grain? on page 101.
Table 5-1 Background copy rate/copy speed relationship Background copy rate value in % 1-10 11-20 21-30 31-40 41-50 51-60 61-70 71-80 81-90 91-100 Data copied/sec 128KB 256KB 512KB 1MB 2MB 4MB 8MB 16MB 32MB 64MB 256KB Grains split/sec 0.5 1 2 4 8 16 32 64 128 256 64KB Grains split/sec 2 4 8 16 32 64 128 256 512 1024
The SVC will try to achieve these copy rates. The background copy rate and the foreground copies share the same infrastructure. If the bandwidth needed (to copy data to the underlying managed disks provided by back end storage subsystems) is used up, a latency increase and a reduction of throughput on both the foreground copies and the background copy can occur. The degradation will be graceful. The background copy is performed by both nodes of the I/O group providing the source VDisk.
5.2.4 FlashCopy autodelete option

There is a new option which enables the automatic deletion of a FlashCopy Mapping once the background copy has finished copying all data to the target VDisk. This is especially useful for use in scripts.
5.2.5 Multiple Target FlashCopy

Multiple Target FlashCopy (MTFC) is when there is a VDisk which serves as the source VDisk in multiple mappings, where in each mapping the target is a different VDisk, shown in Figure 5-3 on page 97.
96
Source Vdisk m1 - m16
Mapping 1
Mapping 2
Mapping 16
Target Vdisk m1
Target Vdisk m2
...
Target Vdisk m16
Order of FlashCopy Start (time)
Figure 5-3 Multiple Target FlashCopy
For each source VDisk, up to 16 different mappings are possible. These mappings are independently controllable from each other. For instance, the background copy priority can be different for these mappings. The implementation of MTFC introduces some dependencies, discussed in 5.5, Dependencies between FlashCopy Mappings on page 110. MTFC Mappings can be members of the same or different consistency groups. In a case where all the mappings are in the same consistency group, the result of starting the consistency group will be multiple identical target VDisks.
5.2.6 Cascaded FlashCopy

A new feature of FlashCopy is Cascaded FlashCopy (CFC). With CFC a VDisk is a source for one FlashCopy mapping, and the target for another FlashCopy Mapping, as shown in Figure 5-4 on page 98 and Figure 5-4 on page 98. These FlashCopy Mappings are generally independent of each other, nonetheless, dependencies of how to perform actions on them are in place, as discussed in 5.5, Dependencies between FlashCopy Mappings on page 110. For each cascade, 16 mappings are possible.
97
Mapping m1
Mapping m2
Mappings m3-16
Source Vdisk m1
Vdisk t m1 s m2
Vdisk t m2 s m3-16
...
Vdisk t m3-16
Figure 5-4 Cascaded FlashCopy
The use of consistency groups is restricted when using CFC. A consistency group serves the purpose of invoking FlashCopy Mappings at the same point-in-time.
Within the same consistency group it is not possible to have mappings where:
The source VDisk of one mapping is the target of another mapping. The target VDisk of one mapping is the source VDisk for another mapping. These combinations would not be useful because within a consistency group there is no way to have the mappings established in a certain order, which renders the content of the target VDisks undefined (for instance, was the first mapping established before the target VDisk of the first mapping that acts as a source VDisk for the second mapping?). Even if there was a way to ensure an order in which the mappings are established within a consistency group, the result would be equal to MTFC (two VDisks which hold the same target data for one source VDisk). In other words, a cascade is useful when we want to copy VDisks in a certain order (and copying the changed content targets of FlashCopies) rather than at the same time in an undefined order (from within one single consistency group).
5.2.7 Combined Multiple Target FlashCopy and Cascaded FlashCopy

MTFC and CFC can be combined in any way, as long as combinations of both do not exceed 16 mappings in one tree. A simple illustration of combining MTFC and CFC is shown in Figure 5-5.
98
Source Vdisk m1 m2
Mapping m1
Mapping m2 Mapping m3
Target Vdisk m1
Target Vdisk m2 / Source Vdisk m3
Target Vdisk m3
Figure 5-5 A Tree of combined MTFC and CFC
As we can see in this picture, what we call a target VDisk or a source VDisk is in reality a VDisk which can be in the role of being the source, the target or a source and a target, if it is a member of one or more FlashCopy Mappings.
5.2.8 Incremental FlashCopy

Without Incremental FlashCopy (IFC), once all data in a FlashCopy Mapping has been copied to a target VDisk, and the FlashCopy Mapping is restarted, all data has to be copied again to create an independent copy. By creating a FlashCopy Mapping with the incremental flag, only the data which has been changed since the last FlashCopy was started gets written to the target VDisk, illustrated in Figure 5-6 on page 100.
99
Data change Source Vdisk m1 Source Vdisk m1
Start Mapping m1: copy all data Restart Mapping m1: copy changed data only
Target Vdisk m1
Target Vdisk m1
T0 (1)
time
T0 (2)
Figure 5-6 Incremental FlashCopy
The amount of data to be copied depends on the grain size as well: if the grain size is 64KB (as compared to 256KB), there might be less data to copy in order to get a full independent copy of the source again. A difference value is provided in the query of a mapping, which makes it possible to know how much data has changed, which would need to be copied once the IFC is restarted. The difference value is the percentage (0% - 100%) of how many grains have been changed and would need to be copied to the target VDisk in order to get a fully independent copy of the source VDisk again. This functionality is needed in cases where we want to have, for example, a full copy of a VDisk for disaster tolerance, and/or application testing and/or data mining. It greatly reduces the time to establish a full copy of the source data again as a new snapshot, once the first background copy completed. In cases where customers maintain full independent copies of data as part of their disaster tolerance strategy, using IFC can be of benefit for their Business Continuity goals.
5.3 FlashCopy internals

Understanding how FlashCopy works internally helps us to configure it the way we need it, and to get the most benefit from it.
100
5.3.1 What is the indirection layer and what is it for?

For FlashCopy to function, the I/O path of the SVC is being altered, with the introduction of a facility called the indirection layer. It is an additional layer of functionality in the I/O path which causes the I/O to make a detour to enable the FlashCopy function. The indirection layer becomes active in the I/O path at the start of a FlashCopy Mapping and intercepts I/Os directed to both the source VDisk and the target VDisk. It decides for each I/O what actions are to be performed. It either: Allows the I/O to go directly to the VDisk Redirects the I/O from the target VDisk to another VDisk Stalls the I/O until it arranges the copying of data from the source VDisk to the target VDisk The indirection layer is placed below the cache, and this means any latency introduced by its function does not affect the interaction between the layer above it (the storage client and the SVC cache). If latency occurs because of I/Os which are stalled, this only affects the I/O generated by destaging of the cache. The decision on how to handle I/Os is based upon three attributes: The destination of the I/O (VDisk and its Logical Block Address (LBA)) Its direction (either it is a write or a read operation) The state of the FlashCopy bitmap
How read operations are handled

Read operations targeted to the source VDisk are always passed directly to the source VDisk. Read operations from the target VDisk use the information stored in the FlashCopy bitmap. In a case where the data to be read has not yet been copied to the target VDisk, it will be read from the source VDisk, or from another target VDisk in the case of MTFC. In a case where it has been copied, the data is read from the target VDisk.
How write operations are handled

Write operations for which the grain containing the data to be written to has been split already will go to the VDisk. Write operations to source VDisks and target VDisks which target an area which has not yet been copied will result in the write operation being stalled. Then a copy operation is started to copy the data from the source VDisk to the target VDisk, and after this completes, the stalled write operation resumes.
5.3.2 What is a grain?

Data gets copied between VDisks in a FlashCopy Mapping in units of the FlashCopy Mapping address space called grain (the granularity of the data to be copied from the source VDisk to the target VDisk is one grain). The grain size can be 256KB (default) or 64KB. The grain size can affect the amount of data needed to be copied for Incremental FlashCopy, as with IFC, the smaller grain size of 64KB can lead to less data to be copied. If a new FlashCopy Mapping gets added to a tree, it will use the grain size which is already in use for the mappings in the tree. It is also possible to join two existing trees of FlashCopy Mappings. To be able to create the mapping used to join the trees, both trees must use the same grain size.
101
5.3.3 What is a bitmap?

The bitmaps are an internal data structure, used to track which grains in FlashCopy mappings have been copied from the source VDisk to the target VDisk (which grains have been split). One bit in each bitmap represents the state of one grain and either it is split or it is not. The FlashCopy bitmap uses up bitmap space in the memory of the SVC. The maximum size of the bitmap space is 512MB per I/O Group, which has to be shared between FlashCopy bitmaps and Remote Copy bitmaps. How we assign the bitmap space to the copy functions is a variable. The default is 20MB for FlashCopy and 20MB for Remote Copy (Metro/Global Mirror). The maximum shared bitmap space is 512MB per I/O group, that is to say 256MB for FlashCopy and 256 for Remote Copy, or 512 for FC and 0 RC or any similar combination. This feature allows us to trade off memory between the cache, FlashCopy and Remote Copy (Metro Mirror/Global Mirror).
5.3.4 Usable bitmap space

Table 5-2 on page 102 shows the bitmap space / FlashCopy address space relationship depending on the size of the grain and the kind of copy service being used.
Table 5-2 Bitmap space / FlashCopy address space relationship for the specified I/O group Copy Service Grain size in KB: 256 1Byte of bitmap space gives a total of: 2MB of VDisk capacity 2MB of target VDisk capacity 512KB of target VDisk capacity 1MB of target VDisk capacity 256KB of target VDisk capacity 4KB of bitmap space gives a total of: 8GB of VDisk capacity 8GB of target VDisk capacity 2GB of target VDisk capacity 4GB of target VDisk capacity 1GB of target VDisk capacity 1MB of bitmap space gives a total of: 2TB of VDisk capacity 2TB of target VDisk capacity 512GB of target VDisk capacity 1TB of target VDisk capacity 256GB of target VDisk capacity 20MB of bitmap space gives a total of: 40TB of VDisk capacity 40TB of target VDisk capacity 10TB of target VDisk capacity 20TB of target VDisk capacity 5TB of target VDisk capacity 512MB of bitmap pace gives a total of: 1024TB of VDisk capacity 1024TB of target VDisk capacity 256TB of target VDisk capacity 512TB of target VDisk capacity 128TB of target VDisk capacity
Metro Mirror/Globa l Mirror FlashCopy
256
FlashCopy
64
Incremental FlashCopy Incremental FlashCopy
256
64
102
Note: For the FlashCopy Mapping address space, the amount of mappings has to be taken into account. Each mapping target for a FlashCopy uses up bitmap space. This means for MTFC, that even if there is only one source VDisk involved, multiple targets will consume the same amount of bitmap space as the same number of targets of independent FlashCopy Mappings would use up.
5.3.5 Number of FlashCopy Mappings in a cluster

The new maximum number of FlashCopy Mappings is 3855 for the whole cluster. Each single mapping (which might be part of a Multiple Target FlashCopy or Cascaded FlashCopy) contributes to the maximum number of mappings. Before Multiple Target FlashCopy and Cascaded FlashCopy, when a VDisk could only take part in one 1-on-1 relationship, the maximum number of mappings was the maximum number of VDisks per cluster (4096) divided by two (2048). The limit of 4096 VDisks per cluster still exists and it is still responsible for the maximum number of FlashCopy mappings. This time though, the maximum number depends on how we use FlashCopy, and to what extent we use MTFC and CFC. With the introduction of Multiple Target FlashCopy the maximum limit for mappings went up even though the maximum number of VDisks per cluster stayed the same. This is because with MTFC, one VDisk (which obviously only takes one spot in the maximum VDisk limit of 4096) can be the source VDisk in up to 16 mappings, and with CFC a VDisk can be a member of two mappings as well as increasing the possible number of mappings in a cluster. Although it is good to have the increased flexibility of MTFC and CFC and the possible higher number of mappings, using this flexibility comes at an expense. That is the more VDisks we use up as target VDisks for MTFC, or the more we use CFC, the number of VDisks in a cluster left to be used as source VDisks for MTFC, or a cascade (as a root VDisk), declines accordingly. For instance, in the extremely unlikely case (and this would be the worst case), that we try to use every VDisk to be copied as a source VDisk in as many mappings as possible (mapped with 16 different target VDisks), we could only have 240 VDisks to be source VDisks mapped to 16 target VDisks and 1 remaining VDisk mapped to 15 target VDisks. The calculation here is as follows: 17 VDisks (1 source VDisk and 16 target VDisks) multiplied by 240 equals 4080 VDisks. The maximum number of VDisks supported by an SVC cluster is 4096, which leaves 16 VDisks to be used in mappings, which could be 1 VDisk as the source VDisk and 15 acting as target VDisks. So, as we can see, the practical number of VDisks to act as source (root) VDisks in FlashCopy Mappings is somewhere between 241 (worst case, maximum usage of MTFC) and 2048 (no usage of MTFC and CFC). Note: These numbers apply to a cluster consisting of 4 I/O Groups supporting 4096 VDisks. A cluster consisting of 2 nodes in one I/O Group supports 1024 VDisks. This cluster would only support between 60 root source VDisks, each in 16 mappings and 1 source VDisk in 3 mappings (worst case) and 512 source VDisks (no use of MTFC and CFC).
103
5.3.6 How can we specify and view the bitmap space on an I/O group
The user can increase and decrease the amount of memory used on nodes for keeping FlashCopy and Remote Copy bitmaps at runtime. For example, for FlashCopy we specify the amount of bitmap space of 20MB for io_grp0, shown in Example 5-1 on page 104 and list it.
Example 5-1 Specifying and viewing the bitmap space located on an I/O group
admin>svctask chiogrp -feature flash -size 20 io_grp0 admin> admin>svcinfo lsiogrp 0 id 0 name io_grp1_r node_count 2 vdisk_count 382 host_count 4 flash_copy_total_memory 20.0MB flash_copy_free_memory 20.0MB remote_copy_total_memory 20.0MB remote_copy_free_memory 19.9MB
5.3.7 Where is the bitmap stored?

The bitmap for FlashCopy Mappings is stored in a particular I/O group. When we create a new FlashCopy Mapping we can force which I/O group we want the bitmap to reside in. Using the CLI this can be achieved with the -iogrp parameter. This feature can be useful if changes are planned, for example creating, joining or re-organizing trees. This ability can be useful if the user knows that trees of mappings may be joined in the future, or for reconstructing a tree of mappings in a different order to their original creation order during a configuration restore. If we specify the I/O group of the target vdisk when creating the mapping, the bitmap space will go towards the I/O group of the target VDisk. If we do not specify the I/O group for the bitmap space, the bitmap will be placed on the I/O group which owns the source VDisk. If we add a mapping to a Cascaded FlashCopy or a Multiple Target FlashCopy or a cascaded tree (a combination of both), discussed in 5.3.8, FlashCopy characteristics listed on page 105, the bitmap will be stored on the I/O group where the bitmap for the other mappings is stored already. Note: This feature can lead to FlashCopy Mapping behavior which should be taken into consideration. If, for example, we assume the bitmap of a tree is stored in I/O group 0, and some VDisks which are members of mappings in that tree are owned by I/O group 1, when I/O group 0, where the bitmap for the tree is stored is taken offline, the FlashCopy Mappings for the VDisks owned by I/O group 1 (which is still online, and whose VDisks are still online) will stop.
104
5.3.8 FlashCopy characteristics listed

Following is a list of FlashCopy characteristics: FlashCopy is a copy process involving two VDisks associated with a FlashCopy Mapping. Once used in a FlashCopy Mapping, a VDisk becomes a source or target VDisk. A FlashCopy copies a whole source VDisk to a target VDisks within one cluster. The VDisks chosen as the source and target VDisks can be provided by different I/O groups. A VDisk can be the source VDisk in multiple FlashCopy Mappings. A VDisk can be the target VDisk in one FlashCopy Mapping. A VDisk can be the source in one mapping and the target in another at the same time. The VDisks used for one FlashCopy Mapping must be of the same size. The size of VDisks which are members in a FlashCopy Mapping can not be changed. The content of the source VDisk does not get changed by the FlashCopy process. The original content of the target VDisk gets destroyed by the FlashCopy process. After the FlashCopy is established, the target VDisk represents a clone of the source VDisk. After the FlashCopy is established, the target VDisk can be accessed as read/write. Until all the data is copied, the target VDisk presents the data as long as the mapping exists. Once all the data got copied, the mapping can be deleted and the VDisks are independent again. Once all the data got copied, the former target VDisk holds the data even without the mapping.
5.3.9 FlashCopy maximum configurations

To plan for and to implement FlashCopy, some limits have to be checked and adhered to. The limits for an SVC cluster are shown in Table 5-3 on page 105.
Table 5-3 Configuration limits for FlashCopy Property FlashCopy targets per source Maximum 16 Notes The maximum number of FC Mappings that can exist with the same source VDisk The maximum number of FC Mappings is calculated by noticing that 240 source VDisks with 16 FC Mappings plus one more with 15 mappings uses up all 4096 VDisks per cluster An arbitrary limit policed by the software This is a limit on the quantity of FC Mappings using bitmap space from this I/O Group. This maximum configuration will consume all 512MB of bitmap space for the I/O Group and allow no Metro and Global Mirror bitmap space. The default is 40TB
FlashCopy Mappings per cluster
3855
FlashCopy consistency groups per cluster FlashCopy VDisk space per I/O group
128 1024TB
105
Property FlashCopy Mappings per consistency group
Maximum 512
Notes Due to the time taken to prepare a consistency group with a large number of mappings.
5.4 FlashCopy events and states

The state of a FlashCopy Mapping is managed on a per FlashCopy Mapping basis (commands which prepare and start a FlashCopy Mapping are targeted to a consistency group).
5.4.1 FlashCopy Mapping states

A FlashCopy Mapping can have states, and it is put in these states by FlashCopy Mapping events.
Copying
The FlashCopy has been started. From now on, the target VDisk presents the content of the source VDisk for the point in time when the FlashCopy was started (but is not holding all the data yet).
Idle/Copied
VDisks are members of a mapping but behave independently. The target represents a full copy of the source for the point in time when the FlashCopy was started. On both VDisks, write- and read caching is enabled.
Stopping
The mapping is in the process of transferring data to a mapping which depends on it.
Stopped
The data on the target VDisk is lost and the target VDisk is put offline. To regain access to the target VDisk, the mapping must be either deleted or it must be started again.
Suspended
The FlashCopy Mapping was in the Copying or the Stopping state and the target has been flashed from the source. The access to the metadata has been lost resulting in both the source VDisk and the target VDisk going offline.
Preparing
The FlashCopy Mapping gets prepared for a FlashCopy start while I/O still continues to the source VDisk. This includes flushing the modified write data of the source VDisk from the cache, placing the cache for the source VDisk into write through mode, and discarding any data associated with the target VDisk from the cache.
Prepared
The FlashCopy Mapping is ready to be started.
106
Table 5-4 on page 107 summarizes the FlashCopy Mapping states and the corresponding cache states for the VDisks in a FlashCopy Mapping.
Table 5-4 FlashCopy Mapping states Source VDisk State Idling/Copied Copying Stopped Stopping Mapping State online online online online Cache State write-back write-back write-back write-back Target VDisk Mapping State online online offline Online if copy complete, offline if not offline online, but not accessible online, but not accessible Cache State write-back write-back n/a n/a
Suspended Preparing Prepared
offline online online
write-back write-through write-through
n/a n/a n/a
5.4.2 FlashCopy events

In Figure 5-7 on page 109 we show the events alter the state a FlashCopy Mapping. We describe them here.
Create
Places the FlashCopy Mapping into the Idle/Copied state. It creates a new FlashCopy Mapping between two VDisks, assigning roles to them, one now being a source VDisk and one being the target VDisk for the new mapping.
Prepare
Places the FlashCopy Mapping in the Preparing state. The prepare command is directed to a consistency group (single mappings are members of consistency group 0, in this case the command is targeted at the mapping name for the given FlashCopy Mapping). Note: Even though the FlashCopy Mapping has not been started yet, the target VDisk has to be considered corrupt from now on, since beginning with the act of preparing, cached writes are discarded.
Flush Done
Places the FlashCopy Mapping in the Prepared state. This is done once all cached data from the source VDisk has been flushed and all cached data of the target VDisk has been invalidated.
Start
Places the FlashCopy Mapping in the Copying state. The FlashCopy Mapping is being started. The start of the FlashCopy Mapping is synchronized (ensuring consistency across all involved mappings in the consistency group).
107
Stop
Places the FlashCopy Mapping in either the Stopped or Stopping state. A FlashCopy Mapping can be stopped by a user command or by an I/O error.
Stop Complete
Places the FlashCopy Mapping in the Stopped state. It occurs as a transition from Stopping to Stopped as a result of a complete copy operation to break dependencies between FlashCopy Mappings.
Flush Failed
Places the FlashCopy Mapping in the Stopped state. This happens when the flush of data from the cache fails.
Copy Complete
Places the FlashCopy Mapping in the Idle/Copied state. This happens once every grain has been split. With the autodelete feature being used, the FlashCopy Mapping will be deleted once it completes the copy. Without it, the FlashCopy Mapping will remain and can be prepared and started again. The copy complete event will not occur while there are still mappings which depend on it in the copying state.
Bitmap Offline/Bitmap Online

If all nodes in the I/O group that is storing the bitmap for a FlashCopy mapping go offline, then the bitmap will go offline. When the last node to go offline comes back online the bitmap will come back online. Bitmap Offline places the FlashCopy Mapping in the Suspended state. Bitmap Online places the FlashCopy Mapping in the Copying or Stopping state.
Modify
This does not place the FlashCopy Mapping in another state. Many properties can now be modified - name, consistency group, background copy rate, cleaning rate,and the autodelete attribute.
Delete
The Delete event deletes the FlashCopy Mapping. Figure 5-7 on page 109 is the FlashCopy Mapping state diagram. It illustrates which state a mapping can be in, and what events are responsible for a state change.
108
Stop Completes (target is not complete) Stop Completes (target is complete)

S TOPPED
Stop Completes (target is not complete)
S TOPPING
Bitmap Online
Stop
S USPENDED
Bitmap Offline
Delete (Force) Create Delete, Autodelete

P REPARING P REPARED C OPYING
Prepare
Flush Failed
Stop Stop
Bitmap Online
Bitmap Offline
IDLE_OR_C OPIED
Prepare
Flush Done
Start
Copy Complete
Legend
Stop Completes (target is complete)
F LASHC OPY M APPING S TATE
FlashCopy Event
Figure 5-7 FlashCopy Mapping State diagram
5.4.3 FlashCopy consistency group states

For reference, the FlashCopy consistency group state diagram is shown in Figure 5-8 on page 110. This diagram explains the state transitions for consistency groups, and the possible states of the mappings within the consistency groups.
109
S TOPPED S TOPPED IDLE_OR_ C OPIED
S TOPPING S TOPPING S TOPPED IDLE _OR_C OPIED
S USPENDED C OPYING S USPENDED S TOPPING S TOPPED IDLE_C OPIED IDLE_OR_C OPIED IDLE_OR_C OPIED P REPARING S TOPPED P REPARED P REPARING IDLE_OR_C OPIE D P REPARED P REPARED C OPYING C OPYING S TOPPING S TOPPED IDLE _OR_C OPIED
Legend
C ONSISTENCY_G ROUP S TATE V ALID M APPING S TATE
Figure 5-8 FlashCopy Consistency Group state diagram
5.5 Dependencies between FlashCopy Mappings

FlashCopy Mappings, which are members in a tree, can depend on each other. A dependency is when one or more mappings are responsible to provide grains for another mapping. In such a case, actions like the deletion of one mapping can affect other mappings which depend on the mapping being deleted. A dependency between mappings only exists as long as a source VDisk of a mapping does not physically holds all grains. This can happen only when there are preceding mappings in a tree which have not yet completed the background copy process to establish an independent clone of the source VDisk).
5.5.1 Linked lists of mappings and resulting dependencies

Multiple Target FlashCopy and Cascaded FlashCopy (as well as single FlashCopy) are implemented using the same method, with structures called linked lists. This means that the restrictions dictated by this implementation apply to both, MTFC and CFC. Hence, the linked list implementation of FlashCopy is the cause for dependencies between mappings when using MTFC as well as using CFC.
110
What is a linked list

Linked lists are dynamic internal data structures which hold informations about active FlashCopy Mappings in a MTFC setup, a FlashCopy cascade or a combination of both, a tree. A mapping gets inserted into a linked list of mappings when it is being started, not at the time of the creation of the mapping. A cycle of mappings is possible to be set up, but it is not allowed to have them all active at the same time, no cycle is allowed in a linked list.
Dependencies and effect on mappings

Dependencies in a dependency chain exist as long as there are unsplit grains in a single mapping. When a target VDisk in a mapping has received all grains from the source VDisk, the target VDisk is no longer dependent on the source VDisk. Similarly, when in a linked list of mappings all grains from a target VDisk (acting as a source) have been copied to a dependent target VDisk, the dependency ceases to exist. Dependencies between mappings have effects such as the dependent mappings will stop if a mapping they depend on stops (either because it is force-stopped or has an error). Also, the target VDisks of the dependent mappings go offline if the target VDisk of the mapping they depend on goes offline. This happens if the source VDisk goes offline whilst the background copy is incomplete.
Dependencies with Cascaded FlashCopy

In the case of CFC, the dependency exists because the source VDisk of a mapping can be the target VDisk of another mapping which does not hold all the data physically, because the background copy process of the preceding mapping is not complete. Thus grains could still reside on the source VDisk of the preceding mapping in the cascade. A cascaded mapping cannot be prepared in case active mappings exist further down the cascade. The command to display these dependent mappings is shown in 5.5.3, Command Line commands for information on dependencies on page 116. For Cascaded Mappings, if there are mappings in the linked list below the current mapping that are active (not stopped or idle-copied) then this mapping can not be started. In order to start this mapping, the downstream mappings must be stopped.
Dependencies with Multiple Target FlashCopy

With MTFC there is no obvious preceding mapping, since all target VDisks share the same source VDisk, which holds all data physically. Nonetheless, because of the implementation of FlashCopy using linked lists, an order is being established, which makes mappings precede other mappings. To avoid too much load on the source VDisk in case of many mappings with the same source, MTFC is implemented in a way that mappings try to get grains from preceding mappings in the list, which are newer members of the list (mappings started later).
Position of mappings and resulting dependencies

The first mapping in a linked list just maps one source VDisk to a target VDisk (first member, oldest mapping). New mappings are inserted into the linked List (when the mapping is being started) using the following rules: A new Cascaded Mapping is inserted behind the mapping whose target VDisk is being used as the source VDisk. A new Multiple Target Mapping is inserted in front of the next oldest mapping of the same source VDisk. If multiple mappings are started at the same time, because they are in the same Consistency Group, an unspecified ordering will be chosen.
111
The mapping being in the first position, at the head of the inked list (not necessarily the one being inserted first) does not depend on other mappings. Every other succeeding mapping depends on the preceding mappings in the list. The following illustrations show how mappings get inserted into a linked list, depending upon the being set up as MTFC or CFC or both. A simple MTFC is shown in Figure 5-9 on page 112.
Vdisk 1 s m1, s m2
Mapping m1 started at t1
Vdisk 2 t m1
Vdisk 3 t m2
time
Figure 5-9 Multiple Target FlashCopy - m2 started after m1
Both mappings, sharing the same source VDisk result in an MTFC which results in the linked list consisting of two mappings, shown in Figure 5-10. These lists start with the leftmost mapping, which is called the head of the list. The arrow originates from the mapping which is dependent on the mapping to which the arrow points to.
m2 started at t2
m1 started at t1
Figure 5-10 Linked list of MTFC Mappings m1 and m2
The mapping m2, which was started after mapping m1, precedes mapping m1 in the list. Mapping m1 now depends on mapping m2. This means, for any grains not located in mapping m1, the target VDisk of mapping m2 gets asked. Figure 5-11 on page 113 shows which VDisks provide grains for VDisk 2. 112
Vdisk 1 provides grains to Vdisk 2 in case Vdisk 3 could not provide the grain
Vdisk 1 provides grains to Vdisk 3
Vdisk 1 s m1, s m2
Vdisk 3 t m2
Vdisk 2 t m1
Figure 5-11 Vdisks providing grains
A simple CFC is shown in Figure 5-12.
Vdisk 1 s m1
Vdisk 2 t m1 s m2
Vdisk 3 t m2
time
Figure 5-12 Cascaded FlashCopy - m2 started after m1
Figure 5-13 on page 113 shows the resulting linked list.
m1 started at t1
m2 started at t2
Figure 5-13 Linked list of CFC Mappings m1 and m2
113
Figure 5-14 shows which VDisks provide grains.
Vdisk 1 provides grains to Vdisk 3 in case Vdisk 2 could not provide the grain
Vdisk 1 s m1
Vdisk 2 t m1 s m2
Vdisk 3 t m2
Figure 5-14 Vdisks providing grains
The rules about inserting mappings to the list also apply when combining MTFC and CFC. An example of such a tree is shown in Figure 5-15.
Vdisk 1 s m1, s m2
Vdisk 2 t m1
Vdisk 3 t m2 s m3
Vdisk 4 t m3 s m4
Vdisk 5 t m4
time
Figure 5-15 Combined MTFC and CFC
The resulting linked list is shown in Figure 5-16 on page 115.
114
m2 started at t2
m3 started at t3
m4 started at t4
m1 started at t1
Figure 5-16 Linked List of combined MTFC and CFC
Writes to target VDisks in a linked list

For MTFC, writes to a target VDisk as well cause writes to target VDisks of the next dependent mapping. When a target VDisk gets written to by an application, its content changes compared to the content of the source VDisk for the specified point-in-time. This change, however, is not supposed to happen on other mappings. To retain the data of the point-in-time for other dependent mappings, the target VDisk of the preceding mapping copies the grains to be changed to the target VDisk of the dependent mapping. For CFC, a write to a target VDisk causes a write to the target VDisk of the next mapping in the cascade. This write to an unsplit grain on this target VDisk merges the data from the target VDisk of the previous mapping in the cascade with the storage client write data and writes the result to the next in the cascade.
5.5.2 Stopping state and cleaning

One case where dependencies are to be coped with is when a mapping is to be stopped and deleted. This would have an impact on dependent mappings for which a VDisk in the deleted mapping held grains which were still not split. This, of course, needs to be prevented from happening. To avoid impacting mappings which depend on the mapping to be stopped, the Stopping state is used. During the Stopping state, a copy process finds and copies all grains, which are unique on the mapping to be stopped, to another mapping in the tree. After this has been done, the mapping can enter the Stopped state and can be deleted. The target VDisk of a FlashCopy Mapping in the Stopping state is offline if the target is not an independent copy, and is therefore not accessible to storage clients for an what may be an unacceptably long time. If it is an independent copy then it is online whilst the mapping is stopping. To tackle this, a new method of copying grains is being implemented, and this is the Cleaning mode. In this mode, grains are copied from the mapping to be stopped while it is still in the Copying state (and while its target VDisk is still accessible to the storage client) to an older mapping. This is implemented using a new Cleaning Rate parameter. If this parameter is combined with a zero copy background rate, it causes the mapping to copy the grains to a downstream mapping. Information about the progress is contained in a new Cleaning Progress field introduced into the query output of the mapping. After the cleaning is complete (all unique grains are copied) the mapping can be stopped, and it will be in a stopped state immediately. If the mapping is stopped before the cleaning process completed, the mapping enters the Stopping state first, and the remaining grains will get copied. After this completes, the mapping will enter the Stopped state.
115
5.5.3 Command Line commands for information on dependencies

Using the command svcinfo lsvdiskdependentmaps we can display all mappings with target VDisks that are dependent upon data held on the specified VDisk. Cascaded Mappings can not be prepared (and then started) in case active mappings exist further down the cascade, which would be displayed using this command. The syntax is shown in Example 5-2 on page 116.
Example 5-2 svcinfo lsvdiskdependentmaps
>>- svcinfo -- -- lsvdiskdependentmaps --+- vdisk_id ---+------>< '- vdisk_name -' We can as well specify the mapping using the command is svcinfo lsfcmapdependentmaps, which displays all the FlashCopy Mappings that are dependent on the user specified mapping. The syntax is shown in Example 5-3.
Example 5-3 svcinfo lsfcmapdependentmaps
>>- svcinfo -- -- lsfcmapdependentmaps -- --+----------+-- -----> '- -nohdr -' >--+-----------------------+-- --+- fc_id ---+----------------->< '- -delim -- delimiter -' '- fc_name -'
116
7574 Implementing FlashCopy.fm
Chapter 6.
Implementing FlashCopy
In this chapter we show how to use FlashCopy in three different cases using the SVC GUI, and in one case using the CLI. First we show a basic FlashCopy, secondly a Multiple Target FlashCopy and the third a Cascaded FlashCopy which uses the Incremental FlashCopy feature as well. The example using the CLI is FlashCopy using Metro Mirror secondary VDisks as the FlashCopy source VDisks.
117
6.1 Example 1: basic FlashCopy using the GUI

The first example we show is a basic FlashCopy consisting of one mapping where we do not need all data of the VDisk to be copied physically. Goal: The goal is to backup the production data using a very small backup window, which is one of the most basic problems. As the data to backup gets larger, the time required to backup the data increases. In cases where a full consistent copy is needed, the application changing this data would have to be suspended from changing the data for the whole time the backup is being created. Solution: Using FlashCopy to create a snapshot of the data. This is achieved through the use of FlashCopy to create a virtual copy of the data for the given point in time. Since it only needs a short time to complete its copy, the application has to be prevented from changing the data only for a short time. Of course, the data should be consistent, all involved caches need to be flushed and written to the VDisk. The target copy can instantly be accessed by the backup application while the source is ready to be accessed and used (and to be changed) again.
6.1.1 Creating a FlashCopy mapping

We can start to create a single FlashCopy as shown in Figure 6-1 on page 118, on the Viewing FlashCopy Mappings screen. Here, existing mappings are displayed (none for now). The only action possible right now is to create a new mapping.
Figure 6-1 Viewing FlashCopy mappings - no mapping exists
The first screen of the Create a Mapping wizard is shown in Figure 6-2 on page 119. It gives an overview over the steps we go through to create the mapping.
118
Figure 6-2 Create a FlashCopy Mapping - Introduction
The wizard is very straight forward and consists of setting the properties of the FlashCopy Mapping, choosing the VDisks to be used and to verify the settings. The first step, to set the properties of the FlashCopy mapping, is shown in Figure 6-3 on page 120.
Chapter 6. Implementing FlashCopy
119
Figure 6-3 Create a FlashCopy fcm- Set the Properties
We can choose the FlashCopy Mapping Name, or let the system decide. We have not defined a Consistency Group yet, so the related input field Consistency Group Name is empty. Therefore, this mapping will be associated with Consistency Group 0, which is used for stand-alone mappings. The Background Copy Priority specifies the speed of the background copy the SVC attempts to achieve, in case we would want to have the source copied to the target in fully. The possible speed is from 128 KBps at 10% priority, up to 64 MB/s at 100% priority. Priority 50 has a target copy rate of 2 MBps. Refer to 5.2.3, FlashCopy background copy on page 95 for a discussion about background copy. For our purpose, we chose NOCOPY, which means no data gets copied by the background copy process. The Cleaning Priority does not need to be set because there is no mapping yet which would depend on this mapping (this new parameter is discussed in 5.5, Dependencies between FlashCopy Mappings on page 110). The type of mapping will be Standard, as opposed to Incremental. We do not need it to be incremental because we do not want a full copy of the source VDisk which could benefit from being incrementally refreshed. Also, incremental FlashCopy would use up to twice the amount of bitmap space compared to standard FlashCopy (Incremental FlashCopy is discussed in 5.2.8, Incremental FlashCopy on page 99). The grain size we use is 256KB. The new option of a grain size of 64KB uses four times the amount of bitmap space compared to a grain size of 256KB. A Grain Size of 64KB can be used to decrease the amount of data to be copied when restarting an Incremental FlashCopy mapping and thus the time it takes to re-establish a full independent clone of a source VDisk. 120
The last feature which can be used on this screen is that we can choose to have the system automatically delete the FlashCopy mapping when the background copy completes. We are not creating a full copy here, so we do not use this option. Once we click on the Next button, we get a screen where we can filter the source VDisk candidates we want to see in the next step, shown in Figure 6-4 on page 121.
Figure 6-4 Create a FlashCopy mapping - Filtering Source VDisk Candidates
We can filter by Virtual Disk (VDisk) Name and Managed Disk Group Name (using a wildcard), by Virtual Disk Type (any, sequential, striped, image), I/O Group and State (offline, online, degraded). In our example we have chosen to filter by VDisk name, since our naming scheme indicates the production VDisks to be used as source VDisk with the characters prod as pat of the VDisk name. The next screen shows us the filtered VDisks, as shown in Figure 6-5 on page 122.
121
Figure 6-5 Create a FlashCopy mapping - Select the Source VDisk
We select the VDisk vd-prod-0001 to be the source VDisk for our FlashCopy mapping. Pressing Next, we get to the screen where we can choose the target VDisk, this time without a filter screen before, therefore displaying all VDisks in the system. To get the target VDisk candidates filtered, we select Show Filter Row from the pull down menu, then click on filter on top of the Name column to get to the filter properties, shown in Figure 6-6 on page 122.
Figure 6-6 Create a FlashCopy mapping - Select the Target VDisk - Filter
Here we do not need to use the wildcard as we did within the filter screen for the source VDisk candidates. After selecting the VDisk to be the target VDisk and pressing Next, we verify the settings of the FlashCopy mapping as shown in Figure 6-7 on page 123.
122
Figure 6-7 Create a FlashCopy mapping - Verify FlashCopy mapping
Pressing Finish creates the mapping and takes Viewing FlashCopy screen, shown in Figure 6-8 on page 123.
Figure 6-8 Viewing FlashCopy mappings - one mapping created, state: Idle or Copied
We have successfully created the first FlashCopy mapping whose properties we can view, clicking on the FlashCopy mapping Name, shown in Figure 6-9 on page 124.
123
Figure 6-9 View FlashCopy mapping Details for fcm-001
We see the state of the mapping is Idle_or_Copied, therefore it is ready to be prepared and started. The consistency group related fields hold no information, internally this mapping belongs to Consistency Group 0.
6.1.2 Starting a FlashCopy mapping

Given that the application accessing and altering the data on the source VDisk has been stopped from changing the data, we can now start the FlashCopy mapping. To start it, we need to prepare and then start the mapping. This can be done either by selecting Prepare a Mapping from the pull down menu or by selecting Start a Mapping , as shown in Figure 6-10 on page 124.
Figure 6-10 Starting a FlashCopy Mapping
124
We get to the screen used to chose to prepare the mapping, shown in Figure 6-11 on page 125.
Figure 6-11 Starting FlashCopy Mapping fcm-001
Pressing OK first prepares (puts the mapping in the state Preparing) and then starts the mapping. The state of the mapping is now Copying, shown in Figure 6-12 on page 125. The application altering the data residing on the source VDisk can resume to do so, the target VDisk is brought online and ready to be used.
Figure 6-12 Viewing FlashCopy Mappings - one mapping started, state: Copying
6.1.3 Modify a mapping

It is possible to modify the properties of the mapping we created and started. Selecting the mapping and choosing Modify a Mapping from the pull down menu takes us to the screen shown in Figure 6-13 on page 126.
125
Figure 6-13 Modifying FlashCopy Mapping fcm-001
Here, the name of the mapping, the Consistency group it belongs to, the background copy priority, the cleaning priority and the autodelete option can be modified. What cannot be modified is the grain size and the type of mapping (standard, incremental) and the I/O group the bitmap resides on.
6.1.4 Stopping a mapping

After we have our data from the target VDisk backed up we stop the mapping, shown in Figure 6-14 on page 126. This is needed in the next step, when we use this mapping as a member in a new Consistency Group.
Figure 6-14 Stop a FlashCopy Mapping
126
Now we get a warning screen, shown in Figure 6-15 on page 127, telling us that the content of the target VDisk will be unusable after we stop the mapping.
Figure 6-15 Stopping a FlashCopy Mapping fcm-001 - Stop or Forced Stop
This is because most of the data of the target VDisk is still located on the source VDisk, only data which would have been overwritten on the source VDisk has been copied to the target VDisk (we had chosen not to enable the background copy process). What this means is that the mapping will still exist, but whatever data was written to the target VDisk will be invalidated. Also, any mapping which depends on this mapping would be affected. When we use the Forced Stop, no further cleaning occurs and dependent mappings will as well be stopped. We do not have any dependent mappings at that point in time, so we press the Stop button (we do not need to use the Forced Stop button). Now the status of the mapping is Stopping, as seen in Figure 6-16 on page 127, and shortly thereafter in the state Stopped.
Figure 6-16 FlashCopy Mapping, state: Stopping
6.1.5 Creating a FlashCopy Consistency Group

Our application has related data spanning multiple VDisks. To be able to produce a consistent snapshot of the data we put them into a Consistency Group to ensure not only
127
consistency of the data within one VDisk, but also across data spanning all the involved VDisks. First we choose FlashCopy Consistency Groups on the My Work pane as in Figure 6-17 on page 128.
Figure 6-17 Viewing FlashCopy Consistency Groups, no groups exist
There are no Consistency Groups set up yet. To create one we choose Create a Consistency Group from the pull down menu. The next screen, shown in Figure 6-18 on page 128, lets us choose the name of the Consistency Group and what FlashCopy mappings we want to be members in it.
Figure 6-18 Creating FlashCopy Consistency Groups - including the one existing mapping
We want to include the mapping FlashCopy fcm-001 as the first member in the Consistency Group, therefore we select it and press OK. If we had created some more mappings, they would be displayed here and could be included in the Consistency Group at once.
128
The next screen, shown in Figure 6-19 on page 129, displays the result of the creation, which in our case was a success. If, for instance, the mapping put into the Consistency Group was in the Copying state, there would be an error message indicating this. Nonetheless, an empty Consistency Group would have been created.
Figure 6-19 Creating FlashCopy Consistency Groups - Success
Closing this screen brings us back to the Viewing FlashCopy Consistency Group screen seen in Figure 6-20 on page 129.
Figure 6-20 Viewing FlashCopy Consistency Groups - one group created
Clicking on the Consistency Group Name shows us the properties of the Consistency Group. The first panel, General, shows just the name, ID and the state of the Consistency Group, as shown in Figure 6-21 on page 130.
129
Figure 6-21 View FlashCopy Consistency Group Details for fcg-001 - General
The Consistency Group state is Stopped, since its member, the FlashCopy Mapping fcm-001 is in the Stopped state. The Relationship panel shows the details for the relationship as shown in Figure 6-22 on page 130.
Figure 6-22 View FlashCopy Consistency Group Details for fcg-001 - Relationships
6.1.6 Creating a FlashCopy mapping and adding it to the Consistency Group

We can create a new mapping without choosing a Consistency Group, thus associating it with Consistency Group 0, as we did in 6.1.1, Creating a FlashCopy mapping on page 118. We can then later place it into a Consistency Group from within Viewing FlashCopy Mappings screen with using Modify a Mapping from the pull down menu. If we already know which Consistency Group the mapping should be a member of, we can put the mapping into the Consistency Group from within the Create a FlashCopy Mapping wizard in one step, shown in Figure 6-23 on page 131.
130
Figure 6-23 Create a FlashCopy Mapping - Set Properties - Include in existing Consistency Group
The next screen lets us again filter the VDisks to be displayed as source VDisk candidates. This time we filter using the Managed Disk Group name (the backend storage system which holds the production VDisks is the DS4500), shown in Figure 6-24 on page 131.
Figure 6-24 Create a FlashCopy Mapping - Filtering Source VDisk Candidates - filter by mdgroup
131
Now we choose the source VDisk for the mapping from the filtered list, shown in Figure 6-25 on page 132.
Figure 6-25 Create a FlashCopy Mapping - Select the Source VDisk
We now choose the target VDisk for the new mapping shown in Figure 6-26.
Figure 6-26 Create a FlashCopy Mapping - Select the Target VDisk
The last step before the mapping is created is to verify the settings, as shown in Figure 6-27 on page 133.
132
Figure 6-27 Create a FlashCopy Mapping - Verify FlashCopy Mapping
Now we return to the Consistency Group screen, shown in Figure 6-28, and we see that both mappings are now members of the Consistency Group fcg-001.
Figure 6-28 Viewing FlashCopy Mappings - two mappings are members of one Consistency Group
The state of both mappings is Idle_or_Copied and therefore also the state of the Consistency Group, shown in Figure 6-29 on page 134.
133
Figure 6-29 Viewing FlashCopy Consistency Groups - fcg-001, state Idle or Copied
When clicking on the Name of the Consistency Group, we see the details of the Consistency Group. Clicking on Relationships now, as shown in Figure 6-30, we now see both FlashCopy Mappings as members of the Consistency Group.
Figure 6-30 View FlashCopy Consistency Group Relationship Details for fcg-001
6.1.7 Preparing a FlashCopy Consistency Group

The FlashCopy Mapping states are described in 5.4.3, FlashCopy consistency group states on page 109. There we can see that a Consistency Group (as well as a FlashCopy Mapping) needs to be prepared before it is started. We can do this manually before we start the Consistency Group. Even if we do not prepare the Consistency Group (and therefore the mappings within), it will be prepared when choosing Start a Consistency Group. Doing it manually has some advantages. One is that, depending on the amount of data associated with the VDisks to be copied still in the cache and the load on the system, it can take some time to prepare a mapping.
134
A command needs to be completed within a certain period of time. The prepare command is asynchronous and if it takes more than two minutes for the maps to get to the "prepared" state that will not cause a problem. The start command is synchronous and must complete within two minutes. In a case where the start command is being prevented from being executed within two minutes, the event flush failed takes the mapping into the Stopped state. As it is in the Stopped state, it can be prepared again. The advantage of preparing the mappings first is that in the case just described, the mappings are already prepared, thus the time does not run out for the completion of the Start process. Another advantage is that when preparing it first, we can specify the point-in-time to start the Consistency Group or the mapping, whereas with an unprepared Consistency Group, the (variable) time to prepare the Consistency Group decides the exact point-in-time it is started. From the Consistency Group screen we choose Prepare a Consistency Group, as shown in Figure 6-31 on page 135.
Figure 6-31 Prepare a Consistency Group
A Consistency Group can only be put into the Preparing state, shown in Figure 6-32 when it is either in the Idle_or_Copied or Stopped state.
135
Figure 6-32 Prepare a Consistency Group - state: Preparing
After a refresh, the state changes to Prepared, as shown in Figure 6-33 on page 136.
Figure 6-33 Prepare a Consistency Group - state: Prepared
6.1.8 Starting a FlashCopy Consistency Group

To start our Consistency Group without preparing it first, we use the pull-down menu from the Viewing FlashCopy Consistency Group screen, as shown in Figure 6-34 on page 137.
136
Figure 6-34 Viewing FlashCopy Consistency Groups - Start a Consistency Group
As with starting the standalone mapping, when starting the Consistency Group, the next screen asks us to prepare the Consistency Group, as shown in Figure 6-35. This only applies if we did not prepare the Consistency Group before; if it has been done before the checkbox will be greyed out.
Figure 6-35 Starting FlashCopy Consistency Group fcg-001 - preparing the mappings within the group
There is a checkbox to prepare the FlashCopy Consistency group (which is checked by default, but can be unchecked). As stated before, a Consistency Group or a FlashCopy Mapping always needs to be prepared before being started. The checkbox serves no other purpose than telling us that the Consistency Group will be prepared before it is being started. If we uncheck it, the start of the Consistency Group will fail as shown in Figure 6-36.
137
Figure 6-36 Starting FlashCopy Consistency Group fcg-001 - error when using the checkbox
Clicking the OK button, with the checkbox left checked, The Consistency Group first enters the Preparing, then Prepared and finally Copying state, in one uninterrupted sequence, as shown in Figure 6-37 on page 138.
Figure 6-37 Viewing FlashCopy Consistency Groups, fcg-001 in state: Copying
Looking at the details of the Consistency Group fcg-001 we see that both mappings, which are members of the Consistency Group, are in the Copying state, shown in Figure 6-38.
138
Figure 6-38 Viewing FlashCopy Consistency Group Relationship Details for fcg-01, state: Copying
6.1.9 Stopping a Consistency Group

Because of the NOCOPY option we have chosen, the target VDisks will be a valid snapshot of the source VDisks as long as the mappings exist. All we can do to manipulate the Consistency Group now is to stop it, which will end the relationship and which renders all data on the target VDisk invalid. To stop the Consistency Group, we choose Stop a Consistency Group from the pull-down menu, seen in Figure 6-39.
Figure 6-39 Stop a Consistency Group
Figure 6-40 on page 140 shows the confirmation screen where we can just stop the Consistency Group or chose to force the Consistency Group to be stopped in case there are mappings within which have dependencies to other mappings outside the Consistency Group to be stopped.
139
Figure 6-40 Stopping FlashCopy Consistency Group fcg-001
Before reaching the Stopped state, the Consistency Group enters the Stopping state, as shown in Figure 6-41.
Figure 6-41 Stopping a Consistency Group, state: Stopping
6.1.10 Renaming a Consistency Group

The only property of a Consistency Group to be changed from within the Consistency Group screen is to rename it. Again, we use the pull-down menu to chose the action as shown in Figure 6-42 on page 141.
140
Figure 6-42 Rename a Consistency Group - pull-down menu
The next screen shows is where we can change the name of the Consistency Group, this can be done in the Copying state, as shown in Figure 6-43 on page 141.
Figure 6-43 Renaming FlashCopy Consistency Group fcg-001
6.1.11 Issuing a command to a mapping within a Consistency Group

From within the FlashCopy Mapping screen, we can stop a FlashCopy Mapping and if this mapping is a member of a Consistency Group, we stop the whole Consistency Group while stopping the mapping. We will do this with the mapping FlashCopy fcm-001. This also puts the other mappings, which are members of that Consistency Group in the Stopping state, shown in Figure 6-44 on page 142, and then into the Stopped state.
141
Figure 6-44 Viewing FlashCopy Mappings, one Mapping in state: Stopped, the other in state: Stopping
The same applies to preparing and starting a FlashCopy Mapping. When we prepare or start a mapping which is a member of a Consistency Group, we get an additional confirmation screen which tells us that since the FlashCopy Mapping is a member of a Consistency Group, we will issue the command to all mappings which are members of the Consistency Group, as shown in Figure 6-45 on page 142.
Figure 6-45 Starting FlashCopy Mappings Confirmation
6.2 Example 2: Multiple Target FlashCopy using the GUI

Here we use FlashCopy where one VDisk is acting as a source VDisk and it can be in multiple mappings with different VDisks, acting as target VDisk. Goal: The goal is to create multiple consistent copies of the same data. If we need copies of VDisks for different purposes (and with different properties), it could happen that we require these copies of the source at the same time. Solution: The solution is using MTFC to create multiple snapshots of one source. For instance, with MTFC we can create FlashCopies for our daily backup and others for application testing using the same sources. The snapshots for backup could be set up as described in 6.1, Example 1: basic FlashCopy using the GUI on page 118. The snapshots to be used for application testing we set up with different properties, for instance, we would like to have the VDisks copied fully using the background copy process to establish real independent clones. 142
6.2.1 Creating FlashCopy Mappings and a Consistency Group

We already have the FlashCopy Mappings established for our backup purposes, shown in Figure 6-46.
Figure 6-46 Viewing FlashCopy Mappings - two mappings for backup in Consistency Group fcg-001
Clicking the name of the mapping gives us the properties of the mapping as shown in Figure 6-47.
143
Figure 6-47 View FlashCopy Details for fcm-001
Now we are going to establish two new mappings which use the same source VDisk but have a different target VDisk, shown in Figure 6-48 on page 144.
Figure 6-48 Viewing FlashCopy Mappings - two more Mappings using the same sources
144
The properties of the mapping FlashCopy fcm-test-001 are shown in Figure 6-49.
Figure 6-49 View FlashCopy Mapping Details for FlashCopy fcm-test-001
The background copy priority is 50% for these mappings. This tells the SVC to perform a background copy of the data of the source VDisk to the target VDisk with 2 MBps. As we do not need the mapping after the clone of the source VDisks is established, the option to delete the mapping when the background copy has finished is enabled. The mappings are not yet in a Consistency Group. As we want the timing of the FlashCopy to be independent from the Consistency Group we use for backup, we put the mappings in a new Consistency Group. We now create the Consistency Group and while doing so, put both new FlashCopy Mappings into the Consistency Group, as shown in Figure 6-50 on page 146.
145
Figure 6-50 Creating FlashCopy Consistency Groups - including both Mappings for test data
To check if all settings are correct, we look up the FlashCopy relationship details of the Consistency Group, as shown in Figure 6-51.
Figure 6-51 FlashCopy Consistency Group Details for fcg-test-001
6.2.2 Starting a Consistency Group in an MTFC tree

After we set up the Consistency Group, we can prepare and start it. As we can see in Figure 6-52 on page 147, the mappings in Consistency Group fcg-test-001 are in the Copying state while the mappings in fcg-001 are in the Idle_Copyied state.
146
Figure 6-52 Viewing FlashCopy Mappings - mappings in fcg-test-001 in state: Copying
Now we start (and prepare in one step) the FlashCopy Mapping fcg-001 again for our backup purposes. Both of our Consistency Groups which have mappings which share the same source VDisks, are in the Copying state. Selecting a mapping in fcg-001 and choosing View FC Mappings in Dependency Order from the pull-down menu, shown in Figure 6-53 on page 147, lets us obtain information about the dependencies the mappings have.
Figure 6-53 View FC Mappings in Dependency Order
The list of mappings which are dependent on the mapping fcm-001 are shown in Figure 6-54.
147
Figure 6-54 Viewing FlashCopy Mappings in Dependency Order for mapping FlashCopy fcm-001
The only mapping which depends on the chosen mapping is fcm-test-001. The mapping fcm-001 itself is not shown in the list. The mapping fcm-test-001 depends on the mapping fcm-001 because fcm-001 is newer (started later) than fcm-test-001. After the SVC finished copying the data of the source VDisks to the target VDisks for the mapping used to establish a clone of the production VDisks for test purposes, it deleted the related mapping as we specified. Now, we only have 2 mappings existing which are the ones we use for our backups,as shown in Figure 6-55.
Figure 6-55 Viewing FlashCopy Mappings - only the mappings in Consistency Group fcg-001 left
6.3 Example 3: Cascaded and Incremental FC using the GUI

In this example we implement FlashCopy where a VDisk can be a target VDisk in one mapping, and a Source VDisk in another mapping. Also, we implement a FlashCopy which can be incrementally refreshed. This will be implemented using Consistency Groups.
148
Goal 1: The first goal is to take a snapshot of a VDisk, which is already the target VDisk in a mapping. We create and start a FlashCopy Mapping and, additionally, want to establish a FlashCopy where the target VDisk of the first FlashCopy Mapping is the source of the second FlashCopy Mapping. The first copy we set up with background copy to get a fully copied clone of a production VDisk. This first copy is to be used for application testing. The application working with the first copy might change the data residing on the target VDisk of the first FlashCopy Mapping. We now want to have a copy of the target VDisk of the first FlashCopy Mapping. With a FlashCopy function where a VDisk can be a member of only one FlashCopy Mapping, we would have to wait until the first copy has finished to fully copy the data to the target VDisk. Then we would have to delete the mapping (or have the autodelete option enabled). This can be a time-consuming procedure. Solution1 : We use CFC to create a FlashCopy where a source VDisk of the second mapping is still the target VDisk in the first FlashCopy Mapping. This enables us to create the second copy before the first FlashCopy Mapping has finished copying all the data. Goal 2: The second goal is to refresh a FlashCopy target. We would like to have the first copy (the copy of the production VDisk) to be re-established within a short period of time and without much impact on the production VDisk. Solution 2: To re-establish the first copy in a short period of time and without much impact on the production volume, we use IFC for the first FlashCopy Mapping.
6.3.1 Creating FlashCopy Consistency Groups

First we create two FlashCopy Consistency Groups, named in such a way so that we know that the second Consistency Group is supposed to hold mappings for the copy of the copy, as shown in Figure 6-56.
Figure 6-56 Viewing FlashCopy Consistency Groups - two empty Consistency Groups created
6.3.2 Create Incremental FlashCopy Mappings

We put the mapping to be created into the first Consistency Group. Also, we set it up to be incremental, which is shown in Figure 6-57 on page 150.
149
Figure 6-57 Create FlashCopy Mapping - Incremental
We now associate VDisks to be the source VDisk and the target VDisk with the mapping and finish the wizard. The properties of the mapping we created are shown in Figure 6-58 on page 151.
150
Figure 6-58 View FlashCopy Mapping Details for FlashCopy fcm-test-001
We set up the second mapping, for the second production VDisk the same way we did with the first. Both mappings are in the Idle_or_Copied state and are shown in Figure 6-59 on page 151.
Figure 6-59 Viewing FlashCopy Mappings - two mappings in the first Consistency Group
151
We can now switch to the Consistency Group screen and start the first Consistency Group, as shown in Figure 6-60 on page 152.
Figure 6-60 Start a Consistency Group
This puts the Consistency Group in the Copying state, as shown in Figure 6-61.
Figure 6-61 View FlashCopy Consistency Groups
Now the target VDisks of the first mapping are ready to be used for our application testing. the application testing will need to be temporarily stopped (and any host caches flushed) before the cascaded mappings are started. While this is ongoing, we create the next two mappings, where the target VDisks of the now Copying first mapping are to be the source VDisks. We put the mappings in another Consistency Group and specify them to be incremental as well, as shown in the Verify FlashCopy Mapping screen at the final stage of creating the mapping, as shown in Figure 6-62 on page 153.
152
Figure 6-62 Verify FlashCopy Mapping
Now we have set up four mappings in two Consistency Groups, shown in Figure 6-63.
Figure 6-63 Viewing FlashCopy Mappings - 1 new Consistency Group in state: Idle_or_Copied
While the first Consistency Group and with it the mappings associated with it are still in the Copying state, we start the second Consistency Group, shown in Figure 6-64 on page 154.
153
Figure 6-64 Start a Consistency Group
We now have four mappings in two Consistency Groups in a Copying state, where VDisks act as both, as source VDisks and target VDisks, shown in Figure 6-65.
Figure 6-65 View FlashCopy Mappings - VDisks in the role of Source VDisk and Target VDisk
The mapping FlashCopy fcm-test-003 depends upon FlashCopy fcm-test-001, shown in Figure 6-66 on page 155. The same applies to the other two mappings.
154
Figure 6-66 Viewing FlashCopy Mappings in Dependency Order for mapping FlashCopy fcm-test-001
After some time, the progress indicates that the copy process for the first FlashCopy Mapping is almost complete, shown in Figure 6-67.
Figure 6-67 Viewing FlashCopy Mappings - first mapping copy process almost complete
We can see the progress using the Manage Progress screens, shown in Figure 6-68 on page 156, where we see that the first mapping successfully finished copying all the data to the target VDisk.
155
Figure 6-68 Viewing FlashCopy Progress
After both copies are copied fully, the state changes back to Idle_or_Copied for both Consistency Groups, shown in Figure 6-69 on page 156.
Figure 6-69 Viewing FlashCopy Consistency Groups - both in state Idle_or_Copied
We now restart the first Consistency Group, which has the production VDisks as source VDisks, shown in Figure 6-70 on page 157.
156
Figure 6-70 (Re-)Start a Consistency Group
The screenshot in Figure 6-71 was taken soon after we started the FlashCopy Mapping. Since the mapping is incremental, it just had to copy the grains which were changed, which was about 200 MB, instead of the whole 10 GB VDisk.
Figure 6-71 Viewing FlashCopy Mappings - mappings in fcg-test-001 refreshed
6.4 Example 4: FlashCopy of Metro Mirror VDisks using the CLI

Here we implement FlashCopy where the source VDisks are the secondary VDisks in a remote copy relationship. Goal: The goal is to create a full consistent copy of production data for a given point-in-time at a remote location. We have a Remote Copy Relationship set up to copy VDisks to a second site using intercluster Metro Mirror, for disaster tolerance. We want to have a consistent
157
backup of our production data taken at the remote site. Also, we want to have a full copy (a clone) of our production VDisks on a daily basis, residing at the second location. This copy serves the purpose of providing a consistent point-in-time copy of the production data in case the data of both the primary and secondary of the Metro Mirror relationship get invalidated (for example, due to a logical error which affects both the primary and the secondary due to them being in a synchronized copy relationship). This clone is a building block for our Business Continuity strategy. Note: Even though Global Mirror is asynchronous, this use case applies to GM as well, because the data gets copied over to the secondary VDisk as soon as possible and thus is object to get invalidated by logical errors. Solution: The solution is to setup FlashCopy where the source VDisks for the mappings are the secondary VDisks of the one MetroMirror Relationship. The target VDisks are then used to take a backup from. We establish the clone of the Metro Mirror secondary using the background copy process and we use IFC to refresh this clone once created to minimize load on the system, and to establish the clone quickly. To have the clone established in a short time decreases the time where no full clone of the VDisks is available, thus enhancing our Recovery Time Objectives.
6.4.1 Checking VDisks and Metro Mirror relationship to be used

To check the VDisks to be used for our FlashCopy Mapping we use the command svcinfo lsvdisk -delim :, shown in Example 6-1. To eliminate the white space between the output columns we use the -delim switch.
Example 6-1 svcinfo lsvdisk -delim : - VDisks to be used
IBM_2145:ITSOCL3:admin>svcinfo lsvdisk -delim : id:name:IO_group_id:IO_group_name:status:mdisk_grp_id:mdisk_grp_name:capacity:type :FC_id:FC_name:RC_id:RC_name:vdisk_UID:fc_map_count 0:vd-prod-l2-0002:0:io_grp0:online:1:DS4700:25.0GB:striped:::0:MetroMirror Relationship-002:6005076801A100E90800000000000003:0 1:vd-prod-l2-0001:0:io_grp0:online:1:DS4700:25.0GB:striped:::1:MetroMirror Relationship-001:6005076801A100E90800000000000002:0 2:vd-back-l2-0001:0:io_grp0:online:1:DS4700:25.0GB:striped:::::6005076801A100E9080 0000000000004:0 3:vd-back-l2-0002:0:io_grp0:online:1:DS4700:25.0GB:striped:::::6005076801A100E9080 0000000000005:0 IBM_2145:ITSOCL3:admin> The VDisks which have the characters prod in the name we are going to use them as the source VDisk for the FlashCopy Mapping and the VDisks with the characters back in the name we will use as the target VDisks. As an example, we display the properties of the VDisk with the id 2, using the command svcinfo lsvdisk -delim : 2, shown in Example 6-2.
Example 6-2 svcinfo lsvdisk -delim : 2 - details of VDisk 2
IBM_2145:ITSOCL3:admin>svcinfo lsvdisk -delim : 2 id:2 name:vd-back-l2-0001 IO_group_id:0 IO_group_name:io_grp0 158
status:online mdisk_grp_id:1 mdisk_grp_name:DS4700 capacity:25.0GB type:striped formatted:no mdisk_id: mdisk_name: FC_id: FC_name: RC_id: RC_name: vdisk_UID:6005076801A100E90800000000000004 throttling:0 preferred_node_id:2 fast_write_state:empty cache:readwrite udid:0 fc_map_count:0 IBM_2145:ITSOCL3:admin> To display the one MetroMirror Relationship we use the command svcinfo lsrcconsistgrp, as shown in Example 6-3, to display it.
Example 6-3 svcinfo lsrcconsistgrp -delim - existing remote copy consistency groups
IBM_2145:ITSOCL3:admin>svcinfo lsrcconsistgrp -delim : id:name:master_cluster_id:master_cluster_name:aux_cluster_id:aux_cluster_name:prim ary:state:relationship_count:copy_type 255:mmg-001:000002006B603A38:ITSOCL2:0000020068403A42:ITSOCL3:master:consistent_sy nchronized:2:metro IBM_2145:ITSOCL3:admin> As we can see, only one relationship exists. To show the properties of only this relationship, we specify the id with the command, shown in Example 6-4.
Example 6-4 svcinfo lsrcconsistgrp -delim : 255 - properties of the Consistency Group with id 255
IBM_2145:ITSOCL3:admin>svcinfo lsrcconsistgrp -delim : 255 id:255 name:mmg-001 master_cluster_id:000002006B603A38 master_cluster_name:ITSOCL2 aux_cluster_id:0000020068403A42 aux_cluster_name:ITSOCL3 primary:master state:consistent_synchronized relationship_count:2 freeze_time: status:online sync: copy_type:metro RC_rel_id:0 RC_rel_name:MetroMirror Relationship-002 RC_rel_id:1 RC_rel_name:MetroMirror Relationship-001
159
IBM_2145:ITSOCL3:admin> All important information is shown to verify that we have successfully set up the remote copy consistency group, which is associated with two Metro Mirror relationships. We can also verify the properties of the two Metro Mirror relationships, with one of the two shown in Example 6-5, using the command svcinfo lsrcrelationship -delim : 0.
Example 6-5 .svcinfo lsrcrelationship -delim : 0 - properties of the Relationship with id 0
IBM_2145:ITSOCL3:admin>svcinfo lsrcrelationship -delim : 0 id:0 name:MetroMirror Relationship-002 master_cluster_id:000002006B603A38 master_cluster_name:ITSOCL2 master_vdisk_id:1 master_vdisk_name:vd-prod-l1-0002 aux_cluster_id:0000020068403A42 aux_cluster_name:ITSOCL3 aux_vdisk_id:0 aux_vdisk_name:vd-prod-l2-0002 primary:master consistency_group_id:255 consistency_group_name:mmg-001 state:consistent_synchronized bg_copy_priority:50 progress: freeze_time: status:online sync: copy_type:metro IBM_2145:ITSOCL3:admin> The second relationship we use VDisks from for our FlashCopy Mapping is shown in Example 6-6.
Example 6-6 svcinfo lsrcrelationship -delim : 0 - properties of the Relationship with id 1
IBM_2145:ITSOCL3:admin>svcinfo lsrcrelationship -delim : 1 id:1 name:MetroMirror Relationship-001 master_cluster_id:000002006B603A38 master_cluster_name:ITSOCL2 master_vdisk_id:0 master_vdisk_name:vd-prod-l1-0001 aux_cluster_id:0000020068403A42 aux_cluster_name:ITSOCL3 aux_vdisk_id:1 aux_vdisk_name:vd-prod-l2-0001 primary:master consistency_group_id:255 consistency_group_name:mmg-001 state:consistent_synchronized bg_copy_priority:50 progress: freeze_time: 160
status:online sync: copy_type:metro IBM_2145:ITSOCL3:admin> The VDisks of our MetroMirror Relationship in location 1 (Production Location) are named: vd-prod-l1-0001 vd-prod-l1-0002 In location 2 (Business Continuity Location) they are named: vd-prod-l2-0001 vd-prod-l2-0002 The two VDisks which serve as the secondary VDisks for the MetroMirror Relationship in location 2 are the ones we use as the source for the FlashCopy Mapping.
6.4.2 Create FlashCopy mappings

We set up the FlashCopy mappings and Consistency Group in the following order. 1. Create FlashCopy mappings 2. Create a FlashCopy Consistency Group 3. Add FlashCopy mappings to the Consistency Group Note: Another method would be to create the Consistency Group first and then create the mappings and add them to the Consistency Group in the same step. Mappings are to be created using the command svctask mkfcmap, and the syntax is shown in Example 6-7.
Example 6-7 svctask mkfcmap - syntax
>>- svctask -- -- mkfcmap -- -----------------------------------> >-- -source --+- src_vdisk_id ---+-- ---------------------------> '- src_vdisk_name -' >-- -target --+- target_vdisk_id ---+-- ------------------------> '- target_vdisk_name -' >--+-------------------------+-- -------------------------------> '- -name -- new_name_arg -' >--+-----------------------------------------+-- ---------------> '- -consistgrp --+- consist_group_id ---+-' '- consist_group_name -' >--+------------------------+-- --+---------------+-------------> '- -copyrate -- percent -' '- -autodelete -' >--+-------------------------+--+----------------+--------------> '- -grainsize --+- 64 --+-' '- -incremental -' '- 256 -' >--+-----------------------------+------------------------------>
161
'- -cleanrate ---- percent ---' >--+------------------------------+---------------------------->< '- -iogrp --+- iogroup_name -+-' '- iogroup_id --' We do not use all the optional parameters, for instance we leave the Grain Size to default at 256KB. In our example we create the first mapping as shown in Example 6-8.
Example 6-8 svctask mkfcmap - create first mapping
IBM_2145:ITSOCL3:admin>svctask mkfcmap -source vd-prod-l2-0001 -target vd-back-l2-0001 -name fcm-back-001 -copyrate 70 -incremental FlashCopy Mapping, id [0], successfully created IBM_2145:ITSOCL3:admin> We then create the second mapping, shown in Example 6-9.
Example 6-9 svctask mkfcmap - create second mapping
IBM_2145:ITSOCL3:admin>svctask mkfcmap -source vd-prod-l2-0002 -target vd-back-l2-0002 -name fcm-back-002 -copyrate 70 FlashCopy Mapping, id [1], successfully created IBM_2145:ITSOCL3:admin> We review the creation of both mappings using the command svcinfo lsfcmap -delim :, shown in Example 6-10.
Example 6-10 svcinfo lsfcmap -delim : - both mappings
IBM_2145:ITSOCL3:admin>svcinfo lsfcmap -delim : id:name:source_vdisk_id:source_vdisk_name:target_vdisk_id:target_vdisk_name:group_ id:group_name:status:progress:copy_rate:clean_progress:incremental 0:fcm-back-001:1:vd-prod-l2-0001:2:vd-back-l2-0001:::idle_or_copied:0:70:100:on 1:fcm-back-002:0:vd-prod-l2-0002:3:vd-back-l2-0002:::idle_or_copied:0:70:100:off IBM_2145:ITSOCL3:admin> Reviewing the mappings we realize that we forgot the -incremental parameter in our second mapping. We cannot modify this mapping to be incremental (modifying a mapping would be done using the command svctask chfcmap), so we have to delete the mapping and re-create it. We delete the mapping using the command svctask rmfcmap, shown in Example 6-11.
Example 6-11 svctask refcmap - syntax
>>- svctask -- -- rmfcmap -- --+----------+-- ------------------> '- -force -' >--+- fc_map_id ---+------------------------------------------->< '- fc_map_name -' After we recreated the second mapping, this time using the flag -incremental, we view the properties of that mapping using the command svcinfo lsfcmap 1, shown in Example 6-12.
Example 6-12 svcinfo lsfcmap 1 - properties of the mapping with id 1
IBM_2145:ITSOCL3:admin>svcinfo lsfcmap 1 id 1 162

name fcm-back-002 source_vdisk_id 0 source_vdisk_name vd-prod-l2-0002 target_vdisk_id 3 target_vdisk_name vd-back-l2-0002 group_id group_name status idle_or_copied progress 0 copy_rate 70 start_time dependent_mappings 0 autodelete off clean_progress 100 clean_rate 50 incremental on difference 100 grain_size 256 IO_group_id 0 IO_group_name io_grp0 IBM_2145:ITSOCL3:admin> Using svcinfo lsvdisk again, shown in Example 6-13, we see that all four VDisks are now part of FlashCopy Mappings.
Example 6-13 svcinfo lsvdisk -delim : - all four VDisks are now part of FlashCopy mappings
IBM_2145:ITSOCL3:admin>svcinfo lsvdisk -delim : id:name:IO_group_id:IO_group_name:status:mdisk_grp_id:mdisk_grp_name:capacity:type :FC_id:FC_name:RC_id:RC_name:vdisk_UID:fc_map_count 0:vd-prod-l2-0002:0:io_grp0:online:1:DS4700:25.0GB:striped:1:fcm-back-002:0:MetroM irror Relationship-002:6005076801A100E90800000000000003:1 1:vd-prod-l2-0001:0:io_grp0:online:1:DS4700:25.0GB:striped:0:fcm-back-001:1:MetroM irror Relationship-001:6005076801A100E90800000000000002:1 2:vd-back-l2-0001:0:io_grp0:online:1:DS4700:25.0GB:striped:0:fcm-back-001:::600507 6801A100E90800000000000004:1 3:vd-back-l2-0002:0:io_grp0:online:1:DS4700:25.0GB:striped:1:fcm-back-002:::600507 6801A100E90800000000000005:1 IBM_2145:ITSOCL3:admin>
6.4.3 Creating a FlashCopy Consistency Group

Now we go on and create a FlashCopy Consistency Group to put the mappings into. The command to be used is svctask mkfcconsistgrp, and the syntax is shown in Example 6-14.
Example 6-14 svctask mkfcconsistgrp - syntax
>>- svctask -- -- mkfcconsistgrp -- ----------------------------> >--+-------------------------------+--------------------------->< '- -name -- consist_group_name -' The actual command we used to create our Consistency Group is shown in Example 6-15 on page 164.
163
Example 6-15 svctask mkfcconsistgrp - Consistency Group successfully created
IBM_2145:ITSOCL3:admin>svctask mkfcconsistgrp -name fcg-back-001 FlashCopy Consistency Group, id [1], successfully created IBM_2145:ITSOCL3:admin> Example 6-16 shows the syntax of the command svcinfo lsfcconsistgrp, which can be used to display the Consistency Groups which have been created.
Example 6-16 svcinfo lsfcconsistgrp - syntax
>>- svcinfo -- -- lsfcconsistgrp -- ----------------------------> >--+-----------------------------------+-- --+----------+-- ----> '- -filtervalue -- attribute=value -' '- -nohdr -' >--+-----------------------+-- --+---------------+-- -----------> '- -delim -- delimiter -' +- object_id ---+ '- object_name -' >--+-----------------+----------------------------------------->< '- -filtervalue? -' We use the command now to verify the creation of the Consistency Group, as shown in Example 6-17.
Example 6-17 svcinfo lsfcconsistgrp - empty Consistency Group
IBM_2145:ITSOCL3:admin>svcinfo lsfcconsistgrp id name status 1 fcg-back-001 empty IBM_2145:ITSOCL3:admin> For now, what we have is two FlashCopy mappings and one empty FlashCopy Consistency Group.
6.4.4 Adding FlashCopy mappings to the Consistency Group

As the final step before we are able to start the FlashCopy, we include the two mappings into the Consistency Group we just created. This is done using the command svctask chfcmap, and the syntax is shown in Example 6-18.
Example 6-18 svctask chfcmap - syntax
>>- svctask -- -- chfcmap -- --+-------------------------+-- ---> '- -name -- new_name_arg -' >--+----------+-- ----------------------------------------------> '- -force -' >--+-----------------------------------------+-- ---------------> '- -consistgrp --+- consist_group_id ---+-' '- consist_group_name -' >--+------------------------+-- --+-------------------------+---> '- -copyrate -- percent-' '- -autodelete --+-on--+-'
164
'-off-' >--+-----------------------------+--+- fc_map_id ---+---------->< '- -cleanrate ---- percent ---' '- fc_map_name -' The command we used to put the mappings into the Consistency Group is shown in Example 6-19.
Example 6-19 svctask chfcmap - associating both mappings with the same Consistency Group
IBM_2145:ITSOCL3:admin>svctask chfcmap -consistgrp fcg-back-001 fcm-back-001 IBM_2145:ITSOCL3:admin>svctask chfcmap -consistgrp fcg-back-001 fcm-back-002 IBM_2145:ITSOCL3:admin> To view the Consistency Group we use the command svcinfo lsfcconsistgrp again, shown in Example 6-20 on page 165.
Example 6-20 svcinfo lsfcconsistgrp - Consistency Group in state: Idle_or_Copied
IBM_2145:ITSOCL3:admin>svcinfo lsfcconsistgrp id name status 1 fcg-back-001 idle_or_copied IBM_2145:ITSOCL3:admin> The Consistency Group state changed from empty to idle_or_copied. The properties, including the FlashCopy mappings which are now associated with the Consistency group we get when specifying the id of the Group with the command, as shown in Example 6-21.
Example 6-21 svcinfo lsfcconsistgrp - properties of the Consistency Group with id 1
IBM_2145:ITSOCL3:admin>svcinfo lsfcconsistgrp 1 id 1 name fcg-back-001 status idle_or_copied FC_mapping_id 0 FC_mapping_name fcm-back-001 FC_mapping_id 1 FC_mapping_name fcm-back-002 IBM_2145:ITSOCL3:admin> Example 6-22 shows the two FlashCopy mappings, using the command svcinfo lsfcmap, which shows the fact that both mappings are now members of the same Consistency Group.
Example 6-22 svcinfo lsfcmap - both mappings associated with the same Consistency Group
IBM_2145:ITSOCL3:admin>svcinfo lsfcmap -delim : id:name:source_vdisk_id:source_vdisk_name:target_vdisk_id:target_vdisk_name:group_ id:group_name:status:progress:copy_rate:clean_progress:incremental 0:fcm-back-001:1:vd-prod-l2-0001:2:vd-back-l2-0001:1:fcg-back-001:idle_or_copied:0 :70:100:on 1:fcm-back-002:0:vd-prod-l2-0002:3:vd-back-l2-0002:1:fcg-back-001:idle_or_copied:0 :70:100:on IBM_2145:ITSOCL3:admin>
165
6.4.5 Preparing a FlashCopy mapping

We chose to prepare the FlashCopy Consistency Group (and thus the FlashCopy mappings associated with it) prior to issuing the start of the Group. This is done using the command svctask prestartfcconsistgrp, and the syntax is shown in Example 6-23
Example 6-23 .svctask prestartfcconsistgrp - syntax
>>- svctask -- -- prestartfcconsistgrp -- ----------------------> >--+- fc_consist_group_id ---+--------------------------------->< '- fc_consist_group_name -' We prepare the Group and immediately afterwards we issue the command to see that the state of the Consistency Group is Prepared, which is shown in Example 6-24.
Example 6-24 svctask prestartfcconsistgrp, svcinfo lsfcconsistgrp - prepare and view Group
IBM_2145:ITSOCL3:admin>svctask prestartfcconsistgrp fcg-back-001 IBM_2145:ITSOCL3:admin>svcinfo lsfcconsistgrp id name status 1 fcg-back-001 prepared IBM_2145:ITSOCL3:admin> This time we have chosen the Consistency Group name to specify which Group to start, not the id. We can also use the command again, to see the state of the mappings, shown in Example 6-25.
Example 6-25 svcinfo lsfcmap - both mappings in state: Prepared
IBM_2145:ITSOCL3:admin>svcinfo lsfcmap -delim : id:name:source_vdisk_id:source_vdisk_name:target_vdisk_id:target_vdisk_name:group_ id:group_name:status:progress:copy_rate:clean_progress:incremental 0:fcm-back-001:1:vd-prod-l2-0001:2:vd-back-l2-0001:1:fcg-back-001:prepared:0:70:10 0:on 1:fcm-back-002:0:vd-prod-l2-0002:3:vd-back-l2-0002:1:fcg-back-001:prepared:0:70:10 0:on IBM_2145:ITSOCL3:admin>
6.4.6 Starting a FlashCopy Consistency Group

Example 6-26 shows the syntax of the command svctask startfcconsistgrp.
Example 6-26 svctask startfcconsistgrp - syntax
>>- svctask -- -- startfcconsistgrp -- --+---------+-- ---------> '- -prep -' >--+- fc_consist_group_id ---+--------------------------------->< '- fc_consist_group_name -' We could prepare the Consistency Group first using the -prep option, but since we have already prepared the Group we do not need to do it now. Example 6-27 on page 167 shows how we start the Consistency Group (using the id to specify it) and immediately after we view the status of the Consistency Group. 166
Example 6-27 svctask startfcconsistgrp, svcinfo lsfcconsistgrp - start and view Group
IBM_2145:ITSOCL3:admin>svctask startfcconsistgrp 1 IBM_2145:ITSOCL3:admin>svcinfo lsfcconsistgrp id name status 1 fcg-back-001 copying IBM_2145:ITSOCL3:admin> The state of the Consistency Group is now Copying, as expected. In Example 6-28 we see the state of both of the mappings is Copying as well.
Example 6-28 svcinfo lsfcmap - both mappings in state: Copying
IBM_2145:ITSOCL3:admin>svcinfo lsfcmap -delim : id:name:source_vdisk_id:source_vdisk_name:target_vdisk_id:target_vdisk_name:group_ id:group_name:status:progress:copy_rate:clean_progress:incremental 0:fcm-back-001:1:vd-prod-l2-0001:2:vd-back-l2-0001:1:fcg-back-001:copying:34:70:10 0:on 1:fcm-back-002:0:vd-prod-l2-0002:3:vd-back-l2-0002:1:fcg-back-001:copying:34:70:10 0:on IBM_2145:ITSOCL3:admin> Example 6-29 shows the details of the FlashCopy mapping with the id 0.
Example 6-29 svcinfo lsfcmap -delim : 0
IBM_2145:ITSOCL3:admin>svcinfo lsfcmap -delim : 0 id:0 name:fcm-back-001 source_vdisk_id:1 source_vdisk_name:vd-prod-l2-0001 target_vdisk_id:2 target_vdisk_name:vd-back-l2-0001 group_id:1 group_name:fcg-back-001 status:copying progress:51 copy_rate:70 start_time:071107011320 dependent_mappings:0 autodelete:off clean_progress:100 clean_rate:50 incremental:on difference:48 grain_size:256 IO_group_id:0 IO_group_name:io_grp0 IBM_2145:ITSOCL3:admin> We see that the state of the mapping is Copying and that the progress is 51%, using a background copy priority of 70%. The last output we look at are the details of one of the VDisks involved, shown in Example 6-30 on page 168.
167
Example 6-30 svcinfo lsvdisk -delim : 1
IBM_2145:ITSOCL3:admin>svcinfo lsvdisk -delim : 1 id:1 name:vd-prod-l2-0001 IO_group_id:0 IO_group_name:io_grp0 status:online mdisk_grp_id:1 mdisk_grp_name:DS4700 capacity:25.0GB type:striped formatted:no mdisk_id: mdisk_name: FC_id:0 FC_name:fcm-back-001 RC_id:1 RC_name:MetroMirror Relationship-001 vdisk_UID:6005076801A100E90800000000000002 throttling:0 preferred_node_id:2 fast_write_state:empty cache:readwrite udid: fc_map_count:1 IBM_2145:ITSOCL3:admin> The VDisk vd-prod-l2-0001 is a member of both a Remote Copy Relationship and a FlashCopy mapping. After the background copy process has finished copying all the data to the target VDisk, the state changes to Idle_or_Copied, as shown in Example 6-31.
Example 6-31 svcinfo lsfcconsistgrp 1 - state: Idle_or_Copied
IBM_2145:ITSOCL3:admin>svcinfo lsfcconsistgrp 1 id 1 name fcg-back-001 status idle_or_copied FC_mapping_id 0 FC_mapping_name fcm-back-001 FC_mapping_id 1 FC_mapping_name fcm-back-002 IBM_2145:ITSOCL3:admin> We can also look at the details of one of the mappings, shown in Example 6-32.
Example 6-32 svcinfo lsfcmap -delim : 0
IBM_2145:ITSOCL3:admin>svcinfo lsfcmap -delim : 0 id:0 name:fcm-back-001 source_vdisk_id:1 source_vdisk_name:vd-prod-l2-0001 target_vdisk_id:2 target_vdisk_name:vd-back-l2-0001 168
group_id:1 group_name:fcg-back-001 status:idle_or_copied progress:100 copy_rate:70 start_time:071107011320 dependent_mappings:0 autodelete:off clean_progress:100 clean_rate:50 incremental:on difference:0 grain_size:256 IO_group_id:0 IO_group_name:io_grp0 IBM_2145:ITSOCL3:admin> Since the mappings are set up to be incremental, we can prepare and start the Consistency Group again to refresh it, and it would only copy the grains which have changed since the last FlashCopy process. This puts less load on the SVC and makes the full clone of the VDisks available faster.
6.4.7 Modify a mapping

To modify a mapping we use the command svctask chfcmap, which uses the syntax shown in Example 6-33.
Example 6-33 svctask chfcmap - syntax
>>- svctask -- -- chfcmap -- --+-------------------------+-- ---> '- -name -- new_name_arg -' >--+----------+-- ----------------------------------------------> '- -force -' >--+-----------------------------------------+-- ---------------> '- -consistgrp --+- consist_group_id ---+-' '- consist_group_name -' >--+------------------------+-- --+-------------------------+---> '- -copyrate -- percent-' '- -autodelete --+-on--+-' '-off-' >--+-----------------------------+--+- fc_map_id ---+---------->< '- -cleanrate ---- percent ---' '- fc_map_name -' We want to change both mappings to a lower background copy priority. Example 6-34 shows how we change the background copy priority to 50% and verify the settings for the mapping with the id 0. We do the same with the second mapping.
Example 6-34 svctask chfcmap -copyrate 50 0 - changing the copy priority to 50%
IBM_2145:ITSOCL3:admin>svctask chfcmap -copyrate 50 0 IBM_2145:ITSOCL3:admin>svcinfo lsfcmap 0 id 0
169
name fcm-back-001 source_vdisk_id 1 source_vdisk_name vd-prod-l2-0001 target_vdisk_id 2 target_vdisk_name vd-back-l2-0001 group_id 1 group_name fcg-back-001 status idle_or_copied progress 100 copy_rate 50 start_time 071107021634 dependent_mappings 0 autodelete off clean_progress 100 clean_rate 50 incremental on difference 0 grain_size 256 IO_group_id 0 IO_group_name io_grp0 IBM_2145:ITSOCL3:admin>
6.4.8 Stopping a mapping and a Consistency Group

Example 6-35 shows the syntax of the command svctask stopfcmap, which is used to stop a FlashCopy mapping.
Example 6-35 svctask stopfcmap - syntax
>>- svctask -- -- stopfcmap -- --+---------+-- -----------------> '- -force-' >--+- fc_map_id ---+------------------------------------------->< '- fc_map_name -' The Consistency Group can be stopped with the command svctask stopfcconsistgrp for which the syntax shown in Example 6-36.
Example 6-36 svctask stopfcconsistgrp - syntax
>>- svctask -- -- stopfcconsistgrp -- --+---------+-- ----------> '- -force-' >--+- fc_consist_group_id --+--------------------------------->< '- fc_consist_group_name -'
6.4.9 Deleting a mapping and a Consistency Group

Example 6-37 shows the syntax of the command svctask rmfcmap, which is used to delete a FlashCopy mapping.
Example 6-37 svctask rmfcmap - syntax
>>- svctask -- -- rmfcmap -- --+----------+-- ------------------> '- -force -' 170

>--+- fc_map_id ---+------------------------------------------->< '- fc_map_name -' Example 6-38 shows the syntax of the command svctask rmfcconsistgrp, used to delete a FlashCopy Consistency Group.
Example 6-38 svctask rmfcconsistgrp - syntax
>>- svctask -- -- rmfcconsistgrp -- --+----------+-- -----------> '- -force -' >--+- fc_consist_group_id ---+--------------------------------->< '- fc_consist_group_name -'
171
172
7574 Server integration.fm
Chapter 7.
Server integration
In this chapter, we discuss the steps that need to be taken to access copied VDisks from servers running various Operating Systems. In addition, we discuss some of the steps that need to be taken on the server to ensure that the copies that are generated will be easily accessed when needed. As far as servers are concerned, there is no difference between a VDisk which was copied using FlashCopy and one that was copied using Metro/Global Mirror. As a result, this chapter will focus on accessing the VDisks with no concern as to how they were created.
173
7.1 What data is copied?

When a VDisk is copied using SVCs Copy Services, a full block-by-block replica is created. This means that everything that is on the source/primary VDisk is copied to the target/secondary VDisk. For many systems, this means that it isnt just application data which is copied. Many systems write configuration data onto the disks to allow their volume managers to uniquely identify them and determine their place in volume groups. This is can be an advantage or a disadvantage. When presenting the copied VDisks to a brand new server, it make bringing the volumes back online much simpler. However, when presenting these VDisks to the same server which accesses the original VDisks, there are challenges involved, because the metadata on the VDisks are often expected to be unique and presenting the copied VDisk breaches that requirement. There is no way to selectively copy data from one VDisk to another; all data is copied. This chapter will discuss the techniques required to handle this fact.
7.2 What data is on the copied VDisk?

As discussed in Chapter 2, Planning for Advanced Copy Services implementation, the data which is on the original VDisk may not match what your applications and filesystems believe are on it. This is because of the caches between that layer and the SVC layer. It is important to ensure one of the following: That data in the caches above the SVC are fully flushed before initiating the copy, or That the systems that will access the copied VDisks will be able to recover from the fact that the data is not an identical copy of the systems view The second option can be done by using journalled filesystems and having applications that protect their actions with a transaction log. There is critical moment in a Copy Services mapping/relationships life cycle that affects the contents of the target/secondary VDisk. After this critical moment, the readable contents of the VDisk are no longer updated. This critical moment depends on which Copy Service is in use. For FlashCopy, the critical moment is when the mapping is started For Metro/Global Mirror, the critical moment is when the relationship is stopped (either manually or due to errors) In this chapter, the action is referred to as freezing and the moment is referred to as the freeze point. At the freeze point, the target/secondary VDisk will be an image of the source/primary VDisk at that moment in time (plus or minus a few seconds for Global Mirror). Any data in the caches above SVC will not be on the target/secondary VDisk. In the case of manual actions, it is relatively straightforward to ensure that the target/secondary VDisk is a useful copy. For Metro/Global Mirror relationships that are stopped due to SAN errors, this isnt possible, so its important that the systems accessing these VDisks are configured to allow recovery.
174
Some things are not copied. Persistent Reserves are made against a specific Logical Unit and are not copied with FlashCopy or Metro/Global Mirror.
7.3 AIX specifics

In this section, we describe the steps needed to create and access copied VDisks on AIX hosts. SVC supports the use of SDD and MPIO+SDDPCM to provide multipathing. When SDD is in use, AIX takes hdisks and creates vpaths. When SDDPCM is in use, AIX creates hdisks and each one has multiple paths. For the sake of clarity, the following section will assume the use of SDD and so talk about vpaths. The methods here apply just as well with SDDPCM. However, in those instances, the term hdisk should replace vpath.
7.3.1 Creating useful copies

Prior to the freeze point, steps must be taken to ensure that any information in the filesystem cache gets written to all of the relevant VDisks. The simplest way to do this is to unmount the filesystem, as in Example 7-1
Example 7-1 Unmounting a filesystem prior to making the break
#umount <source filesystem> If unmounting filesystems is not an option, an alternative is to force an update of the node table and a flush of buffered files. The AIX freeze/thaw functionality does precisely this. This is only available on JFS2 filesystems. Freezing a file system writes all dirty file system metadata and user data to the disk. While frozen, the file system is read-only, and anything that attempts to modify the file system or its contents must wait for the freeze to end. At this point, the break can be made and the resulting copy can be easily picked up by another server. Example 7-2 shows how this is done
Example 7-2 Showing a simple freeze/thaw process
#chfs -a freeze=60 <source filesystem> <freeze the VDisk(s)> #chfs -a freeze=0 <source filesystem> Of course, sometimes the freeze point occurs in an unplanned way, such as a data center outage. So long as the mapping/relationships are consistent, your filesystems should be recoverable and, in the case of a journalled filesystem, this recovery should not result in any data loss. Example 7-3 shows the initial configuration of the SVC cluster presenting VDisks to this AIX server. The two VDisks represent a data drive and a log drive.
Example 7-3 Cluster configuration prior to FlashCopying two VDisks
IBM_2145:ITSOCL2:admin>svcinfo lsvdisk -delim : id:name:IO_group_id:IO_group_name:status:mdisk_grp_id:mdisk_grp_name:capacity:type :FC_id:FC_name:RC_id:RC_name:vdisk_UID:fc_map_count

Chapter 7. Server integration
175
5:kanagaFS:0:io_grp0:online:1:DS4700:36.0GB:striped:::::6005076801AD80E8E000000000 000005:0 6:kanagaLog:0:io_grp0:online:0:DS4500:512.0MB:striped:::::6005076801AD80E8E0000000 00000006:0 IBM_2145:ITSOCL2:admin>svcinfo lshost KANAGA id 2 name KANAGA port_count 2 type generic mask 1111 iogrp_count 4 WWPN 10000000C932A7FB node_logged_in_count 2 state active WWPN 10000000C932A800 node_logged_in_count 0 state active Example 7-4 shows how these VDisks have been configured on an AIX server to hold a JFS2 filesystem. The vdisk_UID property of a VDisk matches the SERIAL property of a vpath. Thus you can use these two properties to determine which vpath represents which VDisk.
Example 7-4 AIX configuration prior to FlashCopying two VDisks
#datapath query device Total Devices : 2
DEV#: 0 DEVICE NAME: vpath0 TYPE: 2145 POLICY: Optimized SERIAL: 6005076801AD80E8E000000000000005 ========================================================================== Path# Adapter/Hard Disk State Mode Select Errors 0 fscsi0/hdisk3 OPEN NORMAL 209 0 1 fscsi0/hdisk5 OPEN NORMAL 220 0 2 fscsi0/hdisk7 OPEN NORMAL 0 0 3 fscsi0/hdisk9 OPEN NORMAL 0 0 DEV#: 1 DEVICE NAME: vpath1 TYPE: 2145 POLICY: Optimized SERIAL: 6005076801AD80E8E000000000000006 ========================================================================== Path# Adapter/Hard Disk State Mode Select Errors 0 fscsi0/hdisk4 OPEN NORMAL 0 0 1 fscsi0/hdisk6 OPEN NORMAL 0 0 2 fscsi0/hdisk8 OPEN NORMAL 74 0 3 fscsi0/hdisk10 OPEN NORMAL 79 0 #lspv hdisk0 hdisk1 hdisk2 hdisk3 hdisk4 hdisk5 hdisk6 176
0009cddaea97bf61 0009cdda43c9dfd5 0009cddabaef1d99 none none none none
rootvg rootvg rootvg None None None None
active active active
hdisk7 hdisk8 hdisk9 hdisk10 vpath0 vpath1 #lsvg -l st7s99 st7s99: LV NAME itsoLV itsoLog #lsfs Name Accounting /dev/hd4 /dev/hd1 /dev/hd2 /dev/hd9var /dev/hd3 /proc /dev/hd10opt /dev/itsoLV
none none none none 0009cdda1b5058e4 0009cdda1b55c48c
None None None None st7s99 st7s99
active active
TYPE jfs2 jfs2log
LPs 32 4
PPs 32 4
PVs 1 1
LV STATE open/syncd open/syncd
MOUNT POINT /itsoFS N/A
Nodename ---------
Mount Pt / /home /usr /var /tmp /proc /opt /itsoFS
VFS
Size
Options
Auto yes yes yes yes yes yes yes no no no no no no no no no
jfs2 196608 -jfs2 65536 -jfs2 19005440 -jfs2 65536 -jfs2 393216 -procfs --jfs2 2031616 -jfs2 4194304 rw
#mount node mounted -------- --------------/dev/hd4 /dev/hd2 /dev/hd9var /dev/hd3 /dev/hd1 /proc /dev/hd10opt /dev/itsoLV
mounted over --------------/ /usr /var /tmp /home /proc /opt /itsoFS
vfs -----jfs2 jfs2 jfs2 jfs2 jfs2 procfs jfs2 jfs2
date -----------Nov 07 09:55 Nov 07 09:55 Nov 07 09:55 Nov 07 09:55 Nov 07 09:56 Nov 07 09:56 Nov 07 09:56 Nov 07 11:17
options --------------rw,log=/dev/hd8 rw,log=/dev/hd8 rw,log=/dev/hd8 rw,log=/dev/hd8 rw,log=/dev/hd8 rw rw,log=/dev/hd8 rw,log=/dev/itsoLog
The Volume Group (VG) which is being copied is st7s99, containing vpath0 and vpath1. Is this is a JFS2 filesystem, we can freeze the filesystem prior to starting the FlashCopy. The applications that access this filesystem will see write requests delayed until the filesystem is thawed. With appropriate planning and scripting, this freeze can be kept to a few seconds at most. 1. Create the FlashCopy target VDisks IBM_2145:ITSOCL2:admin>svctask mkvdisk -vtype striped -size 36 -unit gb -mdiskgrp 1 -iogrp 0 -name kanagaFS-FC Virtual Disk, id [4], successfully created IBM_2145:ITSOCL2:admin>svctask mkvdisk -vtype striped -size 512 -unit mb -mdiskgrp 0 -iogrp 0 -name kanagaLog-FC Virtual Disk, id [3], successfully created 2. Create a FlashCopy consistency group IBM_2145:ITSOCL2:admin>svctask mkfcconsistgrp -name kanagaFC
177
FlashCopy Consistency Group, id [2], successfully created 3. Create FlashCopy mappings for the VDisks and add them to the new FlashCopy consistency group IBM_2145:ITSOCL2:admin>svctask mkfcmap -source kanagaFS -target kanagaFS-FC -consistgrp kanagaFC FlashCopy Mapping, id [0], successfully created IBM_2145:ITSOCL2:admin>svctask mkfcmap -source kanagaLog -target kanagaLog-FC -consistgrp kanagaFC FlashCopy Mapping, id [1], successfully created 4. Prepare the FlashCopy consistency group. IBM_2145:ITSOCL2:admin>svctask prestartfcconsistgrp kanagaFC 5. Wait until the consistency group is in the prepared state. IBM_2145:ITSOCL2:admin>svcinfo lsfcconsistgrp id name status 2 kanagaFC preparing . . . IBM_2145:ITSOCL2:admin>svcinfo lsfcconsistgrp id name status 2 kanagaFC prepared 6. Freeze the filesystem on the AIX server. #chfs -a freeze=60 /itsoFS 7. Start the FlashCopy consistency group. This is the freeze point. IBM_2145:ITSOCL2:admin>svctask startfcconsistgrp kanagaFC 8. Thaw the filesystem on the AIX server. #chfs -a freeze=0 /itsoFS At this point, the FlashCopy operation has been started and VDisks 3 and 4 are exact copies of VDisks 5 and 6 at the time of the filesystem freeze.
7.3.2 Accessing useful copies

If the source/primary VDisk is defined to the AIX Logical Volume Manager (LVM) then all of its data structures and identifiers are copied to the target/secondary VDisk, as well. This includes the Volume Group Descriptor Area (VGDA), which contains the Physical Volume Identifier (PVID) and Volume Group Identifier (VGID). For AIX LVM, it is currently not possible to activate a volume group with a physical volume (vpath) that contains a VGID and a PVID that is already used in a volume group existing on the same server. The restriction still applies even if the vpath PVID is cleared and reassigned with the two commands listed in Example 7-5.
Example 7-5 Clearing PVIDs chdev -l <vpath#> -a pv=clear chdev -l <vpath#> -a pv=yes
Therefore, it is necessary to redefine the volume group information on the FlashCopy target VDisk using special procedures or the recreatevg command. This will alter the PVIDs and 178
VGIDs in all the VGDAs of the FlashCopy target VDisks, so that there are no conflicts with existing PVIDs and VGIDs on existing volume groups that reside on the source VDisks. If you do not redefine the volume group information prior to importing the volume group, then the importvg command will fail.
Accessing target/secondary VDisks from the same AIX host

In this section, we describe a method of accessing the FlashCopy target VDisks on a single AIX host while the source VDisks are still active on the same server. The procedure is intended to be used as a guide and may not cover all scenarios.
AIX recreatevg command

Copying the source volume's content using FlashCopy causes all of the data structures and identifiers used by AIX's Logical Volume Manager to be duplicated to the target volume. The duplicate definitions (PVID and VGID) in turn cause conflicts within LVM. This problem is solved by using the AIX command recreatevg. The recreatevg command is officially available in all levels of AIX that SVC supports The recreatevg command overcomes the problem of duplicated LVM data structures and identifiers caused by a disk duplication process such as FlashCopy. It is used to recreate an AIX Volume Group (VG) on a set of target volumes that are copied from a set of source volumes belonging to a specific VG. The command will allocate new physical volume identifiers (PVIDs) for the member disks and a new volume group identifier (VGID) to the volume group. The command also provides options to rename the logical volumes with a prefix you specify, and options to rename labels to specify different mount points for file systems. To use this command, you must have root user authority. Attention: Running recreatevg will change the PVID on the VDisks. In the case of a Metro/Global Mirror relationship, if you decide to reverse the direction of the relationship, you will change the PVID of the original primary VDisk during the synchronization process. The steps below assume that the AIX server and SVC cluster are in the state following the process in the previous section. Perform these tasks to access the FlashCopy target VDisks: 1. Map the FlashCopy target VDisks to the host IBM_2145:ITSOCL2:admin>svctask mkvdiskhostmap -host KANAGA kanagaFS-FC Virtual Disk to Host map, id [2], successfully created IBM_2145:ITSOCL2:admin>svctask mkvdiskhostmap -host KANAGA kanagaLog-FC Virtual Disk to Host map, id [3], successfully created 2. Run Configuration Manager to pick up the new devices on the AIX server #cfgmgr -vl fscsi0 (skipped the debug output from this command) 3. Run Configuration Manager to create vpaths with the new hdisks #cfgmgr -vl dpo At this point, the configuration of the AIX server has been changed. Example 7-6 shows the new status.
Example 7-6 AIX configuration prior to FlashCopying two VDisks
#datapath query device
179
Total Devices : 4
DEV#: 0 DEVICE NAME: vpath0 TYPE: 2145 POLICY: Optimized SERIAL: 6005076801AD80E8E000000000000005 ========================================================================== Path# Adapter/Hard Disk State Mode Select Errors 0 fscsi0/hdisk3 OPEN NORMAL 214 0 1 fscsi0/hdisk5 OPEN NORMAL 232 0 2 fscsi0/hdisk7 OPEN NORMAL 0 0 3 fscsi0/hdisk9 OPEN NORMAL 0 0 DEV#: 1 DEVICE NAME: vpath1 TYPE: 2145 POLICY: Optimized SERIAL: 6005076801AD80E8E000000000000006 ========================================================================== Path# Adapter/Hard Disk State Mode Select Errors 0 fscsi0/hdisk4 OPEN NORMAL 0 0 1 fscsi0/hdisk6 OPEN NORMAL 0 0 2 fscsi0/hdisk8 OPEN NORMAL 91 0 3 fscsi0/hdisk10 OPEN NORMAL 102 0 DEV#: 2 DEVICE NAME: vpath2 TYPE: 2145 POLICY: Optimized SERIAL: 6005076801AD80E8E000000000000007 ========================================================================== Path# Adapter/Hard Disk State Mode Select Errors 0 fscsi0/hdisk11 CLOSE NORMAL 0 0 1 fscsi0/hdisk13 CLOSE NORMAL 0 0 2 fscsi0/hdisk15 CLOSE NORMAL 0 0 3 fscsi0/hdisk17 CLOSE NORMAL 0 0 DEV#: 3 DEVICE NAME: vpath3 TYPE: 2145 POLICY: Optimized SERIAL: 6005076801AD80E8E000000000000009 ========================================================================== Path# Adapter/Hard Disk State Mode Select Errors 0 fscsi0/hdisk12 CLOSE NORMAL 0 0 1 fscsi0/hdisk14 CLOSE NORMAL 0 0 2 fscsi0/hdisk16 CLOSE NORMAL 0 0 3 fscsi0/hdisk18 CLOSE NORMAL 0 0 #lspv hdisk0 hdisk1 hdisk2 hdisk3 hdisk4 hdisk5 hdisk6 hdisk7 hdisk8 hdisk9 hdisk10 vpath0 vpath1 hdisk11
0009cddaea97bf61 0009cdda43c9dfd5 0009cddabaef1d99 none none none none none none none none 0009cdda1b5058e4 0009cdda1b55c48c 0009cdda1b5058e4
rootvg rootvg rootvg None None None None None None None None st7s99 st7s99 st7s99
180
hdisk12 hdisk13 hdisk14 hdisk15 hdisk16 hdisk17 hdisk18 vpath2 vpath3
0009cdda1b55c48c 0009cdda1b5058e4 0009cdda1b55c48c 0009cdda1b5058e4 0009cdda1b55c48c 0009cdda1b5058e4 0009cdda1b55c48c none none
st7s99 st7s99 st7s99 st7s99 st7s99 st7s99 st7s99 None None
active active active active active active active
The new vpaths, vpath2 and vpath3 appear with no PVID whereas the hdisks that make up those vpaths do. This is as expected. Note also that vpath2 and vpath3 are in a CLOSE state, because nothing is currently accessing them. The following steps will recreate the VG with a new name and recover the Logical Volumes (LV) 1. Create the new VG and prefix all file system path names with /fc and prefix all AIX LVs with fc:
#recreatevg -y st7s99fc -L /fc -Y FC vpath2 vpath3 st7s99fc
Restriction: The LV prefix that is given via the -Y parameter cannot be fc. The reason for this is that fc is already defined as a prefix in the PdDv class of the Device Configuration Database and so is not a valid LV prefix. 2. Run fsck on the new filesystem. Strictly speaking this is not necessary as the steps taken while creating the copy ensure that the filesystem is client, but it is a prudent defensive step. #fsck -y /fc/itsoFS
The current volume is: /dev/FCitsoLV Primary superblock is valid. J2_LOGREDO:log redo processing for /dev/FCitsoLV Primary superblock is valid. *** Phase 1 - Initial inode scan *** Phase 2 - Process remaining directories *** Phase 3 - Process remaining files *** Phase 4 - Check and repair inode allocation map *** Phase 5 - Check and repair block allocation map File system is clean. 3. Mount the new filesystem #mount /fc/itsoFS Replaying log for /dev/FCitsoLV. At this point, the FlashCopy target VDisks are ready for access. Example 7-7 shows the final server state. This new filesystem contains all of the files that were in the original filesystem at the time of the freeze.
Example 7-7 AIX configuration after recreating the VG
#datapath query device
181
Total Devices : 4
DEV#: 0 DEVICE NAME: vpath0 TYPE: 2145 POLICY: Optimized SERIAL: 6005076801AD80E8E000000000000005 ========================================================================== Path# Adapter/Hard Disk State Mode Select Errors 0 fscsi0/hdisk3 OPEN NORMAL 217 0 1 fscsi0/hdisk5 OPEN NORMAL 237 0 2 fscsi0/hdisk7 OPEN NORMAL 0 0 3 fscsi0/hdisk9 OPEN NORMAL 0 0 DEV#: 1 DEVICE NAME: vpath1 TYPE: 2145 POLICY: Optimized SERIAL: 6005076801AD80E8E000000000000006 ========================================================================== Path# Adapter/Hard Disk State Mode Select Errors 0 fscsi0/hdisk4 OPEN NORMAL 0 0 1 fscsi0/hdisk6 OPEN NORMAL 0 0 2 fscsi0/hdisk8 OPEN NORMAL 98 0 3 fscsi0/hdisk10 OPEN NORMAL 109 0 DEV#: 2 DEVICE NAME: vpath2 TYPE: 2145 POLICY: Optimized SERIAL: 6005076801AD80E8E000000000000007 ========================================================================== Path# Adapter/Hard Disk State Mode Select Errors 0 fscsi0/hdisk11 OPEN NORMAL 202 0 1 fscsi0/hdisk13 OPEN NORMAL 206 0 2 fscsi0/hdisk15 OPEN NORMAL 0 0 3 fscsi0/hdisk17 OPEN NORMAL 0 0 DEV#: 3 DEVICE NAME: vpath3 TYPE: 2145 POLICY: Optimized SERIAL: 6005076801AD80E8E000000000000009 ========================================================================== Path# Adapter/Hard Disk State Mode Select Errors 0 fscsi0/hdisk12 OPEN NORMAL 0 0 1 fscsi0/hdisk14 OPEN NORMAL 0 0 2 fscsi0/hdisk16 OPEN NORMAL 61 0 3 fscsi0/hdisk18 OPEN NORMAL 57 0 #lspv hdisk0 hdisk1 hdisk2 hdisk3 hdisk4 hdisk5 hdisk6 hdisk7 hdisk8 hdisk9 hdisk10 vpath0 vpath1 hdisk11
0009cddaea97bf61 0009cdda43c9dfd5 0009cddabaef1d99 none none none none none none none none 0009cdda1b5058e4 0009cdda1b55c48c none
rootvg rootvg rootvg None None None None None None None None st7s99 st7s99 None
active active
182
hdisk12 hdisk13 hdisk14 hdisk15 hdisk16 hdisk17 hdisk18 vpath2 vpath3 #lsvg rootvg st7s99 st7s99fc #lsvg -l st7s99 st7s99: LV NAME itsoLV itsoLog
none none none none none none none 0009cdda1bf46d57 0009cdda1bf46f3b
None None None None None None None st7s99fc st7s99fc
active active
TYPE jfs2 jfs2log
LPs 32 4
PPs 32 4
PVs 1 1
#lsvg -l st7s99fc st7299fc: LV NAME FCitsoLV FCitsoLog #lsfs Name Accounting /dev/hd4 /dev/hd1 /dev/hd2 /dev/hd9var /dev/hd3 /proc /dev/hd10opt /dev/itsoLV /dev/FCitsoLV
TYPE jfs2 jfs2log
LPs 32 4
PPs 32 4
PVs 1 1
MOUNT POINT /fc/itsoFS N/A
Nodename ----------
Mount Pt / /home /usr /var /tmp /proc /opt /itsoFS /fc/itsoFS
VFS
Size
Options
Auto yes yes yes yes yes yes yes no no no no no no no no no no no
jfs2 196608 -jfs2 65536 -jfs2 19005440 -jfs2 65536 -jfs2 393216 -procfs --jfs2 2031616 -jfs2 4194304 rw jfs2 4194304 rw
#mount node mounted ------ --------------/dev/hd4 /dev/hd2 /dev/hd9var /dev/hd3 /dev/hd1 /proc /dev/hd10opt /dev/itsoLV /dev/FCitsoLV
mounted over --------------/ /usr /var /tmp /home /proc /opt /itsoFS /fc/itsoFS
vfs -----jfs2 jfs2 jfs2 jfs2 jfs2 procfs jfs2 jfs2 jfs2
date -----------Nov 07 09:55 Nov 07 09:55 Nov 07 09:55 Nov 07 09:55 Nov 07 09:56 Nov 07 09:56 Nov 07 09:56 Nov 07 11:17 Nov 07 13:15
options --------------rw,log=/dev/hd8 rw,log=/dev/hd8 rw,log=/dev/hd8 rw,log=/dev/hd8 rw,log=/dev/hd8 rw rw,log=/dev/hd8 rw,log=/dev/itsoLog rw,log=/dev/FCitsoLog
183
Accessing target/secondary VDisks from a different AIX host

Accessing target/secondary VDisks from a different server to the one accessing the source/primary VDisks is much simpler. The following procedure makes the data of the FlashCopy target VDisk available to another AIX host that has no prior definitions of the target VDisk in its configuration database (ODM). Example 7-8 on page 184 shows the state of the AIX server, prior to presenting the target VDisks.
Example 7-8 AIX configuration prior presenting FlashCopy target VDisks
#datapath query device No device file found #lsvg rootvg #lspv hdisk0 hdisk1 hdisk2 #lsfs Name Accounting /dev/hd4 /dev/hd1 /dev/hd2 /dev/hd9var /dev/hd3 /proc /dev/hd10opt
0009cddaea97bf61 0009cdda43c9dfd5 0009cddabaef1d99
rootvg rootvg rootvg
Nodename --------
Mount Pt / /home /usr /var /tmp /proc /opt
VFS
Size
Options
Auto yes yes yes yes yes yes yes no no no no no no no
jfs2 196608 -jfs2 65536 -jfs2 19005440 -jfs2 65536 -jfs2 393216 -procfs --jfs2 2031616 --
The initial process for accessing the VDisks is as before. 1. Map the FlashCopy target VDisks to the host IBM_2145:ITSOCL2:admin>svctask mkvdiskhostmap -host UMNAK kanagaFS-FC Virtual Disk to Host map, id [2], successfully created IBM_2145:ITSOCL2:admin>svctask mkvdiskhostmap -host UMNAK kanagaLog-FC Virtual Disk to Host map, id [3], successfully created 2. Run Configuration Manager to pick up the new devices on the AIX server #cfgmgr -vl fscsi0 (skipped the debug output from this command) 3. Run Configuration Manager to create vpaths with the new hdisks #cfgmgr -vl dpo At this point, the configuration of the AIX server has been changed. Example 7-9 shows the new status.
Example 7-9 AIX configuration after presenting FlashCopy target VDisks
datapath query device
184
Total Devices : 2
DEV#: 0 DEVICE NAME: vpath2 TYPE: 2145 POLICY: Optimized SERIAL: 6005076801AD80E8E000000000000007 ========================================================================== Path# Adapter/Hard Disk State Mode Select Errors 0 fscsi0/hdisk3 CLOSE NORMAL 0 0 1 fscsi0/hdisk5 CLOSE NORMAL 0 0 2 fscsi0/hdisk7 CLOSE NORMAL 0 0 3 fscsi0/hdisk9 CLOSE NORMAL 0 0 DEV#: 1 DEVICE NAME: vpath3 TYPE: 2145 POLICY: Optimized SERIAL: 6005076801AD80E8E000000000000009 ========================================================================== Path# Adapter/Hard Disk State Mode Select Errors 0 fscsi0/hdisk4 CLOSE NORMAL 0 0 1 fscsi0/hdisk6 CLOSE NORMAL 0 0 2 fscsi0/hdisk8 CLOSE NORMAL 0 0 3 fscsi0/hdisk10 CLOSE NORMAL 0 0 #lspv hdisk0 hdisk1 hdisk2 hdisk3 hdisk4 hdisk5 hdisk6 hdisk7 hdisk8 hdisk9 hdisk10 vpath2 vpath3
0009cddaea97bf61 0009cdda43c9dfd5 0009cddabaef1d99 none none none none none none none none 0009cdda1b5058e4 0009cdda1b55c48c
rootvg rootvg rootvg None None None None None None None None st7s99 st7s99
Note that, this time, its the new vpaths that have PVIDs and the hdisks have none. We use the importvg command to import the information about the VG into this servers ODM: 1. Import the target volume group:
#importvg -y st7s99 vpath2 st7s99
2. Vary on the Volume Group (the importvg command should varyon the volume group):
#varyonvg st7s99
3. Verify consistency of all file systems on the FlashCopy target VDisk:

#fsck -y /itsoFS
The current volume is: /dev/itsoLV Primary superblock is valid. J2_LOGREDO:log redo processing for /dev/itsoLV Primary superblock is valid. *** Phase 1 - Initial inode scan
185
*** Phase 2 *** Phase 3 *** Phase 4 *** Phase 5 File system
- Process remaining directories - Process remaining files - Check and repair inode allocation map - Check and repair block allocation map is clean.
4. Mount all the target file systems:

#mount /itsoFS
At this point, the FlashCopy target VDisks are ready for access. Example 7-10 shows the final server state. This new filesystem contains all of the files that were in the original filesystem at the time of the freeze.
Example 7-10 AIX configuration after importing the VG
#datapath query device Total Devices : 2
DEV#: 0 DEVICE NAME: vpath2 TYPE: 2145 POLICY: Optimized SERIAL: 6005076801AD80E8E000000000000007 ========================================================================== Path# Adapter/Hard Disk State Mode Select Errors 0 fscsi0/hdisk3 OPEN NORMAL 150 0 1 fscsi0/hdisk5 OPEN NORMAL 161 0 2 fscsi0/hdisk7 OPEN NORMAL 0 0 3 fscsi0/hdisk9 OPEN NORMAL 0 0 DEV#: 1 DEVICE NAME: vpath3 TYPE: 2145 POLICY: Optimized SERIAL: 6005076801AD80E8E000000000000009 ========================================================================== Path# Adapter/Hard Disk State Mode Select Errors 0 fscsi0/hdisk4 OPEN NORMAL 0 0 1 fscsi0/hdisk6 OPEN NORMAL 0 0 2 fscsi0/hdisk8 OPEN NORMAL 40 0 3 fscsi0/hdisk10 OPEN NORMAL 44 0 #lspv hdisk0 hdisk1 hdisk2 hdisk3 hdisk4 hdisk5 hdisk6 hdisk7 hdisk8 hdisk9 hdisk10 vpath2 vpath3 #lsvg rootvg st7s99
0009cddaea97bf61 0009cdda43c9dfd5 0009cddabaef1d99 none none none none none none none none 0009cdda1b5058e4 0009cdda1b55c48c
rootvg rootvg rootvg None None None None None None None None st7s99 st7s99
active active
186
#lsvg -l st7s99 st7s99: LV NAME itsoLV itsoLog #lsfs Name Accounting /dev/hd4 /dev/hd1 /dev/hd2 /dev/hd9var /dev/hd3 /proc /dev/hd10opt /dev/itsoLV
TYPE jfs2 jfs2log
LPs 32 4
PPs 32 4
PVs 1 1
Nodename ---------
Mount Pt / /home /usr /var /tmp /proc /opt /itsoFS
VFS
Size
Options
Auto yes yes yes yes yes yes yes no no no no no no no no no
jfs2 196608 -jfs2 65536 -jfs2 19005440 -jfs2 65536 -jfs2 393216 -procfs --jfs2 2031616 -jfs2 4194304 rw
#mount node mounted -------- --------------/dev/hd4 /dev/hd2 /dev/hd9var /dev/hd3 /dev/hd1 /proc /dev/hd10opt /dev/itsoLV
mounted over --------------/ /usr /var /tmp /home /proc /opt /itsoFS
vfs -----jfs2 jfs2 jfs2 jfs2 jfs2 procfs jfs2 jfs2
date -----------Nov 07 14:04 Nov 07 14:04 Nov 07 14:04 Nov 07 14:04 Nov 07 14:05 Nov 07 14:05 Nov 07 14:05 Nov 07 14:36
options --------------rw,log=/dev/hd8 rw,log=/dev/hd8 rw,log=/dev/hd8 rw,log=/dev/hd8 rw,log=/dev/hd8 rw rw,log=/dev/hd8 rw,log=/dev/itsoLog
187
7.4 Windows 2000/2003 and Copy Services

Windows 2000 and Windows 2003 incorporate a stripped-down version of the VERITAS Volume Manager, called the Logical Disk Manager (LDM) for handling their disks. With the LDM, you are able to create logical partitions, perform disk mounts, and create dynamic volumes. There are five types of dynamic volumes: simple, spanned, mirrored, striped, and RAID-5. On earlier versions of Windows, the information relating to the disks was stored in the Windows registry. With Windows 2000/2003, this information is stored on the disk drive itself in a partition called the LDM database, which is kept on the last few tracks of the disk. Each volume has its own 128-bit Globally Unique Identifier (GUID) and belongs to a disk group. This is similar to the concept of Physical Volume Identifier (PVID) and Volume Group in AIX. As the LDM is stored on the physical drive itself, with Windows 2000/2003 it is possible to move disk drives between different computers.
Copy Services limitations with Windows 2000/2003

It is not possible to have FlashCopy source and target VDisks mapped t0 the same Windows host when they were created as dynamic volumes. The reason is that each dynamic volume has to have its own 128-bit GUID. As its name implies, the GUID must be unique on one system. When you perform FlashCopy, the GUID gets copied as well, so this means that if you tried to mount the source and target volume on the same host system, you would have two volumes with exactly the same GUID. This is not allowed, and you will not be able to mount the target volume.

As with AIX, the way to create useful copies is to ensure that there is no data in the system write cache when you freeze the VDisks. The methods for doing this are conceptually similar. Both of these methods can be done by accessing the Computer Management console on your Windows host. One option is to remove the drive letter associated with the disk. Figure 7-1 on page 189 shows a VDisk which is currently being used by a Windows server. Selecting Change Drive Letter and Paths... opens up the dialog box shown in Figure 7-2 on page 189. Doing this will result in all outstanding writes to the disk to be completed. Like unmounting a filesystem in AIX, no applications will be able to access data on this drive, without the drive letter.
188
Figure 7-1 Choosing to change the Drive Letter of a Windows Disk
Figure 7-2 Removing a Drive Letter from a Windows Disk
An alternative method is to disable the write cache for the disk in question. Figure 7-3 shows how to access the Properties of a Windows Disk. This will bring up a dialog box shown in Figure 7-4 on page 190.
189
Figure 7-3 Setting the properties of a Windows Disk
By selecting Optimize for quick removal, the write cache is disabled, so you can make the break and be sure that the VDisk contents matches the image that the host has.
Figure 7-4 Disabling the write cache of a Windows Disk
190
Example 7-11 shows the initial configuration of the SVC cluster presenting VDisks to this Windows server. The two VDisks represent a data drive and a log drive.
Example 7-11 Cluster configuration prior to FlashCopying two VDisks
IBM_2145:ITSOCL2:admin>svcinfo lsvdisk -delim : id:name:IO_group_id:IO_group_name:status:mdisk_grp_id:mdisk_grp_name:capacity:type :FC_id:FC_name:RC_id:RC_name:vdisk_UID:fc_map_count 4:senegalDynamic2:0:io_grp0:online:1:DS4700:36.0GB:striped:2:fcmap2:::6005076801AD 80E8E00000000000000C:1 5:senegalDynamic1:0:io_grp0:online:1:DS4700:36.0GB:striped:1:fcmap1:::6005076801AD 80E8E00000000000000B:1 6:senegalBasic:0:io_grp0:online:1:DS4700:36.0GB:striped:0:fcmap0:::6005076801AD80E 8E00000000000000A:1 IBM_2145:ITSOCL2:admin>svcinfo lshost SENEGAL id 1 name SENEGAL port_count 2 type generic mask 1111 iogrp_count 4 WWPN 210000E08B89B9C0 node_logged_in_count 2 state active WWPN 210000E08B89CCC2 node_logged_in_count 0 state active Example 7-12 shows the SDD configuration of the Windows 2003 server to which these VDisks are mapped. Figure 7-5 shows how these VDisks have been configured on a Windows 2003 server.
Example 7-12 SDD Configuration of Windows 2003 server
C:\Program Files\IBM\Subsystem Device Driver>datapath query device Total Devices : 3
DEV#: 0 DEVICE NAME: Disk1 Part0 TYPE: 2145 POLICY: OPTIMIZED SERIAL: 6005076801AD80E8E00000000000000C ============================================================================ Path# Adapter/Hard Disk State Mode Select Errors 0 Scsi Port3 Bus0/Disk1 Part0 OPEN NORMAL 8 0 1 Scsi Port3 Bus0/Disk1 Part0 OPEN NORMAL 10 0 2 Scsi Port3 Bus0/Disk1 Part0 OPEN NORMAL 0 0 3 Scsi Port3 Bus0/Disk1 Part0 OPEN NORMAL 0 0 4 Scsi Port3 Bus0/Disk1 Part0 OPEN NORMAL 0 0 5 Scsi Port3 Bus0/Disk1 Part0 OPEN NORMAL 0 0 DEV#: 1 DEVICE NAME: Disk2 Part0 TYPE: 2145 POLICY: OPTIMIZED SERIAL: 6005076801AD80E8E00000000000000B ============================================================================ Path# Adapter/Hard Disk State Mode Select Errors 0 Scsi Port3 Bus0/Disk2 Part0 OPEN NORMAL 0 0
191
1 2 3 4 5
Scsi Scsi Scsi Scsi Scsi
Port3 Port3 Port3 Port3 Port3
Bus0/Disk2 Bus0/Disk2 Bus0/Disk2 Bus0/Disk2 Bus0/Disk2
Part0 Part0 Part0 Part0 Part0
OPEN OPEN OPEN OPEN OPEN
NORMAL NORMAL NORMAL NORMAL NORMAL
0 3 5 5 5
0 0 0 0 0
DEV#: 2 DEVICE NAME: Disk3 Part0 TYPE: 2145 POLICY: OPTIMIZED SERIAL: 6005076801AD80E8E00000000000000A ============================================================================ Path# Adapter/Hard Disk State Mode Select Errors 0 Scsi Port3 Bus0/Disk3 Part0 OPEN NORMAL 11 0 1 Scsi Port3 Bus0/Disk3 Part0 OPEN NORMAL 9 0 2 Scsi Port3 Bus0/Disk3 Part0 OPEN NORMAL 0 0 3 Scsi Port3 Bus0/Disk3 Part0 OPEN NORMAL 0 0 4 Scsi Port3 Bus0/Disk3 Part0 OPEN NORMAL 0 0 5 Scsi Port3 Bus0/Disk3 Part0 OPEN NORMAL 0 0
Figure 7-5 Showing how VDisks appear on a Windows 2003 host, prior to mapping target/secondary VDisks
192
Figure 7-6 Identifying a Windows Disk as a VDisk
The vdisk_UID property of a VDisk matches the SERIAL property of a vpath. Thus you can use these two properties to determine which vpath represents which VDisk. The Device Name of the vpath matches the Disk property shown under the Volumes tab as shown in Figure 7-6. Thus you can use these two properties to determine which Windows Disk represents which vpath. Subsequently, you can match a Windows Disk to a VDisk. The process below shows how to make sure that the FlashCopy target VDisks contain useful copies 1. Create the FlashCopy target VDisks IBM_2145:ITSOCL2:admin>svctask mkvdisk -vtype striped -unit gb -size 36 -iogrp 0 -mdiskgrp DS4700 -name senegalBasic-FC Virtual Disk, id [3], successfully created IBM_2145:ITSOCL2:admin>svctask mkvdisk -vtype striped -unit gb -size 36 -iogrp 0 -mdiskgrp DS4700 -name senegalDyn1-FC Virtual Disk, id [8], successfully created IBM_2145:ITSOCL2:admin>svctask mkvdisk -vtype striped -unit gb -size 36 -iogrp 0 -mdiskgrp DS4700 -name senegalDyn2-FC Virtual Disk, id [7], successfully created 2. Create a FlashCopy consistency group IBM_2145:ITSOCL2:admin>svctask mkfcconsistgrp -name senegalFC FlashCopy Consistency Group, id [1], successfully created 3. Create FlashCopy mappings for the VDisks and add them to the new FlashCopy consistency group IBM_2145:ITSOCL2:admin>svctask mkfcmap -source senegalBasic -target senegalBasic-FC -consistgrp senegalFC FlashCopy Mapping, id [0], successfully created
193
IBM_2145:ITSOCL2:admin>svctask mkfcmap -source senegalDynamic1 -target senegalDyn1-FC -consistgrp senegalFC FlashCopy Mapping, id [1], successfully created IBM_2145:ITSOCL2:admin>svctask mkfcmap -source senegalDynamic2 -target senegalDyn2-FC -consistgrp senegalFC FlashCopy Mapping, id [2], successfully created 4. Prepare the FlashCopy consistency group IBM_2145:ITSOCL2:admin>svctask prestartfcconsistgrp senegalFC 5. Wait until the consistency group is in the prepared state IBM_2145:ITSOCL2:admin>svcinfo lsfcconsistgrp id name status 1 senegalFC preparing . . . IBM_2145:ITSOCL2:admin>svcinfo lsfcconsistgrp id name status 1 senegalFC prepared 6. Disable the write cache for the Windows Disks as shown in Figure 7-4 on page 190 7. Start the FlashCopy consistency group. This is the freeze point. IBM_2145:ITSOCL2:admin>svctask startfcconsistgrp senegalFC 8. Re-enable the write cache for the Windows Disks At this point, the FlashCopy operation has been started and VDisks 3, 8 and 7 are exact copies of VDisks 6, 5 and 4 at the time of the FlashCopy consistency group being started.

Basic disks relatively straightforward to access from a Windows 2000/2003 host. Dynamic disks are a little more complicated because of the restrictions imposed on duplicate GUIDs.
Accessing target/secondary VDisks from the same Windows host

In this section, we describe a method of accessing the FlashCopy target VDisks on a single Windows host while the source VDisks are still active on the same server. The steps below assume that the Windows server and SVC cluster are in the state following the process in the previous section. Perform these tasks to access the FlashCopy target VDisks: 1. Map the FlashCopy target VDisks to the host IBM_2145:ITSOCL2:admin>svctask mkvdiskhostmap -host SENEGAL senegalBasic-FC Virtual Disk to Host map, id [3], successfully created IBM_2145:ITSOCL2:admin>svctask mkvdiskhostmap -host SENEGAL senegalDyn1-FC Virtual Disk to Host map, id [4], successfully created IBM_2145:ITSOCL2:admin>svctask mkvdiskhostmap -host SENEGAL senegalDyn2-FC Virtual Disk to Host map, id [5], successfully created 2. Scan for new Hardware Changes as shown in Figure 7-7 on page 195 to pick up the newly presented VDisks
194
Figure 7-7 Scanning for hardware changes to pick up target/secondary VDisks
At this point, the configuration of the Windows host has been changed. Figure 7-8 on page 196 shows the new status. The new Disks, Disk 4 and Disk 6 appear as unallocated. These are the Dynamic Disks (as is reported by the LDM). Disk 5, the Basic Disk has already been recognized as one. All that needs to be done is to assign it a drive letter as show in Appendix 7-1, Choosing to change the Drive Letter of a Windows Disk on page 189. Clicking on Add brings up the dialog box as shown in Figure 7-9 on page 197 Once a drive letter has been assigned to your Basic Drives, you should run chkdsk to ensure filesystem consistency as shown in Example 7-13. This is particularly important if the break was made due to an unplanned Metro/Global Mirror stoppage.
Example 7-13 Running chkdsk on a newly mapped target/secondary VDisk
C:\Program Files\IBM\Subsystem Device Driver>chkdsk /f e: The type of the file system is NTFS. Volume label is basicVol. CHKDSK is verifying files (stage 1 of 3)... File verification completed. CHKDSK is verifying indexes (stage 2 of 3)... Index verification completed. CHKDSK is verifying security descriptors (stage 3 of 3)... Security descriptor verification completed. Windows has checked the file system and found no problems.
195
37744685 20 4 0 67137 65536 37677524
KB KB KB KB KB KB KB
total disk space. in 2 files. in 9 indexes. in bad sectors. in use by the system. occupied by the log file. available on disk.
4096 bytes in each allocation unit. 9436171 total allocation units on disk. 9419381 allocation units available on disk. Target/secondary VDisks which are Dynamic Disks should never be presented to a host which has visibility of the source/primary VDisk as there is a risk of data loss.
Figure 7-8 Windows configuration after discovering target/secondary VDisks
196
Figure 7-9 Adding a drive letter to a Basic Drive
Accessing taget/secondary VDisks from the same Windows host

Accessing target/secondary VDisks from a different Windows server to the one accessing the source/primary VDisks is much simpler. In the following example, server has no VDisks mapped to it, to begin with. In Example 7-14, the VDisks are mapped to the server
Example 7-14 Mapping FlashCopy targets to a new Windows Server
IBM_2145:ITSOCL2:admin>svctask mkvdiskhostmap -host SIAM senegalBasic-FC Virtual Disk to Host map, id [2], successfully created IBM_2145:ITSOCL2:admin>svctask mkvdiskhostmap -host SIAM senegalDyn1-FC Virtual Disk to Host map, id [3], successfully created IBM_2145:ITSOCL2:admin>svctask mkvdiskhostmap -host SIAM senegalDyn2-FC Virtual Disk to Host map, id [4], successfully created As in Figure 7-7 on page 195, the next step is to scan for hardware changes. The result of this is shown in Figure 7-10. As is shown, the Basic Disk is picked up normally and just needs a Drive Letter assigned to it. Figure 7-10 on page 198 also shows the state of the Dynamic Disks. They register as Foreign. By selecting Import Foreign Disks..., you will see the dialog box shown in Figure 7-11 on page 198. This is followed by Figure 7-12 on page 199 which lists the drives that you will import. At this point, all of the drives letter have been assigned to your VDisks; you should run chkdsk to ensure filesystem consistency as shown in Example 7-13. This is particularly important if the freezing was due to an unplanned Metro/Global Mirror stoppage.
197
Figure 7-10 Results of hardware scan on a Windows 2003 server
Figure 7-11 Importing Foreign Disks
198
Figure 7-12 Importing Foreign Disk Volumes
199
7.5 Linux Specifics

In this section, we describe the steps needed to create and access copied VDisks on Linux Hosts. SVC currently supports Red Hat and Suse Linux, but the steps are distribution independent. SVC supports the use of multiple multi-pathing solutions when accessed via a Linux server. The examples below focus on the use of SDD.

As with AIX and Windows, it is necessary to ensure that any information in the filesystem cache gets written to the VDisks, prior to freezing the VDisks. The simplest way to do this is to unmount the filesystem, as was shown in Example 7-1 on page 175. The XFS filesystem includes support for freezing filesystems in a similar manner to AIX. Details on this functionality is beyond the scope of this redbook, but more details can be found in an IBM Developerworks article: http://www.ibm.com/developerworks/linux/library/l-fs9.html. Example 7-15 shows the initial configuration of the SVC cluster presenting VDisks to a Red Hat Linux server.
Example 7-15 Configuration of clusters prior to copying a VDisk with Metro Mirror
IBM_2145:ITSOCL2:admin>svcinfo lsvdisk -delim : id:name:IO_group_id:IO_group_name:status:mdisk_grp_id:mdisk_grp_name:capacity:type :FC_id:FC_name:RC_id:RC_name:vdisk_UID:fc_map_count 8:diomede-1ary:0:io_grp0:online:0:DS4500:36.0GB:striped:::::6005076801AD80E8E00000 0000000014:0 IBM_2145:ITSOCL2:admin>svcinfo lshost DIOMEDE id 0 name DIOMEDE port_count 2 type generic mask 1111 iogrp_count 4 WWPN 210000E08B0548BC node_logged_in_count 2 state active WWPN 210000E08B0541BC node_logged_in_count 2 state active
IBM_2145:ITSOCL3:admin>svcinfo lsvdisk -delim : id:name:IO_group_id:IO_group_name:status:mdisk_grp_id:mdisk_grp_name:capacity:type :FC_id:FC_name:RC_id:RC_name:vdisk_UID:fc_map_count 6:diomede-2dary:0:io_grp0:online:0:DS4500:36.0GB:striped:::::6005076801A100E908000 00000000008:0 Example 7-16 on page 201 shows the configuration of an Linux server prior to copying a VDisk with Metro Mirror.
200
Example 7-16 Linux host configuration prior to copying 2 VDisks with Metro Mirror
[root@diomede ~]# datapath query device Total Devices : 1
DEV#: 0 DEVICE NAME: vpatha TYPE: 2145 POLICY: Optimized Sequential SERIAL: 6005076801ad80e8e000000000000014 ============================================================================ Path# Adapter/Hard Disk State Mode Select Errors 0 Host0Channel0/sda OPEN NORMAL 1 0 1 Host0Channel0/sdb OPEN NORMAL 41815 0 2 Host1Channel0/sdc OPEN NORMAL 0 0 3 Host1Channel0/sdd OPEN NORMAL 41679 0
[root@diomede ~]# pvdisplay --- Physical volume --PV Name /dev/hda2 VG Name VolGroup00 PV Size 74.41 GB / not usable 0 Allocatable yes PE Size (KByte) 32768 Total PE 2381 Free PE 3 Allocated PE 2378 PV UUID Xq0UQs-94ID-tCq7-4jK2-KRt7-4iYF-lCf9wf --- Physical volume --PV Name /dev/vpatha VG Name itso PV Size 36.00 GB / not usable 0 Allocatable yes PE Size (KByte) 4096 Total PE 9215 Free PE 4607 Allocated PE 4608 PV UUID dAZQYq-VsI1-n2xp-vp0c-hfSE-glCx-MjnPV3
[root@diomede ~]# vgdisplay itso --- Volume group --VG Name itso System ID Format lvm2 Metadata Areas 1 Metadata Sequence No 2 VG Access read/write VG Status resizable MAX LV 0 Cur LV 1 Open LV 1 Max PV 0 Cur PV 1 Act PV 1
201
VG Size PE Size Total PE Alloc PE / Size Free PE / Size VG UUID
36.00 GB 4.00 MB 9215 4608 / 18.00 GB 4607 / 18.00 GB 0PsMcJ-nDr8-b3ds-YyCM-Mlde-OegZ-LZFddT
[root@diomede /]# df -T Filesystem /dev/mapper/VolGroup00-LogVol00 /dev/hda1 none /dev/mapper/itso-mmPrimary
Type ext3 ext3 tmpfs ext3
1K-blocks 74699952 101086 1033144 18578172
Used 11571300 22817 0 77888
Available 59334120 73050 1033144 17556568
Use% Mounted on 17% / 24% /boot 0% /dev/shm 1% /mmSource
[root@diomede /]# mount /dev/mapper/VolGroup00-LogVol00 on / type ext3 (rw) none on /proc type proc (rw) none on /sys type sysfs (rw) none on /dev/pts type devpts (rw,gid=5,mode=620) usbfs on /proc/bus/usb type usbfs (rw) /dev/hda1 on /boot type ext3 (rw) none on /dev/shm type tmpfs (rw) none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw) sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw) /dev/mapper/itso-mmPrimary on /mmSource type ext3 (rw) The VG which is being copied is itso which contains a single vpath, vpatha. This is an ext3 filesystem, There are no user commands for freezing this filesystem, so the only way to freeze the VDisk cleanly is to unmount the filesystem and deactivate the LV. However, with appropriate planning and scripting, the disruption can be minimized. 1. Create the Metro Mirror consistency group IBM_2145:ITSOCL2:admin>svctask mkrcconsistgrp -name diomedeMM -cluster ITSOCL3 RC Consistency Group, id [253], successfully created 2. Create the Metro Mirror relationship connecting the two VDisks and add it to the consistency group IBM_2145:ITSOCL2:admin>svctask mkrcrelationship -master diomede-1ary -aux diomede-2dary -cluster ITSOCL3 -name diomedeMM1 -consistgrp diomedeMM RC Relationship, id [8], successfully created 3. Start the consistency group IBM_2145:ITSOCL2:admin>svctask startrcconsistgrp diomedeMM 4. Monitor the consistency group and wait until it attains consistency and synchronization IBM_2145:ITSOCL2:admin>svcinfo lsrcconsistgrp diomedeMM id 253 name diomedeMM master_cluster_id 000002006B603A38 master_cluster_name ITSOCL2 aux_cluster_id 0000020068403A42 aux_cluster_name ITSOCL3 primary master 202
state consistent_synchronized relationship_count 1 freeze_time status online sync copy_type metro RC_rel_id 8 RC_rel_name diomedeMM1 5. Unmount the filesystem to flush all the data to the primary VDisk [root@diomede ~]# umount /mmSource 6. Stop the Metro Mirror consistency group. This is the freeze point. IBM_2145:ITSOCL3:admin>svctask stoprcconsistgrp -access diomedeMM IBM_2145:ITSOCL3:admin>svcinfo lsrcconsistgrp diomedeMM id 253 name diomedeMM master_cluster_id 000002006B603A38 master_cluster_name ITSOCL2 aux_cluster_id 0000020068403A42 aux_cluster_name ITSOCL3 primary state idling relationship_count 1 freeze_time status sync in_sync copy_type metro RC_rel_id 6 RC_rel_name diomedeMM1 7. Remount the filesystem [root@diomede ~]# mount /dev/itso/mmPrimary /mmSource At this point the Metro Mirror relationship has been stopped and diomede-2ary is an exact copy of diomede-1ary.

As with AIX, if the source/primary VDisk is defined to the AIX Logical Volume Manager (LVM) then all of its data structures and identifiers are copied to the target/secondary VDisk, as well. This includes the Universal Unique Identifiers (UUIDs) that are used to identify PVs, VGs and LVs. It is currently not possible to activate a volume group with a physical volume (vpath) that contains a VG UUID and a PV UUID that is already used in a volume group existing on the same server.
Accessing target/secondary VDisks from the same Linux host

Our research into method for bringing copied volume groups online on a Linux server have have shown up bespoke solutions that users have published on various newsgroups. There appears to be no clear method for doing this in a non-disruptive manner. Most strategies involve manually altering backups of the LVM configuration. As a result, we recommend that
203
you do not present copied VDisks back to their original servers. If you determine that this is a necessary process, you should consult with LVM experts to determine a suitable process.
Accessing target/secondary VDisks from a different Linux host

In this example, we show how to recover a VDisk which was not frozen cleanly. This can happen if a Metro Mirror relationship stops while a filesystem is in use. 1. Map the target VDisk to the new server IBM_2145:ITSOCL3:admin>svctask mkvdiskhostmap -host PALAU diomede-2dary Virtual Disk to Host map, id [0], successfully created 2. Discover the LU on the Linux server. QLogic provide a tool to perform this step. More information can be found at http://download.qlogic.com/ms/53595/readme_dynamic_lun_util_18.html 3. Run cfgvpath to create the relevant vpath [root@palau ~]# cfgvpath c--------- 1 root root 254, 0 Nov 14 22:32 /dev/IBMsdd Added vpatha 252 0 path sda 8 0.... Added vpatha 252 0 path sdb 8 16.... Added vpatha 252 0 path sdc 8 32.... Added vpatha 252 0 path sdd 8 48.... Writing out new configuration to file /etc/vpath.conf Waiting for hotplug/udev system to configure all vpath device nodes... 4. Run pvscan to pick up the copied PV [root@palau ~]# pvscan PV /dev/hda2 VG VolGroup00 lvm2 [74.41 GB / 96.00 MB free] PV /dev/vpatha VG itso lvm2 [36.00 GB / 18.00 GB free] Total: 2 [110.40 GB] / in use: 2 [110.40 GB] / in no VG: 0 [0 ] 5. Run vscan to make sure that you have your VG [root@palau ~]# vgscan Reading all physical volumes. This may take a while... Found volume group "VolGroup00" using metadata type lvm2 Found volume group "itso" using metadata type lvm2 6. Run lvscan to make sure that you have your expected LVs [root@palau ~]# lvscan ACTIVE '/dev/VolGroup00/LogVol00' [72.38 GB] inherit ACTIVE '/dev/VolGroup00/LogVol01' [1.94 GB] inherit inactive '/dev/itso/mmPrimary' [18.00 GB] inherit 7. You will need to activate your newly discovery volume group [root@palau ~]# vgchange -ay itso 1 logical volume(s) in volume group "itso" now active 8. Run fsck on the LV and then mount it for use
204
7.6 Other Operating Systems

The principles outlined above are equally relevant to other operating systems. You must be aware of the following when determining operating procedures What metadata is written to the disk (and so is copied by Copy Services) How does the operating system handle duplicated metadata How does the operating system handle foreign metadata, i.e. disks that were originally mapped to a different server How does the operating system handle VDisks that were not frozen cleanly?
205
206
7574 Automation.fm
Chapter 8.
Automation
In this chapter, we discuss methods for automating your Copy Services solution. There are a number of methods open to you, including SVC Command Line Scripting Server side scripting SVC CIMOM Programming Automation is the removal of human decision making from system operating processes. As a general rule, system designs will require a human operator to be involved at some stage, for instance: Deciding when to stop and start FlashCopy mappings or Remote Copy relationships Deciding when to failover Remote Copy relationships (or fail back). This chapter will also discuss the importance of logging as part of automation.
207
7574 Automation.fm
8.1 Running commands on the cluster

In order to automate Copy Services processes, you will need to connect to the cluster. In normal operations, this is done via the GUI or via the command line. The GUI is not an appropriate interface for automating processes, so we will not be discussing that further. All automation techniques will be via the SVC command line or via the Common Information Model Object Manager (CIMOM) (which, currently, acts as a proxy to the command line). In this section we will refer to the user agent. This may be the CIMOM (which connects to the cluster via SSH) or a user connecting directly with an SSH client, either in an interactive mode or by means of a script. Running commands to the cluster follow these steps: 1. 2. 3. 4. 5. Connection Authentication Submission Authorization Execution
8.1.1 Connection
Commands are submitted to the cluster during a connection session to the cluster. User agents make connections via the SSH protocol. SVC has a number of security features that affect how often you can attempt connections. These are to prevent attacks (malicious or accidental) from bringing an SVC node down. Although. at first glance, these features seem restrictive, they are relatively simple to work around in order to attain a valid connection. Any automation system should ensure that it behaves responsibly and does not attempt to breach the connection rules. At the very least, any automation system should ensure that it can gracefully handle rejected connection attempts. Figure 8-1 on page 209 shows how SVCs connection restrictions work. There are 2 queues in action: pending connections and active connections. The connection process follows this process: 1. Connection request comes into the SVC. If the Pending Connections queue has a free position, the request is added to it. Otherwise there are two possibilities; prior to version 4.2.1.0, the connection request times out; as of version 4.2.1.0, the connection is explicitly rejected. 2. Pending connections are handled in one of 2 ways: a. If any of the following are true, the connection request is rejected: No key is provided or the provided key is incorrect The provided username is not admin or service The Active Connections queue is full In this case, a warning is returned to the SSH client as shown in Example 8-1 on page 209 b. If none of the above are true, then the connection request is accepted and is moved from the Pending Connections queue to the Active connections queue 3. Active connections are ended after any of the following events User logs off manually SVCs SSH daemon recognizes that the connection has grown idle Network connectivity fails
208
7574 Automation.fm
The configuration node fails over In this case, both queues will be cleared as the SHH daemon will stop and restart on a different node.
Figure 8-1 SVCs SSH restrictions Example 8-1 SVC command line warning due to too many logins
Using username "admin". Authenticating with public key "rsa-key-20071010" Too many logins for 'admin'. Last login: Thu Oct 25 14:15:30 2007 from 9.67.163.247
8.1.2 Authentication
This stage is represented by step 2 of the Connection process (8.1.1, Connection). The only authentication process available is public-private key authentication. IBM System Storage SAN Volume Controller, SG24-6423 provides detailed instructions on creating keys and uploading them to the SVC. There are a few points that we strengthen here to help make automation simpler to implement
Usernames and keys

The SVC command line only has two users defined: admin
Chapter 8. Automation
209
7574 Automation.fm
service Any attempt to log on as a different user will result in a rejection, regardless of whether a valid key is provided. In this way, keys and usernames are wholly independent. The SVC GUI can have multiple usernames defined: the default username, superuser, and any others that are created. The usernames have no connection to the SVC cluster. They are, in fact, usernames for logging in to the CIMOM which runs on the Master Console. When the CIMOM logs into the SVC cluster, it will still use the admin username and the icat.ppk private key. The SVC cluster allows you to store up to 100 keys. Any of these can be used to authenticate with the cluster, regardless of which username you use on the command line. Once the user agent provide a valid username, the cluster receives the private key. If this matches with one of the clusters public keys then that keys label is remembered and used for auditing purposes. If the connection is made via that CIMOM, then the key label will always be the same (that being the public key to match the icat.ppk file). However, the username that was used to log on to the CIMOM is sent to the SVC cluster and is also remember for auditing purposes. Auditing on page 234 discusses the auditing of commands on the SVC cluster.
8.1.3 Submission
Once connected to a cluster, the user agent may start submitting commands. The syntax is checked first of all. If this fails, an appropriate error message is returned. Any automation implementation must ensure that all commands that are submitted have correct syntax. If they do not, they must be designed to handle syntax errors. Its much easier to design a solution that does not generate invalid syntax than one to handle all potential syntax errors.
8.1.4 Authorization
Commands that have valid syntax are next checked to see if the user agent has the authority to submit this command. Section 2.8.3, Authorization Roles on page 21 outlines the Authorization Roles that SVC supports. The key that was used to authenticate the connection has a role associated with it. SVC will check the submitted command against the authorization role. If the user agent is not authorized to execute this command, the following error will be returned: CMMVC6253E The command failed authorization because the session ssh key does not have the requisite role. If the user agent is authorized, the command is sent of execution.
8.1.5 Execution
When a command is executed, one of the following will occur The command will fail and an error message will be written to STDERR The command will succeed and a warning will be written to STDERR The command will succeed and a warning will be written to STDERR and information will be sent to STDOUT The command will succeed and information will be written to STDOUT The command will succeed and nothing will be written.
210
7574 Automation.fm
Data written to STDOUT and STDERR by the cluster should be written to STDOUT and STDERR by your SSH client, but you will need to check this yourself.
8.2 Creating connections

Connecting to the cluster is the first step to running commands. Any automation solution will require a connection component. This component should be as robust as possible as it forms the foundation of your solution. There are two forms of connection solution: Transient Persistent One command is submitted per connection and the connection is closed once the command is completed. Connection is made and stays open. Multiple commands are submitted via this single connection. These include interactive sessions and the CIMOM.
8.2.1 Transient connections

Transient connections are simple to create. The most common SSH clients allow the submission of a single command as part of their invocation. Example 8-3 and Example 8-2 show this using ssh on a AIX server and plink on a Windows server
Example 8-2 Transient connection to an SVC cluster from an AIX server
[root@heket]/ >ssh -i privateKey -l admin rc-cluster-13 svcinfo lscluster -delim : id:name:location:partnership:bandwidth:cluster_IP_address:cluster_service_IP_addre ss:id_alias 00000200694052D0:rc-cluster-13:local:::9.71.42.19:9.71.42.75:00000200694052D0 [root@heket]/ >
Example 8-3 Transient connection to an SVC cluster from a Windows server
C:\Documents and Settings\Administrator>plink -i "C:\Documents and Settings\Administrator\My Documents\ITSO\private.ppk" -l admin 9.43.86.117 svcinfo lscluster -delim : id:name:location:partnership:bandwidth:cluster_IP_address:cluster_service_IP_addre ss:id_alias 0000020060406FCA:ITSOCL1:local:::9.43.86.117:9.43.86.118:0000020060406FCA C:\Documents and Settings\Administrator> These transient connections go through all 5 stages of running a command and return to the command line. You can redirect the two output streams (STDOUT and STDERR) using the operating systems standard redirection operators to capture the responses. These lengthy invocations can be shorted in client specific ways. The AIX SSH client allows for the use of user configuration files. The configuration file in Example 8-4 would allow you to create a transient connection as shown in Example 8-5.
211
7574 Automation.fm
Example 8-4 Sample SSH configuration file (saved as sampleCfg)
Host rc13 HostName rc-cluster-13 IdentityFile /privateKey User admin Host someOtherCluster IdentityFile /someOtherKey User admin
Example 8-5 Transient connection to an SVC cluster using ssh using a configuration file
[root@heket]/ >ssh -F sampleCfg rc13 svcinfo lscluster -delim : id:name:location:partnership:bandwidth:cluster_IP_address:cluster_service_IP_addre ss:id_alias 00000200694052D0:rc-cluster-13:local:::9.71.42.19:9.71.42.75:00000200694052D0 [root@heket]/ > Shortening the plink invocation requires the creation of a PuTTY session. In order to do this, open up the PuTTY application and enter admin@<cluster IP address> in the Host Name textbox as in Figure 8-2
Figure 8-2 Adding username and IP address to a PuTTY session
The next step is to configure the private key for this session. Select the Connection SSH Auth category as in Figure 8-3 on page 213
212
7574 Automation.fm
Figure 8-3 Setting the private key for a PuTTY SSH session
Select the appropriate private key using the Browse... button and locating it in your filesystem. The final step is to save the session for future use. A descriptive session name is advised as shown in Figure 8-4 on page 213
Figure 8-4 Saving a PuTTY session for use with plink
213
7574 Automation.fm
Once a session has been saved, it can be used for making transient connections via the command line as in Example 8-6
Example 8-6 Transient connection to an SVC cluster using plink with a PuTTY session
C:\Documents and Settings\Administrator>plink -load ITSOCL1 svcinfo lscluster -delim : id:name:location:partnership:bandwidth:cluster_IP_address:cluster_service_IP_addre ss:id_alias 0000020060406FCA:ITSOCL1:local:::9.43.86.117:9.43.86.118:0000020060406FCA C:\Documents and Settings\Administrator>
8.2.2 Persistent connections

A persistent connection is a connection that exists beyond the submission and execution of a single command. As outlined above, the CIMOM provides a persistent connection, but it doesnt provide direct access to the command line. In order to provide a persistent connection to the command line, you need to use multiple processes. There are as many ways to do this as there are programming languages. Example 8-12 includes a method of creating a persistent connection, using Korn Shell. Most methods involve creating a process which connects to the cluster and then writing to its STDIN stream and reading from its STDOUT and STDERR streams. The difficulty comes with detecting when the output from a command ends. This is handled in Example 8-12. Persistent connections can be used in a number of ways. On a per-script basis. A script will open up a connection that will exist for the life of the script, allowing multiple commands to be submitted, but will end when the script ends As a stand-alone script. The connection will be opened and other scripts will communicate with this one script, to submit commands to the cluster. This allows the connection to be shared by multiple scripts. This, in turn, allows a greater number of independent scripts to access the cluster without using up all of the connection slots.
8.3 SVC command line scripting

When connected to the cluster command line, it is possible to do small amounts of automation, for instance Repeatedly submitting a single command to a set of SVC objects Searching the configuration for objects conforming to certain criteria The SVC command line is actually a highly restricted Bash shell. There is no access to the Unix commands like cd or ls. The only commands which are available are those which are built-ins, such as echo or read. In addition, redirection of inputs and outputs are not supported, but commands can be piped together. Example 8-7 shows a sample script that will list all of the VDisks which are not online. This complements the filtervalue parameter of the lsvdisk command. The filtervalue parameter only allows matches when a property matches a value. This command line script allows matches done according to other criteria.
Example 8-7 SVC Command line script for listing all VDisks that are not online
001. svcinfo lsvdisk -nohdr | while read id name IOGid IOGname status rest
214
7574 Automation.fm
002. do 003. if [ "$status" != "online" ] 004. then 005. echo "VDisk '$name' $$id$ is $status" 006. fi 007. done
Line 1 is of particular note. This line submits the svcinfo lsvdisk command and pipes the output to the read command, combined with a while command. This creates a loop which executes once per line of output from the svcinfo command. The read command is followed by a list of variables. A line is read from the svcinfo command and the first word in that line is assigned to the first variable, the second word is assigned to the second variable and so on, with any left over words assigned to the final variable (with intervening spaces included). We use the -nohdr parameter because were not interested in this information. Lines 3 to 6 simply check the status variable. If it is not equal to online, then the information is printed to STDOUT.
8.3.1 Submitting command line scripts

Command line scripts can be submitted from an interactive prompt, if so required. However, the can also be submitted as batch files. Example 8-9 and Example 8-10 show how this can be done with ssh or plink. Both commands submit a simple batch file shown in Example 8-8. This command lists the MDisks which are quorum MDisks; in a large configuration this can be a useful script to have.
Example 8-8 Example command line batch file batchFile.sh
svcinfo lsmdisk -nohdr | while read id name rest do svcinfo lsmdisk $id | while read key value do if [ "$key" == "quorum_index" ] then if [ "$value" != "" ] then echo "MDisk $id ($name) is quorum disk $value" fi fi done done
Example 8-9 Submission of a batch file to SVC via ssh
[root@heket]/ >ssh -F sampleCfg rc13 -T < batchFile.sh MDisk 0 (mdisk0) is quorum disk 0 MDisk 1 (mdisk1) is quorum disk 1 MDisk 2 (mdisk2) is quorum disk 2
215
7574 Automation.fm
Example 8-10 Submission of a batch file to SVC via plink
C:\Documents and batchFile.sh MDisk 0 (mdisk0) MDisk 1 (mdisk1) MDisk 2 (mdisk2)
Settings\Administrator\My Documents\SVC>plink -load ITSOCL1 -m is quorum disk 0 is quorum disk 1 is quorum disk 2
8.3.2 An example SVC Command Line script

Example 8-11 shows a non-trivial script that can be executed directly from the SVC command line via batch submission. The script generates a CSV table showing how many extents each VDisk gets from each MDisk
Example 8-11 Example script for generating a CSV representation of MDisk/VDisk extent usage
001. echo vDisk,mDisk,Controller,mDisk Group,Extents; 002. 003. vdiskIds=(`svcinfo lsvdisk -nohdr | while read id rest; do echo -n "$id "; done`) 004. vdiskNames=(`svcinfo lsvdisk -nohdr | while read id name rest; do echo -n "$name "; done`) 005. vdiskNameMap=() 006. for (( i = 0 ; i < ${#vdiskNames[@]} ; i++ )) 007. do 008. vdiskNameMap[${vdiskIds[$i]}]=${vdiskNames[$i]} 009. done 010. 011. svcinfo lsmdisk -nohdr | while read mdiskId mDiskName status mode mdgId mdgName capacity LUN controllerName UniqueID; 012. do 013. svcinfo lsmdiskextent -nohdr $mdiskId | while read vdiskId extents; 014. do 015. echo ${vdiskNameMap[$vdiskId]},$mDiskName,$controllerName,$mdgName, $extents; 016. done 017. done Line 1 simply generates the table header row Lines 3 and 4 work around a limitation of the bash shell. If you assign a value to a variable while inside a loop that is running from a pipe, that value is not available to you when outside the loop. A way to get around this is command substitution. Line 3 creates an array of VDisk ids by looping through all of the vdisks and printing the id. This output is captured using backticks. The parentheses turn this list of ids into an array. Line 4 does the same with VDisk names Lines 5 to 9 create an associative array that maps a VDisk id to a VDisk name. The loop this time does not involve a pipe, so the changes to the vdiskNameMap array are available after line 9. Line 11 begins a loop over all MDisks.
216
7574 Automation.fm
Line 13 loops over all of the MDisk Extent records for the current MDisk. A line is printed for each VDisk that uses extents from the current MDisk.
8.4 Server side scripting

Server side scripting involves scripting where the majority of the programming logic is executed on a server. Part of server side scripting is the generation and management of connections to the SVC cluster. Section 8.4.1, An example script for creating persistent SVC connections shows how to create and manage a persistent connection to a cluster and how to manage requests coming from multiple scripts. In addition to this example, there is an SVC Perl module available at http://alphaworks.ibm.com/tech/svctools (SVCTools) that handles the connection aspect of any script. As connection management is often the most complex part of any script, we recommend that you investigate this module. Currently, this module uses transient connections to submit commands to a cluster so it may not be the best approach if you plan to have multiple scripts attempting to submit commands independently.
8.4.1 An example script for creating persistent SVC connections

Example 8-12 shows an example of a script that opens up a persistent connection to a cluster and handles commands from other, external, scripts. In this section, we describe the various features of the script to give an idea of some of the issues that need to be resolved.
Example 8-12 Sample script for creating a persistent connection to an SVC cluster
001. 002. 003. 004. 005. 006. 007. 008. 009. 010. 011. 012. 013. 014. 015. 016. 017. 018. 019. 020. 021. 022. 023. 024. 025.
#!/bin/ksh # # # # # # # # # # # # # # # # # # # #
SVC Command Queue This script implements a queue that any number of clients can write to. The clients provide SVC commands and the name of a file to where the output should be written Commands should be written to the queue thus: <command><delimiter><outFile> command: SVC command to be submitted to the cluster delimiter: User definable character (defined in this script) outFile: File to which the output of the SVC command should be written e.g.: echo svcinfo lscluster -delim :+/tmp/output This would send the 'svcinfo lscluster -delim :' command to the cluster and place the output in the file '/tmp/output' The '+' character delimits the command and the outpue file
########################################################### # Functions start
217
7574 Automation.fm
026. 027. 028. 029. 030. 031. 032. 033. 034. 035. 036. 037. 038. 039. 040. 041. 042. 043. 044. 045. 046. 047. 048. 049. 050. 051. 052. 053. 054. 055. 056. 057. 058. 059. 060. 061. 062. 063. 064. 065. 066. 067. 068. 069. 070. 071. 072. 073. 074. 075. 076. 077. 078. 079. 080.
########################################################### # # Opens an SSH pipe and purges any extraneous data from the buffer # SSHPID gets set to the PID of the SSH process # # This function assumes that the connection will work # open_SSH_pipe() { echo "Opening SSH pipe to $CL..." ssh -p 22 -q admin@$CL 2>&1 |& SSHPID=$! echo "Started SSH process: $SSHPID" # # Purge any data from the connection # echo "Purging connection buffer..." svc_transaction "svcinfo lscluster" "/dev/null" echo "Connection buffer purged" } # # Opens the command FIFO and connects it to file descriptor 4 # open_command_FIFO() { # if the FIFO doesn't exists, create it if [ ! -a $FIFO ] then echo "Cannot find $FIFO so creating it" mkfifo $FIFO fi exec 4<$FIFO } # # svc_transaction(command,outputfile) # # Submits the supplied command to the SVC cluster # Send the output to the outputfile # svc_transaction() { # # First, submit the command to the cluster # followed by a command to print the return code # and then a command to print the End Of Output string # print -p "$1" print -p "echo \$?" print -p "echo $COMMAND_ENDER" #
218
7574 Automation.fm
081. 082. 083. 084. 085. 086. 087. 088. 089. 090. 091. 092. 093. 094. 095. 096. 097. 098. 099. 100. 101. 102. 103. 104. 105. 106. 107. 108. 109. 110. 111. 112. 113. 114. 115. 116. 117. 118. 119. 120. 121. 122. 123. 124. 125. 126. 127. 128. 129. 130. 131. 132. 133. 134. 135.
# Now, read back from the SVC cluster # read -p prevLine while true do read -p currLine if [ "${currLine}" == "$COMMAND_ENDER" ] then # # Append the SVC Return Code to the output file # echo $RC_PREPEND$prevLine$RC_APPEND >> $2 echo $COMMAND_ENDER >> $2 return else echo "${prevLine}" >> $2 prevLine=$currLine fi done } ########################################################### # Functions end ###########################################################
# # CL: cluster name - to be provided on the command line # FIFO: FIFO name # DELIM: Delimiter separating commands from their output files # COMMAND_ENDER: String that signifies that the SVC command output has ended # RC_PREPEND: # : Strings that wrap round the return code from the SVC cluster # RC_APPEND : # if [ "$1" == "" ] then echo "Please provide a cluster name" return 0 fi CL=$1 FIFO=/tmp/FIFO.$CL DELIM=+ COMMAND_ENDER="+++COMPLETE+++" RC_PREPEND="+++RC:" RC_APPEND="+++"
open_SSH_pipe open_command_FIFO echo "Queue is ready..." while true do while read CMD <&4
219
7574 Automation.fm
136. 137. 138. 139. 140. 141. 142. 143. 144. 145. 146. 147. 148. 149. 150. 151. 152. 153. 154. 155. 156. 157. 158. 159. 160. 161. 162.
do OUTFILE=`echo $CMD | cut -d $DELIM -f 2` echo "Outfile: $OUTFILE" SVCCMD=`echo $CMD | cut -d $DELIM -f 1` echo "SVC Command: $SVCCMD" # # Probably a good idea to check that the SSH process is still running and # handle it if it is not. # ps -p $SSHPID | grep -q ssh if [ $? = 0 ] then svc_transaction "$SVCCMD" $OUTFILE else # # Handle the fact that SSH is no longer running # echo "SSH pipe seems to have died" return 0 fi done done # # Closes FIFO, although we will never get here. # exec 4<&-
open_SSH_pipe
This function runs from lines 33 to 47 and has the job of opening up the connection to the SVC cluster. The only input is a global variable called $CL which holds the name of the cluster. The function is quite simple, only 3 functional lines (36, 37 and 45). The first line opens up the connection, redirects STDERR to STDOUT and makes this connection a coprocess. A coprocess is a Korn shell term which means a process that is running in the background to the main process. STDIN and STDOUT of a coprocess are available to the main process for reading and writing. Line 37 saves the process id into $SSHPID for future reference. Line 45 runs a subroutine called svc_transaction to clear the IO buffer between the main process and the coprocess. This subroutine is described in , svc_transaction.
open_command_FIFO
This function runs from lines 52 to 61 and has the job of opening up a First In First Out (FIFO) file. The FIFO file acts as a queue of commands that this script must handle. External scripts add commands to this queue and this script takes them off the queue in the order that they came in and submits them to the cluster. A FIFO file is a Unix construct. It is essentially a named pipe. Lines 55 to 59 look to see if the FIFO file has been created (and create it if it hasnt been). Line 60 opens up the FIFO file with file descriptor 4 for reading. A file descriptor is simply a handle to this file; Unix shells can read and write from these file descriptors. 220
7574 Automation.fm
svc_transaction
This function runs from lines 69 to 100 and has the job of submitting a command to the SVC cluster and writing the output to a file. It takes two parameters; the first is the full command to submit to the cluster and the second is a local file, to which the output of the command should be written. Lines 76 to 78 send information to the coprocess, i.e. the persistent connection. This is done with the print -p command.First, the command is sent. Then, a request to echo the return code of the command. Finally, a request to echo a known string. This is used to detect the end of output from the submitted command. Lines 83 to 99 handle the gathering of the clusters response. The read -p reads from the coprocess and puts the information into the supplied variable. The loop is fairly simple: Read one line from the coprocess and save as prevLine Start Of Loop Read next line from coprocess as currLine If currLine matches the end of command string a. Write prevLine to output file with a wrapper indicating that it is the return code b. Write the end of command string to the output file c. End subroutine 5. If not: a. Write prevLine to output file b. Save currLine as prevLine 6. Goto step 2 Once this subroutine is complete, the IO buffer to the coprocess will be clear 1. 2. 3. 4.
Main Script
The main script runs from line 116 to 162 and sets up the FIFO and persistent connection and then sits waiting for commands to be added to the queue, before submitting them to the cluster. Lines 116 to 126 set up various variables required for execution. A more mature implementation would allow these to be set via a configuration file, but this method suffices for a demonstration. The script requires that a cluster name be provided as a command line parameter. Lines 129 and 130 open up the connection to the SVC cluster and open up a connection to the command queue (the FIFO file) Lines 133 to 157 are the main loop. This loop monitors the command queue and handles incoming commands. The following process is followed 1. Read a line from the FIFO. This is a blocking call and will wait here until a line is available 2. Split this line on a predefined delimiter. The first field is the command destined for the cluster. The second field is the file to which all output should be directed 3. Check that the connection to the cluster is still running. In this example, the script ends if the connection ends. A more sophisticated response would be to re-attempt connection. 4. If the connection is still running, the command is passed to the svc_transaction subroutine The final line should never be reached due to the constant loop, but its action is to close the file descriptor to the FIFO queue.
221
7574 Automation.fm
Command submission
If multiple scripts are trying to write to the command queue that was created above, steps must be taken to ensure that the writes are atomic. That means that if two scripts attempt to write to the command queue at overlapping times, the commands written do not interleave. Example 8-13 shows a simple C program which performs an atomic append to a file. This is a perfect utility for writing to the FIFO file, when multiple scripts are accessing at the same time. It will block on the write function on line 59 until it is free to write. All other scripts attempting to write will be blocked at the same time.
Example 8-13 Code listing for a utility that performs atomic appends to a file
001. 002. 003. 004. 005. 006. 007. 008. 009. 010. 011. 012. 013. 014. 015. 016. 017. 018. 019. 020. 021. 022. 023. 024. 025. 026. 027. 028. 029. 030. 031. 032. 033. 034. 035. 036. 037. 038. 039. 040. 041. 042. 043. 044. 222
#include #include #include #include #include
<stdlib.h> <stdio.h> <fcntl.h> <string.h> <unistd.h>
int main(int argc, char *argv[]) { /* * First parameter is the file * Second parameter is the text */ char *filename; int fileD; char *text; int textLen; // // // // Name of file to append to File descriptor (for file access) Text to append to the file in question Length of the text that is appended to the file
/* * Ensure that we have the required parameters */ if (argc != 3) { printf("Usage: append <file> <text>\n"); return -1; } filename = argv[1]; /* * The text appended to the file will have a "newline" appended to it * We also need to include space in the buffer for the null-terminator */ textLen = strlen(argv[2]) + 2; text = (char *) malloc((textLen)*sizeof(char)); if(text == NULL) { printf("Could not allocate memory\n"); return -2; } strcpy(text,argv[2]); strcat(text,"\n"); /*
7574 Automation.fm
045. 046. 047. 048. 049. 050. 051. 052. 053. 054. 055. 056. 057. 058. 059. 060. 061. 062. 063. 064. }
* Open the file that we are appending to * NB: We do not attempt to create this file if it does not exist */ fileD = open(filename, O_WRONLY | O_APPEND, 0); if(fileD < 0) { perror("Cannot open append file for writing"); return -2; } /* * Write to append file and then close the file */ write(fileD,text,textLen); close(fileD); return 0;
8.4.2 Handling the SVC response

From this point on, we assume that you have an implementation that allows you to submit commands to the cluster and retrieve the response in an effective manner. The next step is to be able to handle the responses usefully.
Handling errors
The SVC Command Line Interface Guide provides a list of the possible errors that an SVC command can result in. Its important to be able to recognize and handle these commands. Every SVC Cluster error has a unique code of the following format: CMMVCxxxxE for an error, or CMMVCxxxxW for a warning. These codes map to a single outcome, although different commands may result in identical errors. For example, the CMMVC5786E code maps on to the The action failed because the cluster is not in a stable state error, which may be returned for any command that seeks to change the cluster configuration by adding or removing an object. Its important that any automation solution at the very least reports these errors accurately, as they will be needed for troubleshooting. At the next level, an automation solution should be able to handle errors automatically. A large proportion of the errors are to do with invalid requests such as trying to use objects that do not exist, or trying to change the state of an object to an invalid state. These can be avoided by ensuring that your scripts check object states before changing them. They can also be used as sanity checks: if you hit one of these errors, something is obviously wrong and the user should be alerted.
svcinfo commands
These are critical to automation as they tell you the current state of the cluster. It is important to ascertain object states before changing them if you are to implement an enterprise level automation solution. If you use the SVCTools Perl module, then you will be able to access cluster object properties as a result of executing the functions that this module provides. In this section we discuss how to access these properties via the Korn Shell.
223
7574 Automation.fm
We strongly recommend that commands submitted to the SVC cluster by an automation solution are submitted using the -delim parameter; in general, we use the colon character (:) as the delimiter. This is just a convention and you can use any character that you would not otherwise expect in the output. Once the output has been captured, each line represents an object. This line can then be split into words, separated by the chose delimiter. Each of these words represents a property At this point, you have access to the information you need; write your scripts to perform the processes according to your business logic.
svctask commands
When these commands succeed, the feedback is minimal. For svctask commands that create objects, they will return with the id of the created object. The format of the response will be <object type>, id [<new objects id>], successfully created. Thus, the newly created objects id can be captured from this line. Example 8-14 shows how this might be done in a Perl program
Example 8-14 Method for capturing a newly created SVC objects id in Perl
# # $line is a SCALAR containing the response line from SVC # if ($line =~ /(.*), id \[(\d*)\], successfully created/) { $objectType = $1; $newId = $2; } This id is important as it allows you to execute further commands against the newly created object. Other commands complete successfully with no feedback other than a return code of zero. For instance, commands that change an object will return no feedback if they are successful. It may be prudent to request their details before proceeding, to ensure that any data structures that you have created are in sync with the cluster objects states.
8.5 SVC CIMOM programming

Section 8.4, Server side scripting on page 217 discusses automation by connecting directly to the cluster command line. As outlined in section 8.2, Creating connections on page 211, this requires you to manage connections in order to prevent failures when the connection limit is hit. In addition to this, it is difficult to manage connections without sacrificing the user authentication process. CIMOM Programming provides the opportunity to handle multiple connections from multiple sources while maintaining security. CIM Clients connect to the CIMOM with a username and password and then invoke methods to execute commands. CIM Agents Developer Reference, SC26-7904 contains details of the CIM Classes, Associations and Methods that you need, in order to interact with the cluster. Creating a CIM Client is straightforward, once you have found a suitable framework. Our investigations have uncovered a few, but the Java WBEM Service project seems to be one of the more widely referenced ones. This can be found at
224
7574 Automation.fm
http://wbemservices.sourceforge.net/. There are implementations in other languages, including C++ and Python, but our experience with Java directed us to this one. Example 8-15 shows a simple Java program that connects to an SVC CIMOM. It does nothing more than connect, but this is the first step in automating with the CIMOM.
Example 8-15 Java program for connecting to an SVC CIMOM
import java.util.*; import javax.wbem.cim.*; import javax.wbem.client.*; public class ITSOClient { public static void main(String[] args) { String username = args[0]; String password = args[1]; String masterConsoleIP = args[2]; String masterConsoleSecurePort = args[3]; UserPrincipal user = new UserPrincipal(username); PasswordCredential pwd = new PasswordCredential(password); CIMNameSpace ns = new CIMNameSpace("https://+ masterConsoleIP+:+ masterConsoleSecurePort+/root/ibm"); CIMClient client = null; try { System.out.println("Connecting to CIMOM"); client = new CIMClient(ns,user,pwd); } catch (CIMException e) { // Handle the CIM Exception e.printStackTrace(); } }
Once connected to the CIMOM, the next step is to identify the cluster that you wish to work with. Before we discuss that, we will discuss some CIM Agent concepts.
8.5.1 CIM concepts

A full definition of the CIM specification is beyond the scope of this book, but a brief description of a few concepts here will allow you to experiment with this technology. This section assumes some familiarity with Object Oriented Programming (OOP) concepts. Namespace The scope within which all of the CIM classes are defined. For SVC, this contains all of the SVC CIM classes that your scripting will manipulate The definition of an object within the SVC hierarchy. It is directly equivalent to an OOP class.
Class
225
7574 Automation.fm
Instance
An instance of an object that is a member of a CIM class. It is directly equivalent to an OOP object and contains all the properties of an object in the SVC configuration A way to implement a function on a class An attribute used to characterize the instance of a class A value that defines the behavior of a property, classes and methods A pointer to a CIM instance. This is a typed string that is identified as being a reference to the instance in question A formatted string that is used to access namespaces, classes and instances A class with two references which defines a relationship between the two referenced classes
Method Property Qualifier Reference Object Path Association
The CIM classes are part of a broader, cross-vendor specification and so do not always map exactly onto SVC object types. CIM Agents Developer Reference, SC26-7904 shows all of the relevant classes.
8.5.2 How SVC concepts map to CIM concepts

In order to administer Copy Services via the CIMOM, its important to understand how to translate between SVC and CIM concepts. Table 8-1 shows how these relate to one another.
Table 8-1 Relating SVC concepts to CIM concepts SVC Concept CIM concept CIM Name Cluster Cluster ID VDisk VDisk ID FlashCopy Mapping FlashCopy Mapping Status mkfcmap preparefcmap startfcmap Remote Copy relationship Remote Copy relationship state mkrcrelationship startrcrelationship IBMTSSVC_Cluster ElementName IBMTSSVC_StorageVolume DeviceID IBMTSSVC_LocalStorageSynchronized SyncState AttachReplica ModifySynchronization ModifySynchronization IBMTSSVC_SyncCopyStorageSynchroniz edSet NativeState AttachReplica ModifySynchronization CIM Concept Class Property Class Property Association Property Method Method Method Association Property Method Method
226
7574 Automation.fm
8.5.3 Example script to create and start a FlashCopy mapping

Example 8-16 is the main method from a Java class that is designed to create and start a FlashCopy mapping. The other methods that are called are described in later examples. This serves as a demonstration of how CIMOM Methods can be used to control the cluster. In this section, method refers to a Java method. Method (initial capital) refers to a CIM Method.
Example 8-16 Java main method for creating and starting a FlashCopy Mapping
/* * FC Mapping states */ private static UnsignedInt16 INITIALIZED = new UnsignedInt16(2); private static UnsignedInt16 PREPARING = new UnsignedInt16(3); private static UnsignedInt16 PREPARED = new UnsignedInt16(4); public static void main(String[] args) throws CIMException { /* * First step is to connect to the CIMOM */ UserPrincipal user = new UserPrincipal("superuser"); PasswordCredential pwd = new PasswordCredential("itso13sj"); CIMNameSpace ns = new CIMNameSpace("https://9.43.86.115:5989/root/ibm"); CIMClient client = null; client = new CIMClient(ns,user,pwd); /* * Next, select the cluster that we're interested in */ CIMInstance chosenCluster = getCluster("ITSOCL1",client); /* * At this point, the relevant cluster has been selected * and 'chosenCluster' is a CIMInstance of this cluster * * Get the Config Service of this cluster */ CIMObjectPath cService = getConfigService(chosenCluster, client); /* * Now, get all of the VDisks in this cluster */ Map<Integer,CIMObjectPath> vdisksById = getVDisks(chosenCluster,client); /* * Select the FlashCopy Source * * In this case, VDisk 10 is our source * VDisk 11 is our target */ CIMObjectPath fcSrc = vdisksById.get(new Integer(10)); CIMObjectPath fcTgt = vdisksById.get(new Integer(11));
227
7574 Automation.fm
/* * Now, create FC Mapping */ CIMValue rc = makeFlashCopyMapping("CIMOMTestMap", fcSrc, fcTgt, cService, client,false); /* * Now that this has been created, we need to get an * Object Path to the newly created Association */ List<CIMObjectPath> fcMaps = getFCMappings(fcSrc, client); CIMObjectPath fcMapping = fcMaps.get(0); /* * Now, we prepare the FC Mapping */ CIMArgument[] outArgs = new CIMArgument[2]; rc = prepareFCMapping(cService, fcMapping, client, outArgs); System.out.println("Got value:"+ Integer.toHexString(Integer.parseInt(rc.toString()))); /* * Loop until it is prepared */ CIMValue fcMapState = new CIMValue(PREPARING); while(fcMapState.equals(new CIMValue(PREPARING))) { CIMInstance fcMapInfo = client.getInstance(fcMapping); fcMapState = fcMapInfo.getProperty("SyncState").getValue(); } /* * Now start the FC Mapping */ rc = startFCMapping(cService, fcMapping, client, outArgs); System.out.println("Got value:"+ Integer.toHexString(Integer.parseInt(rc.toString()))); } The assumption in this example is that your Java program is designed to control the same cluster every time. It is a relatively simple process to make it more flexible, but that is left to you.
getCluster method
Example 8-17 shows the getCluster method. This method returns the CIM Instance which corresponds to the cluster with the supplied name. It does this by enumerating all of the instances of the class IBMTSSVC_Cluster and then checking the name of each one. When one is found which matches the supplied name, an object path to that instance is returned.
Example 8-17 getCluster method
static private CIMInstance getCluster(String clusterName, CIMClient client) throws CIMException
228
7574 Automation.fm
{ CIMInstance chosenCluster = null; Enumeration<CIMInstance> clusters = client.enumerateInstances(new CIMObjectPath("/root/ibm:IBMTSSVC_Cluster")); while(clusters.hasMoreElements()) { CIMInstance possibleCluster = clusters.nextElement(); String possibleName = possibleCluster.getProperty("ElementName").getValue().toString(); if(possibleName.equals("\""+clusterName+"\"")) { chosenCluster = possibleCluster; } } return chosenCluster; }
getConfigService method
Example 8-18 shows the getConfigService method. The CIM_StorageConfigurationService class has no direct equivalent in an SVC, but an Instance of this Class is required for invoking Methods against. In this method, we request all of the Instances that are associated with the supplied cluster. The Association that connects a cluster to its configuration service is CIM_HostedService. A cluster will only have configuration service associated with it, so we can select the first Object Path in the enumeration.
Example 8-18 getConfigService method
static private CIMObjectPath getConfigService(CIMInstance cluster, CIMClient client) throws CIMException { Enumeration<CIMObjectPath> configServices = null; configServices = client.associatorNames( cluster.getObjectPath(), "CIM_HostedService", "CIM_StorageConfigurationService", null, null); return configServices.nextElement(); }
getVDisks method
Example 8-19 shows the getVDisks method. This returns a Map, relating VDisk ids (as Integers) to IBMTSSVC_StorageVolume Object Paths. The method requests all of the IBMTSSVC_StorageVolume Instances that are associated with the provided cluster Instance.
229
7574 Automation.fm
Example 8-19 getVDisks method
static private Map<Integer,CIMObjectPath> getVDisks(CIMInstance cluster, CIMClient client) throws CIMException { Enumeration<CIMObjectPath> vdisks = client.associatorNames( cluster.getObjectPath(), null, "IBMTSSVC_StorageVolume", null, null); Map<Integer,CIMObjectPath> vdisksById = new HashMap<Integer, CIMObjectPath>(); while(vdisks.hasMoreElements()) { CIMObjectPath vdiskOP = vdisks.nextElement(); CIMValue vdiskId = vdiskOP.getKey("DeviceID").getValue(); String idAsString = vdiskId.toString(); String idNoQuotes = idAsString.substring(1, idAsString.length()-1); vdisksById.put(Integer.parseInt(idNoQuotes), vdiskOP); } return vdisksById; }
makeFlashCopyMapping method
Example 8-20 shows the makeFlashCopyMapping method. It does this by invoking the AttachReplica against the cluster configuration service. CIM Methods take typed parameters. In this method, you can see the use of the argRef, argString and argUint16 methods. These are defined later in this section, but they act as shortcuts to generating the required arguments for the CIM Method. Note the use of CopyType. The AttachReplica method is used for FlashCopy, Metro Mirror and Global Mirror and its the CopyType argument which indicates which type is required.
Example 8-20 makeFlashCopyMapping
static private CIMValue makeFlashCopyMapping( String name, CIMObjectPath source, CIMObjectPath target, CIMObjectPath configService, CIMClient client, boolean autodelete) throws CIMException { CIMArgument src = argRef("SourceElement", source, "IBMTSSVC_StorageVolume"); CIMArgument tgt = argRef("TargetElement", target, "IBMTSSVC_StorageVolume"); CIMArgument fcName = argString("ElementName",name); CIMArgument type = argUint16("CopyType",autodelete?5:4); CIMArgument[] inArgs = {src,tgt,fcName,type}; CIMArgument[] outArgs = new CIMArgument[1]; CIMValue rc = client.invokeMethod(configService, 230
7574 Automation.fm
"AttachReplica", inArgs, outArgs); return rc; }
getFCMappings
Example 8-21 shows the getFCMappings method. This method returns a List of all the FCMappings associated with the provided vdisk. It requests a list of all of the Associations which reference the provided IBMTSSVC_StorageVolume. Currently, all of the Java WBEM Services methods of this type return Enumerations. This method converts this to a List for ease of use.
Example 8-21 getFCMappings method
static private List<CIMObjectPath> getFCMappings(CIMObjectPath vdisk, CIMClient client) throws CIMException { Enumeration<CIMObjectPath> assocs = client.referenceNames( vdisk, "IBMTSSVC_LocalStorageSynchronized", null); return Collections.list(assocs); }
prepareFCMapping method
Example 8-22 shows the prepareFCMapping method. This method prepares a FlashCopy Mapping. Much like the AttachReplica Method, the ModifySynchronization Method is used to control FlashCopy, Metro Mirror and Global Mirror; its the Operation parameter which indicates what you actually want to do.
Example 8-22 prepareFCMapping
private static CIMValue prepareFCMapping( CIMObjectPath configService, CIMObjectPath fcMapping, CIMClient client, CIMArgument[] outArgs) throws CIMException { CIMArgument operation = argUint16("Operation", 6); CIMArgument synch = argRef("Synchronization", fcMapping,"IBMTSSVC_FlashCopyStorageSynchronized"); CIMArgument[] inArgs = new CIMArgument[]{operation,synch}; outArgs = new CIMArgument[2]; return client.invokeMethod(configService, "ModifySynchronization", inArgs, outArgs);
231
7574 Automation.fm
startFCMapping method
Example 8-23 shows the startFCMapping method. This method starts a FlashCopy Mapping. As in Example 8-22, this invokes the ModifySynchronization Method. Other than using a different Operation parameter, this method is identical.
Example 8-23 startFCMapping method
private static CIMValue startFCMapping( CIMObjectPath configService, CIMObjectPath fcMapping, CIMClient client, CIMArgument[] outArgs) throws CIMException { CIMArgument operation = argUint16("Operation", 4); CIMArgument synch = argRef("Synchronization", fcMapping,"IBMTSSVC_FlashCopyStorageSynchronized"); CIMArgument[] inArgs = new CIMArgument[]{operation,synch}; outArgs = new CIMArgument[2]; return client.invokeMethod(configService, "ModifySynchronization", inArgs, outArgs); }
Argument generators
This class uses 3 argument generators: argUint16 - Example 8-24 argString - Example 8-25 argRef - Example 8-26
argUint16 method
This method returns an unsigned 16-bit integer typed argument
Example 8-24 argUint16 method
static private CIMArgument argUint16(String name, int arg) { return new CIMArgument( name, new CIMValue( new UnsignedInt16(arg), new CIMDataType(CIMDataType.UINT16) ) ); }
argString method
This method returns a string typed argument 232
7574 Automation.fm
Example 8-25 argString method
static private CIMArgument argString(String name, String str ) { return new CIMArgument( name, new CIMValue( str, new CIMDataType(CIMDataType.STRING) ) ); }
argRef method
This method returns a Reference typed argument. It is a Reference to the Instance that the provided Object Path indicates.
Example 8-26 argRef method
static private CIMArgument argRef( String name, CIMObjectPath path, String className ) { return new CIMArgument( name, new CIMValue( path, new CIMDataType(className) ) ); }
8.6 Logging
The object of automation is to remove the human factor from performing systems operations. Many steps in a process follow directly from their previous step with little or no decision making. When performed by a human operator, these processes have a natural logging. A user will recall which steps were taken and the outcome of each step. When the human operator is removed from a process, it is important to log each step to aid in auditing and troubleshooting. The SVC audit log will register whenever an svctask command is successfully executed. The SVC error log will also register when configuration events complete. These are useful, but it is important to generate your own logging as part of your automation solution. You should log the following actions with a time and date stamp: Submission of any command to the SVC, including the full SVC CLI command or CIM Method invocation Completion of any command sent to the SVC (return code and any message) ID of any generated SVC objects
233
7574 Automation.fm
If the automation solution gives up, then full explanation of why. In order to tally server, SVC and automation log, its important that all systems have a correctly set clock. The SVC cluster does not support the NTP protocol, so the only way to synchronize it with an NTP server is to use a proxy. On a Unix server, with access to a cluster, you can use the commands in Example 8-27 on page 234 to synchronize the SVC clock with your local clock. This is a script which requires user interaction, because the local servers timezone may map on to multiple possible cluster time zones. This script can be altered to be fully automatic by hard-coding the timezone code.
Example 8-27 Simple script for setting the SVC cluster clock to the local time
#!/usr/bin/ksh sshInvocation="ssh -F sampleCfg rc13" possTZs=`$sshInvocation svcinfo lstimezones -delim : | grep :\`date +%Z\`` echo "Please select a timezone" for TZ in $possTZs do echo $TZ done echo "Selection (numerical code): \c" read choice echo You chose: $choice $sshInvocation svctask settimezone -timezone $choice $sshInvocation svctask setclustertime -time `date +%m%d%H%M%Y`
8.7 Auditing
When commands are successfully executed on the SVC cluster an entry is made in the SVC audit log. Example 8-28 shows a sample audit log entry. The audit log steadily fills up with entries until it is dumped manually with the svctask dumpauditlog command, or it gets too full and dumps automatically. The entries are dumped to an audit log dumpfile.
Example 8-28 Sample audit log entry
Auditlog Entry 1086 Sequence Num: 1086 Timestamp: Thu Apr 5 08:49:13 2007 : Epoch + 1175780953 Cluster User: admin SSH Label: Administrator ICAT User: admin Result Code: 0 Result Obj ID: Action Cmd: svctask chcluster -icatip 10.2.10.96:9080
The Auditlog Entry number indicates the position of the entry in the audit log dumpfile. It is unique within that file. The Sequence Num is a unique number which continually increments. If
234
7574 Automation.fm
the entry given in Example 8-28 is the last in a dumpfile, then the first entry in the next dumpfile will have an Auditlog Entry number of 1 and a Sequence Num of 1087. The Timestamp is the time that the command was run. Epoch refers to a standard method of measuring time in a Unix environment. It is the number of seconds since midnight UTC of January 1st, 1970, not counting leap seconds. The Cluster user is the username used to access the cluster CLI. This will either be service or admin. This will exist, even if the GUI or CIMOM was used, because, ultimately, the CIMOM accesses the cluster via the CLI. The SSH Label refers to the id given to the public key that authenticated the connection. Its a useful way to track who submitted a command. If private keys are truly kept private, then this can be an effective audit policy. The ICAT User is populated when the connection to the cluster is made via the CIMOM. The username that is used to connect to the CIMOM is the one that is logged. The Result Code is either 0 or 1, depending on whether the command is successful. If the command is a mkxxx command, then Result Obj ID will hold the ID of the object that was created. The final line, Action Cmd, lists the actual command that was submitted. All of this information is sufficient to audit exactly who submitted which commands, and when. The audit log is only as useful as policies put in place to control access. If private keys and usernames are closely controlled, then the audit log give a precise history of who contacted the cluster.
235
7574 Automation.fm
236
7574bibl.fm
Related publications
The publications listed in this section are considered particularly suitable for a more detailed discussion of the topics covered in this book.
IBM Redbooks
For information about ordering these publications, see How to get Redbooks on page 238. Note that some of the documents referenced here may be available in softcopy only. IBM System Storage SAN Volume Controller, SG24-6423-05 Get More Out of Your SAN with IBM Tivoli Storage Manager, SG24-6687 IBM Tivoli Storage Area Network Manager: A Practical Introduction, SG24-6848 IBM System Storage: Implementing an IBM SAN, SG24-6116 Introduction to Storage Area Networks, SG24-5470 SAN Volume Controller: Best Practices and Performance Guidelines, SG24-7521 Using the SVC for Business Continuity, SG24-7371 IBM System Storage Business Continuity: Part 1 Planning Guide, SG24-6547 IBM System Storage Business Continuity: Part 2 Solutions Guide, SG24-6548
Other resources
These publications are also relevant as further information sources: IBM System Storage Open Software Family SAN Volume Controller: Planning Guide, GA22-1052 IBM System Storage Master Console: Installation and Users Guide, GC30-4090 IBM System Storage Open Software Family SAN Volume Controller: Installation Guide, SC26-7541 IBM System Storage Open Software Family SAN Volume Controller: Service Guide, SC26-7542 IBM System Storage Open Software Family SAN Volume Controller: Configuration Guide, SC26-7543 IBM System Storage Open Software Family SAN Volume Controller: Command-Line Interface User's Guide, SC26-7544 IBM System Storage Open Software Family SAN Volume Controller: CIM Agent Developers Reference, SC26-7545 IBM TotalStorage Multipath Subsystem Device Driver User's Guide, SC30-4096 IBM System Storage Open Software Family SAN Volume Controller: Host Attachment Guide, SC26-7563
237
7574bibl.fm
Referenced Web sites

These Web sites are also relevant as further information sources: IBM TotalStorage home page: http://www.storage.ibm.com SAN Volume Controller supported platform: http://www-1.ibm.com/servers/storage/support/software/sanvc/index.html Download site for Windows SSH freeware: http://www.chiark.greenend.org.uk/~sgtatham/putty IBM site to download SSH for AIX: http://oss.software.ibm.com/developerworks/projects/openssh Open source site for SSH for Windows and Mac: http://www.openssh.com/windows.html Cygwin Linux-like environment for Windows: http://www.cygwin.com IBM Tivoli Storage Area Network Manager site: http://www-306.ibm.com/software/sysmgmt/products/support/IBMTivoliStorageAreaNe tworkManager.html Microsoft Knowledge Base Article 131658: http://support.microsoft.com/support/kb/articles/Q131/6/58.asp Microsoft Knowledge Base Article 149927: http://support.microsoft.com/support/kb/articles/Q149/9/27.asp Sysinternals home page: http://www.sysinternals.com Subsystem Device Driver download site: http://www-1.ibm.com/servers/storage/support/software/sdd/index.html IBM TotalStorage Virtualization Home Page: http://www-1.ibm.com/servers/storage/software/virtualization/index.html
How to get Redbooks

You can search for, view, or download Redbooks, Redpapers, Technotes, draft publications and Additional materials, as well as order hardcopy Redbooks, at this Web site: ibm.com/redbooks
Help from IBM

IBM Support and downloads ibm.com/support IBM Global Services 238
7574bibl.fm
ibm.com/services
Related publications
239
7574bibl.fm
240
7574IX.fm
Index
A
AIX 175176, 178, 211 importvg 179 LVM 178 recreatevg 178179 application testing 100, 142, 149, 152 asynchronous 15, 2324, 158 attributes 101 authenticate 210 authentication 209, 224 automation 35, 84, 207209 auxiliary 2830, 55, 6768, 93 auxiliary VDisk 28, 55, 93 availability 42 consistency freeze 34 consistency group 1516, 3031, 33, 5657, 59, 9495, 97, 160, 178, 193194 consistency groups 16, 31, 33, 38, 61, 6364, 9798, 105 consistent 1213, 15, 2526, 28, 34, 5759, 9293, 95, 118, 127, 142, 175 ConsistentDisconnected 37, 89 ConsistentStopped 35, 40, 4243, 73, 8889 ConsistentSynchronized 35, 43, 73, 89 constrained link 30 copy bandwidth 2728 copy process 26, 3536, 4748, 56, 92, 94, 105, 120, 127, 142 copy rate 3, 49, 96, 120 copy service 17, 102 Copy Services 1, 57, 39, 61, 174, 188, 205, 207208, 226 Windows 2000 188 Windows 2000 limitations 188 copying state 3, 36, 108 corruption 33, 39 create a FlashCopy 149, 163
B
background copy 3, 20, 2628, 48, 70, 92, 9495, 120, 126127 background copy rate 96 backup 12, 22, 87, 93, 118, 142, 145 balance 7 bandwidth 7, 20, 2628, 48, 6970, 96, 211212, 214 bitmaps 17, 92, 102, 104 boot 202
D
data consistency 30 data flow 10 data mining 93, 100 delete a FlashCopy 149 a VDisk 149 dependent writes 30, 39, 95 detected 26, 39 disconnected 3133, 7475, 88 disk 2, 4, 8, 12, 20, 24, 28, 30, 59, 77, 82, 175, 179, 188, 215216 distance 3, 2425, 54, 68 documentation 17 dual site 84 dynamic volumes 188
C
cache 2, 4, 79, 101102, 106, 134, 159, 168, 175, 188189 caching 78, 10, 106 capacity 17, 21, 27, 102, 158159, 163, 175, 191, 200, 216 channels 26 chpartnership 22, 70, 90 chrcconsistgrp 22, 79, 90 chrcrelationship 22, 7476 CIMOM 207208, 210 CLI 68, 72, 84, 104, 117, 157, 233, 235 commands 87, 89 cluster 34, 16, 2021, 2426, 4749, 103, 105, 175, 179, 191, 208, 210211 creation 25, 50 IP address 212 cluster partnership 25, 29, 47, 5051 clusters 2426, 4648, 200, 229 command line interface 4546 concepts 225226 configuration 3, 6, 18, 21, 27, 3132, 62, 104105, 174176, 209, 211212 connected 3133, 75, 83, 210, 214, 225 connected state 3436 connectivity 27, 40, 208 consistency 7, 1213, 3031, 33, 46, 53, 56, 92, 9495, 124, 128, 159, 177178, 185
E
e-mail xv empty state 38 error 2526, 33, 77, 93, 108, 111, 129, 138, 158, 210, 223, 233 error log 37 errors 26, 3536, 87, 158, 174, 210, 223 event 27, 32, 34, 108, 135 events 106108, 208, 233 excludes 27 extent 3, 103, 216 extents 216217
241
7574IX.fm
F
failed node 42 failover 8284, 207 file system 175, 181, 195 FlashCopy 2, 6, 1112, 9193, 117119, 177179, 207, 226227 bitmap 1718, 101102, 104, 120 commands 22, 94, 106 create 9192, 118, 130 Delete 108 indirection layer 101 mapping 15, 17, 20, 92, 94, 118120, 174 Modify 130 prepare 106107, 124, 137 source 16, 18, 20, 92, 94, 97, 117, 120, 179, 188 Start 124, 137, 178, 194 target 18, 92, 94, 122, 142, 148, 178179, 184 FlashCopy mapping 2, 15, 1718, 97, 118120, 227 FlashCopy mappings 15, 18, 103, 118, 123, 161 flexibility 22, 93, 103 fmtdisk 30 focal point 26 focal point node 26 foreground I/O latency 27 format 40, 223224
InconsistentStopped 35, 73, 8889 indirection 101 indirection layer 101 install 46 integrity 30 intercluster 2021, 2426, 4648, 157 intercluster link 2628 interval 8 intracluster 24, 27, 6162 iogrp 104, 162, 177, 193 IP address 212
L
latency 3, 16, 1819, 27, 39, 96, 101 LBA 27, 42, 101 LDM 188, 195 LDM database 188 limiting factor 8 Linux 200, 203204 local cluster 2627, 75 log 2122, 35, 37, 85, 95, 174175, 177, 210, 233234 logged 21, 25, 70, 235 Logical Block Address 101 Logical Disk Manager 188 Logical Volume Manager see LVM logins 26, 209 logs 21, 95, 208, 210 LU 204 LUN 1112, 216 LUNs 11, 8384 LVM 178 LVM data structures 179
G
Global xiv, 23, 6, 12, 15, 2325, 46, 4849, 93, 102, 105, 158, 174, 230231 Global Mirror 1517, 2324, 28, 46, 4849, 102 Global Mirror relationship 22 Globally Unique Identifier see GUID GM 158 grain 2, 17, 20, 4143, 100102, 120, 126 grain size 2, 17, 100101, 120 grains 23, 17, 41, 43, 96, 100, 102, 157, 169 granularity 101 GUI 21, 4547, 117118, 142, 208, 210, 235 GUID 188
M
management xiv, 3, 7, 41, 95, 217 mapping 23, 15, 1718, 87, 92, 94, 96, 118, 120121, 174175, 192, 227 maps 4142, 111, 216, 223 master 2830, 67, 7172, 159160, 202 master VDisk 28, 77 MDisk 34, 215217 showing 216 memory 2, 8, 17, 102, 104, 222 Metro 23, 6, 12, 15, 2324, 28, 46, 48, 54, 102, 105, 117, 157158, 200202, 230231 Metro Mirror 3, 17, 20, 2324, 30, 46, 48, 75, 102, 157158, 160, 200, 202203 Metro Mirror consistency group 85 Metro Mirror relationship 24, 83, 158, 203204 mirrored 28, 39, 188 mkpartnership 6970, 90 mkrcconsistgrp 7576, 90, 202 mkrcrelationship 2930, 7172, 90, 202, 226 mount 24, 177, 179, 181 MPIO 175
H
help xv, 82, 209 host configuration 184, 195 creating 55 definitions 184
I
I/O group 3, 24, 28, 38, 96, 102, 104, 126 ICAT 21, 234235 identical data 30 idling 28, 3536, 78, 89, 203 IdlingDisconnected 36, 89 importvg 179, 185 inconsistent 1415, 31, 3334, 93, 95 inconsistent stopped state 36 InconsistentCopying 36, 73, 88 InconsistentDisconnected 37, 89
N
new mapping 107, 118, 130, 132
242
7574IX.fm
NOCOPY 120, 139 node 3, 2627, 42, 175, 177, 183, 208209 failure 42 nodes 3, 2527, 96, 103, 204 non-redundant 26
rmrcconsistgrp 7980, 90 rmrcrelationship 75, 90
S
sample script 214 SAN xiiixiv, 1, 78, 10, 25, 46, 48, 68, 174 SAN Volume Controller xiv, 1, 72, 90 scripting 177, 202, 207, 214, 217 scripts 1, 96, 214215, 217 SDD 87, 89, 175, 191, 200 secondary 20, 2425, 27, 4849, 56, 93, 117, 157158, 174, 178179 secondary site 27 security 195, 208, 224 sequential 19, 121 serialization 20 shared 1617, 94, 102, 214 shells 220 site 20, 27, 33, 46, 8283, 157 SNIA xiv SNMP 35 Software 1 software upgrade 43 solution 67, 9, 142, 158, 207, 210211 sorting 54 source 23, 20, 84, 92, 9495, 118, 120121, 174175, 178, 227, 230 space 24, 78, 16, 41, 101102, 104, 120, 158, 196, 222 split 96, 101102, 224 SSH 21, 208209, 211 SSH client 208, 211 SSH keys 21 startrcrelationship 22, 2930, 7273, 90, 226 state 3, 1213, 24, 26, 28, 52, 5758, 96, 101102, 123125, 176, 178179, 223, 226 consistent 30, 34, 57 ConsistentDisconnected 37 ConsistentStopped 35 ConsistentSynchronized 35 empty 30, 38, 76, 79, 129, 165 idling 3637, 78 IdlingDisconnected 36 inconsistent 3536 InconsistentCopying 36 InconsistentDisconnected 37 InconsistentStopped 35 synchronized 24, 34, 5758, 107 state fragments 32 State overview 31 state transitions 109 states 25, 31, 34, 73, 75, 77, 106, 109, 134, 223224, 227 statistics 19 stoprcconsistgrp 22, 78, 8889, 203 stoprcrelationship 22, 30, 35, 74, 78, 90 storage virtualization xiv striped 121, 158159, 176177, 188 superuser 210, 227 SVC xiii, 13, 79, 2426, 4648, 91, 93, 96, 117, 120, Index
O
offline 40, 42, 8485, 104, 106107, 121 online xv, 4042, 7273, 76, 104, 107, 121, 125, 158, 174, 176, 191, 214215 ordering 12, 20, 33, 39, 111 overwritten 14, 33, 127
P
partnership 25, 29, 4749, 211212, 214 path offline 40 paths 43, 175 peak 21, 28 per cluster 16, 103, 105 performance 3, 7, 15, 19 Physical Volume Identifier see PVID planning 56, 9, 177, 202 plink 211213 point-in-time copy 34, 93, 158 policy 235 PPRC configuration limits 38 prepared state 178, 194 primary 20, 2425, 27, 56, 58, 64, 158160, 174, 178179 priority 37, 97, 120, 126, 145 private key 21, 209210, 212 production VDisk 28, 149, 151 productivity xv public key 21, 209210, 235 PuTTY 212214 PuTTY application 212 PuTTY session 212213 PVID 178 PVIDs 178179, 185
Q
quickly 8, 158 quorum disk 215216
R
recreatevg 178179 AIX 179 recreatevg command 179 Redbooks Web site 238 Contact us xv redundant 9, 26 registry 188 relationship 1214, 24, 26, 28, 4647, 52, 92, 96, 102, 130, 139, 146, 174, 179, 202, 226 remote cluster 20, 26, 50, 52, 55 removed 38, 75, 92, 233 restarting 120
243
7574IX.fm
145, 174175, 179, 207209 SVC cluster 26, 30, 35, 50, 52, 55, 103, 194, 200, 210212 SVC configuration 226 svcinfo 22, 40, 6970, 72, 104, 116, 158160, 175176, 178, 211212, 214 svcinfo lsmdiskextent 216 svctask 22, 6971, 104, 161163, 177179, 224, 233234 svctask mkfcmap 162, 178, 193194 switchrcconsistgrp 22, 81, 90 switchrcrelationship 22, 90 synchronization 15, 2930, 35, 93, 179, 202 Synchronized 3435 synchronized 15, 3031, 34, 5556, 66, 93, 158 synchronizing 29, 77
T
target 23, 20, 34, 8284, 9294, 118, 120, 122, 174, 177178, 227, 230 tasks 1, 61, 82, 94, 179, 194 time 2, 89, 11, 31, 3334, 56, 61, 75, 9294, 118, 120, 122, 174, 178, 181, 216, 222, 228 transitions 3537, 109
U
upgrade 43
V
VDisk 23, 7, 1011, 2425, 28, 52, 5455, 9294, 118, 120121, 173175, 215217 assigning 107 creating 28, 99, 121 information 7, 101, 178 working with 149 VGDA 178 VGID 178 virtualization xiv volume group 178179, 185 Volume Group Descriptor Area see VGDA Volume Group Identifier see VGID
W
Windows dynamic volumes 188 Windows 2000 Copy Services 188 limitations 188 Windows 2003 188, 191192 Write ordering 12, 33 write ordering 12, 39 write through mode 106 writes 89, 11, 25, 2728, 4849, 95, 107, 115, 175, 188, 222 write-through mode 12 WWPN 176, 191, 200
244
To determine the spine width of a book, you divide the paper PPI into the number of pages in the book. An example is a 250 page book using Plainfield opaque 50# smooth which has a PPI of 526. Divided 250 by 526 which equals a spine width of .4752". In this case, you would use the .5 spine. Now select the Spine width for the book and hide the others: Special>Conditional Text>Show/Hide>SpineSize(-->Hide:)>Set . Move the changed Conditional text settings to all files in your book by opening the book file with the spine.fm still open and File>Import>Formats the
Conditional Text Settings (ONLY!) to the book files.

7574spine.fm
245
(1.5 spine) 1.5<-> 1.998 789 <->1051 pages
(1.0 spine) 0.875<->1.498 460 <-> 788 pages
(0.5 spine) 0.475<->0.873 250 <-> 459 pages
(0.2spine) 0.17<->0.473 90<->249 pages
(0.1spine) 0.1<->0.169 53<->89 pages
To determine the spine width of a book, you divide the paper PPI into the number of pages in the book. An example is a 250 page book using Plainfield opaque 50# smooth which has a PPI of 526. Divided 250 by 526 which equals a spine width of .4752". In this case, you would use the .5 spine. Now select the Spine width for the book and hide the others: Special>Conditional Text>Show/Hide>SpineSize(-->Hide:)>Set . Move the changed Conditional text settings to all files in your book by opening the book file with the spine.fm still open and File>Import>Formats the
Conditional Text Settings (ONLY!) to the book files.

7574spine.fm
246
(2.5 spine) 2.5<->nnn.n 1315<-> nnnn pages
(2.0 spine) 2.0 <-> 2.498 1052 <-> 1314 pages
Back cover
Learn about the new Advanced Copy Services functionality Server integration examples Scripting examples
This book describes the new features that have been added with the release of the IBM System Storage SAN Volume Controller 4.2.1 code. Readers are expected to have an advanced knowledge of the SVC and SAN environment, and we recommend these books as background reading: IBM System Storage SAN Volume Controller, SG24-6423 Introduction to Storage Area Networks, SG24-5470 SAN Volume Controller: Best Practices and Performance Guidelines, SG24-7521 Using the SVC for Business Continuity, SG24-7371
INTERNATIONAL TECHNICAL SUPPORT ORGANIZATION
BUILDING TECHNICAL INFORMATION BASED ON PRACTICAL EXPERIENCE

IBM Redbooks are developed by the IBM International Technical Support Organization. Experts from IBM, Customers and Partners from around the world create timely technical information based on realistic scenarios. Specific recommendations are provided to help you implement IT solutions more effectively in your environment.
For more information: ibm.com/redbooks

SG24-7574-00 ISBN

IBM SVC Advanced Copyservices

Hochgeladen von

Dokumentinformationen

Originalbeschreibung:

Copyright

Verfügbare Formate

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

Verfügbare Formate

IBM SVC Advanced Copyservices

Hochgeladen von

Copyright:

Verfügbare Formate

Front cover

Draft Document for Review February 10, 2008 5:17 pm SG24-7574-00

SVC 4.2.1 Advanced Copy Services

Jon Tate Pall Thorir Beck Tom Jahn Dan Rumney

Draft Document for Review February 10, 2008 5:17 pm

Draft Document for Review February 10, 2008 5:17 pm

Draft Document for Review February 10, 2008 5:17 pm

Copyright IBM Corp. 2007. All rights reserved.

Draft Document for Review February 10, 2008 5:17 pm

SVC 4.2.1 Advanced Copy Services

Draft Document for Review February 10, 2008 5:17 pm

Copyright IBM Corp. 2007. All rights reserved.

Draft Document for Review February 10, 2008 5:17 pm

SVC 4.2.1 Advanced Copy Services

Draft Document for Review February 10, 2008 5:17 pm

Draft Document for Review February 10, 2008 5:17 pm

SVC 4.2.1 Advanced Copy Services

Draft Document for Review February 10, 2008 5:17 pm

Draft Document for Review February 10, 2008 5:17 pm

SVC 4.2.1 Advanced Copy Services

Draft Document for Review February 10, 2008 5:17 pm

The team that wrote this book

Figure 0-1 L-R, Dan, Pall, Tom , and Jon

Copyright IBM Corp. 2007. All rights reserved.

Draft Document for Review February 10, 2008 5:17 pm

Draft Document for Review February 10, 2008 5:17 pm

Become a published author

Draft Document for Review February 10, 2008 5:17 pm

SVC 4.2.1 Advanced Copy Services

Draft Document for Review February 10, 2008 5:17 pm

Introduction to SVC 4.2.1 Advanced Copy Services

Copyright IBM Corp. 2007. All rights reserved.

Draft Document for Review February 10, 2008 5:17 pm

1.1 What is new in FlashCopy

1.1.1 Incremental FlashCopy

1.1.2 Cascaded FlashCopy

1.1.3 Dynamically configurable replication services space

1.1.4 Grain size

SVC 4.2.1 Advanced Copy Services

Draft Document for Review February 10, 2008 5:17 pm

1.1.5 Bitmap space

1.1.6 Cleaning mode

1.2 What is new in Remote Copy

1.3 Other new functionality in 4.2.1

1.3.1 Increased storage addressing

1.3.2 Volume management

1.3.3 Managed disks test before added to MDisk Groups

Draft Document for Review February 10, 2008 5:17 pm

SVC 4.2.1 Advanced Copy Services

Draft Document for Review February 10, 2008 5:17 pm

7574 Planning for ACS implementation.fm

Planning for Advanced Copy Services implementation

Copyright IBM Corp. 2007. All rights reserved.

7574 Planning for ACS implementation.fm

Draft Document for Review February 10, 2008 5:17 pm

2.1 First questions

2.2 Reasons for copying data

2.3 Deciding what data to copy

SVC 4.2.1 Advanced Copy Services

Draft Document for Review February 10, 2008 5:17 pm

7574 Planning for ACS implementation.fm

2.4 Ensuring copied data sets are usable

2.4.1 Understanding caching