Cisco MDS 9000 Family SAN Volume Controller Configuration Guide

Cisco MDS 9000 Family
SAN Volume Controller

Configuration Guide
Cisco MDS SAN-OS Release 1.3
September, 2004
Corporate Headquarters
Cisco Systems, Inc.
170 West Tasman Drive
San Jose, CA 95134-1706
USA
http://www.cisco.com
Tel: 408 526-4000
800 553-NETS (6387)
Fax: 408 526-4100
Customer Order Number: DOC-7816119=

Text Part Number: 78-16119-01
THE SPECIFICATIONS AND INFORMATION REGARDING THE PRODUCTS IN THIS MANUAL ARE SUBJECT TO CHANGE WITHOUT NOTICE. ALL
STATEMENTS, INFORMATION, AND RECOMMENDATIONS IN THIS MANUAL ARE BELIEVED TO BE ACCURATE BUT ARE PRESENTED WITHOUT
WARRANTY OF ANY KIND, EXPRESS OR IMPLIED. USERS MUST TAKE FULL RESPONSIBILITY FOR THEIR APPLICATION OF ANY PRODUCTS.
THE SOFTWARE LICENSE AND LIMITED WARRANTY FOR THE ACCOMPANYING PRODUCT ARE SET FORTH IN THE INFORMATION PACKET THAT
SHIPPED WITH THE PRODUCT AND ARE INCORPORATED HEREIN BY THIS REFERENCE. IF YOU ARE UNABLE TO LOCATE THE SOFTWARE LICENSE
OR LIMITED WARRANTY, CONTACT YOUR CISCO REPRESENTATIVE FOR A COPY.
The Cisco implementation of TCP header compression is an adaptation of a program developed by the University of California, Berkeley (UCB) as part of UCB’s public
domain version of the UNIX operating system. All rights reserved. Copyright © 1981, Regents of the University of California.
NOTWITHSTANDING ANY OTHER WARRANTY HEREIN, ALL DOCUMENT FILES AND SOFTWARE OF THESE SUPPLIERS ARE PROVIDED “AS IS” WITH
ALL FAULTS. CISCO AND THE ABOVE-NAMED SUPPLIERS DISCLAIM ALL WARRANTIES, EXPRESSED OR IMPLIED, INCLUDING, WITHOUT
LIMITATION, THOSE OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OR ARISING FROM A COURSE OF
DEALING, USAGE, OR TRADE PRACTICE.
IN NO EVENT SHALL CISCO OR ITS SUPPLIERS BE LIABLE FOR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, OR INCIDENTAL DAMAGES, INCLUDING,
WITHOUT LIMITATION, LOST PROFITS OR LOSS OR DAMAGE TO DATA ARISING OUT OF THE USE OR INABILITY TO USE THIS MANUAL, EVEN IF CISCO
OR ITS SUPPLIERS HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.
CCIP, CCSP, the Cisco Arrow logo, the Cisco Powered Network mark, Cisco Unity, Follow Me Browsing, FormShare, and StackWise are trademarks of Cisco Systems, Inc.;
Changing the Way We Work, Live, Play, and Learn, and iQuick Study are service marks of Cisco Systems, Inc.; and Aironet, ASIST, BPX, Catalyst, CCDA, CCDP, CCIE,
CCNA, CCNP, Cisco, the Cisco Certified Internetwork Expert logo, Cisco IOS, the Cisco IOS logo, Cisco Press, Cisco Systems, Cisco Systems Capital, the Cisco Systems
logo, Empowering the Internet Generation, Enterprise/Solver, EtherChannel, EtherSwitch, Fast Step, GigaStack, Internet Quotient, IOS, IP/TV, iQ Expertise, the iQ logo, iQ
Net Readiness Scorecard, LightStream, MGX, MICA, the Networkers logo, Networking Academy, Network Registrar, Packet, PIX, Post-Routing, Pre-Routing, RateMUX,
Registrar, ScriptShare, SlideCast, SMARTnet, StrataView Plus, Stratm, SwitchProbe, TeleRouter, The Fastest Way to Increase Your Internet Quotient, TransPath, and VCO
are registered trademarks of Cisco Systems, Inc. and/or its affiliates in the U.S. and certain other countries.
All other trademarks mentioned in this document or Web site are the property of their respective owners. The use of the word partner does not imply a partnership relationship
between Cisco and any other company. (0304R)
Cisco MDS 9000 Family SAN Volume Conroller Configuration Guide

Copyright © 2003 Cisco Systems, Inc. All rights reserved.
New and Changed Information
Table 1 summarizes the new and changed features for the Cisco MDS 9000 Family SAN Volume
Controller Configuration Guide, and tells you where they are documented. If a feature has changed in
Release 1.3, a brief description of the change appears in the “Description” column, and that release is
shown in the “Changed in Release” column.
Table 1 Documented Features for the Cisco MDS 9000 Family SAN Volume Controller
Configuration Guide
Changed in Where
Feature Description Release Documented
Dual fabric CSM modules in combination with the 1.3(5m) Configuring a
configuration Inter-VSAN Routing (IVR) feature operate Dual Fabric
independently across two isolated fabrics. SAN
Environment
CSM modules The SVC functionality is only available when 1.3(5m) Getting Started
CSM modules are present in the switch.
Quorum Using the qorum command to overwrite the 1.3(5m) Managing
configuration system-assigned quorum disk and pick a Back-End
particular set of managed disks to be a quorum Storage
disk.
SVC configuration Introduction of SVC configuration using Cisco 1.3(1) This guide
MDS switches
Table 2 contains the history of the changes to the Cisco MDS 9000 Family SAN Volume Controller
Configuration Guide, Release 1.3. When the document is updated for the next release, these changes are
incorporated into the new revision and will no longer appear in this table.
Table 2 Documentation Changes for Cisco MDS 9000 Family SAN Volume Controller
Configuration Guide, Release 1.3
Date Description of Change Where Changed

11/21/2003 Document created ---
03/19/2004 Configuring iSCSI Hosts—sample See Chapter 6, “Configuring Hosts”
procedure added
Cisco MDS 9000 Family SAN Volume Controller Configuration Guide

78-16119-01, Cisco MDS SAN-OS Release 1.3 iii
New and Changed Information

iv 78-16119-01, Cisco MDS SAN-OS Release 1.3
C ON T E N T S
New and Changed Information iii
Preface ix
Audience ix
Organization ix
Document Conventions x
Related Documentation xi
Obtaining Documentation xi
Cisco.com xi
Documentation CD-ROM xii
Ordering Documentation xii
Documentation Feedback xii
Obtaining Technical Assistance xiii
Cisco.com xiii
Technical Assistance Center xiii
Cisco TAC Website xiv
Cisco TAC Escalation Center xiv
Obtaining Additional Publications and Information xiv
CHAPTER 1 SVC Product Overview 1-1
About SVC 1-1
SVC Features 1-2
CHAPTER 2 Getting Started 2-1
Preparing the Cisco MDS Switch 2-2
Setting Up the Cisco MDS Switch 2-2

Setting Up a New Cisco MDS Switch 2-2
Updating an Existing Cisco MDS Switch 2-3
SVC Role Authorization 2-4
Understanding SVC Terminology 2-5

Node 2-5
I/O group 2-5
Cluster 2-5
MDisk 2-5

78-16119-01, Cisco MDS SAN-OS Release 1.3 v
Contents
VDisk 2-5
MDisk Group 2-6
Host 2-6
Separating Hosts and Storage Devices 2-6
Verifying Interface Connectivity 2-8
Assigning VSAN Numbers 2-9
Multiple Initiators and Targets 2-9
CHAPTER 3 Creating and Managing Clusters 3-1
About CSM Nodes 3-2
About Clusters 3-2
Physical Topology 3-2
Selecting Nodes for a Cluster 3-3
Isolating Management Traffic 3-4
Creating a Cluster 3-5
Adding Nodes to Clusters 3-6
Verifying Nodes in a Cluster 3-7
Deleting a node from a Cluster 3-8

Deleting a Cluster 3-9
CHAPTER 4 Managing Back-End Storage 4-1
About Managed Disks 4-2
About MDisk Groups 4-2
Extents 4-2
MDisk Modes 4-2
Isolating Back-end Storage Traffic 4-3
Verifying Traffic Isolation 4-4
Configuring LUN Masking 4-4
Identifying MDisks 4-5
Configuring MDisk Groups 4-5
Configuring Quorum Disks 4-7
CHAPTER 5 Managing Virtual Disks 5-1
Virtualization Policies 5-2

Striped 5-2
Sequential 5-2

vi 78-16119-01, Cisco MDS SAN-OS Release 1.3
Contents
Image 5-2
Licensing Requirements 5-2
Configuring VDisks 5-3
CHAPTER 6 Configuring Hosts 6-1
About Hosts 6-1
Isolating Host Traffic 6-1
Creating Hosts 6-4
Mapping VDisks to Hosts 6-4
Configuring iSCSI Hosts in SVC 6-5
CHAPTER 7 Configuring Copy Services 7-1
Data Migration 7-2

Data Migration Application 7-2
Data Migration Operation 7-2
Migrating VDisks between MDisk groups 7-2
FlashCopy 7-4
FlashCopy Applications 7-4
FlashCopy Mapping 7-4
FlashCopy Consistency Groups 7-5
Starting the FlashCopy 7-6
Stopping FlashCopy 7-7
Remote Copy 7-8
Disaster Recovery 7-8
Remote Copy Relationships 7-8
Remote Copy Consistency Groups 7-8
Configuring Remote Copy 7-9
Starting Remote Copy 7-10
Stopping Remote Copy 7-11
Failover and Recovery Process 7-12
CHAPTER 8 Upgrading CSM Software 8-1
Upgrading Clusters 8-2

Cluster Upgrade Prerequisites 8-2
Recognizing Failure Situations 8-2
Cluster Upgrade Guidelines 8-3
Performing the Cluster Upgrade 8-3
Sample Cluster Upgrade 8-4

78-16119-01, Cisco MDS SAN-OS Release 1.3 vii
Contents
Automatic Upgrade during Node Addition 8-5
Upgrading in Service Mode 8-5

Sample Service Mode Upgrade 8-6
Upgrading the Switch Software 8-7

Performing the Switch Upgrade 8-7
Sample Switch Upgrade 8-7
Replacing CSMs 8-9
Prerequisites to Replacing a CSM 8-10
Replacing a CSM in the Same Slot 8-10
Replacing a CSM in a Different Slot 8-11
Post-Replacement Verification 8-12
CHAPTER 9 Configuring SPAN on SVC Interfaces 9-1
About SVC as a SPAN Source 9-2
Configuring SVC Interfaces as SPAN Sources 9-2
Using VSAN Filters to Specify N-Port Traffic 9-3
CHAPTER 10 Configuring a Dual Fabric SAN Environment 10-1
Overview 10-2
Basic SVC Requirements 10-2
Dual Fabric Prerequisites 10-2
Sample Configuration 10-2

viii 78-16119-01, Cisco MDS SAN-OS Release 1.3
Preface
This preface describes the audience, organization, and conventions of the Cisco MDS 9000 Family
Configuration Guide. It also provides information on how to obtain related documentation.
Audience
This Cisco MDS 9000 Family SAN Volume Controller User Guide is for system administrators who
configure and maintain network and storage systems using the Cisco CSM and the SAN-OS
Command-Line Interface (CLI).
The information contained in this guide assumes that you have a:
• Basic understanding of:
– multilayer switches and related hardware
– system, storage, and some IP network administration
• Working knowledge using a host-based volume management tool
– General understanding of volume management in a SAN environment
– Working knowledge of the MDS 9000 Family switch administration and configuration
Organization
This guide is organized as follows:
Chapter Title Description

Chapter 1 Product Overview Presents an overview of the Cisco MDS 9000
Family of multilayer switches and directors.
Chapter 2 Getting Started Explains the tasks required to set up and
configure each building block of your system
to run SVC for Switches.
Chapter 3 Creating and Managing Clusters Provides details on the cluster-creation
process.
Chapter 4 Managing Back-End Storage Provides details on managed disks, managed
disk groups, and the back-end traffic isolation
process.

78-16119-01, Cisco MDS SAN-OS Release 1.3 ix
Preface
Document Conventions
Chapter Title Description

Chapter 5 Managing Virtual Disks Explains the virtualization policies and
provides details on the virtual disk
configuration process.
Chapter 6 Configuring Hosts Provides detail on isolating host traffic,
creating hosts, and mapping VDisks to hosts.
Chapter 7 Configuring Copy Services Discusses the copy services, including data
migration, FlashCopy, and Remote Copy
available in Cisco MDS 9216 Switches and in
Cisco MDS 9500 Directors.
Chapter 8 Upgrading CSM Software Explains the procedures and processes for the
different upgrade possibilities when a Caching
Services Module (CSM) is used.
Document Conventions
Command descriptions use these conventions:
boldface font Commands and keywords are in boldface.

italic font Arguments for which you supply values are in italics.
[ ] Elements in square brackets are optional.
[x|y|z] Optional alternative keywords are grouped in brackets and separated by
vertical bars.
Screen examples use these conventions:

screen font Terminal sessions and information the switch displays are in screen font.
boldface screen font Information you must enter is in boldface screen font.
italic screen font Arguments for which you supply values are in italic screen font.
< > Nonprinting characters, such as passwords are in angle brackets.
[ ] Default responses to system prompts are in square brackets.
!, # An exclamation point (!) or a pound sign (#) at the beginning of a line of code
indicates a comment line.
This document uses the following conventions:
Note Means reader take note. Notes contain helpful suggestions or references to material not covered in the
manual.
Caution Means reader be careful. In this situation, you might do something that could result in equipment
damage or loss of data.

x 78-16119-01, Cisco MDS SAN-OS Release 1.3
Preface
Related Documentation
Related Documentation
The documentation set for the Cisco MDS 9000 Family includes the following documents:
• Cisco MDS 9000 Family Release Notes for Cisco MDS SAN-OS Releases
• Cisco MDS 9000 Family Interoperability Support Matrix
• Cisco MDS SAN-OS Release Compatibility Matrix for IBM SAN Volume Controller Software for
Cisco MDS 9000
• Regulatory Compliance and Safety Information for the Cisco MDS 9000 Family
• Cisco MDS 9500 Series Hardware Installation Guide
• Cisco MDS 9216 Switch Hardware Installation Guide
• Cisco MDS 9100 Series Hardware Installation Guide
• Cisco MDS 9000 Family Configuration Guide
• Cisco MDS 9000 Family Command Reference
• Cisco MDS 9000 Family Fabric Manager Configuration Guide
• Cisco MDS 9000 Family SAN Volume Controller Configuration Guide
• Cisco MDS 9000 Family MIB Quick Reference
• Cisco MDS 9000 Family CIM Programming Reference Guide
• Cisco MDS 9000 Family System Messages Guide
• Cisco MDS 9000 Family Troubleshooting Guide
• Cisco MDS 9000 Family Port Analyzer Adapter 2 Installation and Configuration Note
• Cisco MDS 9000 Family Port Analyzer Adapter Installation and Configuration Note
For information on IBM TotalStorage SAN Volume Controller Storage Software for the Cisco MDS
9000 Family, refer to the IBM TotalStorage Support website:
http://www.ibm.com/storage/support/2062-2300/
Obtaining Documentation
Cisco provides several ways to obtain documentation, technical assistance, and other technical
resources. These sections explain how to obtain technical information from Cisco Systems.
Cisco.com
You can access the most current Cisco documentation on the World Wide Web at this URL:
http://www.cisco.com/univercd/home/home.htm
You can access the Cisco website at this URL:
http://www.cisco.com
International Cisco websites can be accessed from this URL:
http://www.cisco.com/public/countries_languages.shtml

78-16119-01, Cisco MDS SAN-OS Release 1.3 xi
Preface
Obtaining Documentation
Documentation CD-ROM
Cisco documentation and additional literature are available in a Cisco Documentation CD-ROM
package, which may have shipped with your product. The Documentation CD-ROM is updated regularly
and may be more current than printed documentation. The CD-ROM package is available as a single unit
or through an annual or quarterly subscription.
Registered Cisco.com users can order a single Documentation CD-ROM (product number
DOC-CONDOCCD=) through the Cisco Ordering tool:
http://www.cisco.com/en/US/partner/ordering/ordering_place_order_ordering_tool_launch.html
All users can order monthly or quarterly subscriptions through the online Subscription Store:
http://www.cisco.com/go/subscription
Ordering Documentation
You can find instructions for ordering documentation at this URL:
http://www.cisco.com/univercd/cc/td/doc/es_inpck/pdi.htm
You can order Cisco documentation in these ways:
• Registered Cisco.com users (Cisco direct customers) can order Cisco product documentation from
the Networking Products MarketPlace:
http://www.cisco.com/en/US/partner/ordering/index.shtml
• Nonregistered Cisco.com users can order documentation through a local account representative by
calling Cisco Systems Corporate Headquarters (California, U.S.A.) at 408 526-7208 or, elsewhere
in North America, by calling 800 553-NETS (6387).
Documentation Feedback
You can submit comments electronically on Cisco.com. On the Cisco Documentation home page, click
Feedback at the top of the page.
You can e-mail your comments to mdsfeedback-doc@cisco.com.
You can submit comments by using the response card (if present) behind the front cover of your
document or by writing to the following address:
Cisco Systems
Attn: Customer Document Ordering
170 West Tasman Drive
San Jose, CA 95134-9883
We appreciate your comments.

xii 78-16119-01, Cisco MDS SAN-OS Release 1.3
Preface
Obtaining Technical Assistance
Obtaining Technical Assistance

Note If you purchased this product through a Cisco reseller, contact the reseller directly for technical support.
If you purchased this product directly from Cisco, contact Cisco Technical Support at this URL:
http://www.cisco.com/warp/public/687/Directory/DirTAC.shtml
Cisco provides Cisco.com, which includes the Cisco Technical Assistance Center (TAC) website, as a
starting point for all technical assistance. Customers and partners can obtain online documentation,
troubleshooting tips, and sample configurations from the Cisco TAC website. Cisco.com registered users
have complete access to the technical support resources on the Cisco TAC website, including TAC tools
and utilities.
Cisco.com
Cisco.com offers a suite of interactive, networked services that let you access Cisco information,
networking solutions, services, programs, and resources at any time, from anywhere in the world.
Cisco.com provides a broad range of features and services to help you with these tasks:
• Streamline business processes and improve productivity
• Resolve technical issues with online support
• Download and test software packages
• Order Cisco learning materials and merchandise
• Register for online skill assessment, training, and certification programs
To obtain customized information and service, you can self-register on Cisco.com at this URL:
http://tools.cisco.com/RPF/register/register.do
Technical Assistance Center

The Cisco TAC is available to all customers who need technical assistance with a Cisco product,
technology, or solution. Two types of support are available: the Cisco TAC website and the Cisco TAC
Escalation Center. The type of support that you choose depends on the priority of the problem and the
conditions stated in service contracts, when applicable.
We categorize Cisco TAC inquiries according to urgency:
• Priority level 4 (P4)—You need information or assistance concerning Cisco product capabilities,
product installation, or basic product configuration. There is little or no impact to your business
operations.
• Priority level 3 (P3)—Operational performance of the network is impaired, but most business
operations remain functional. You and Cisco are willing to commit resources during normal business
hours to restore service to satisfactory levels.
• Priority level 2 (P2)—Operation of an existing network is severely degraded, or significant aspects
of your business operations are negatively impacted by inadequate performance of Cisco products.
You and Cisco will commit full-time resources during normal business hours to resolve the situation.

78-16119-01, Cisco MDS SAN-OS Release 1.3 xiii
Preface
Obtaining Additional Publications and Information
• Priority level 1 (P1)—An existing network is “down,” or there is a critical impact to your business
operations. You and Cisco will commit all necessary resources around the clock to resolve the
situation.
Cisco TAC Website

The Cisco TAC website provides online documents and tools to help troubleshoot and resolve technical
issues with Cisco products and technologies. To access the Cisco TAC website, go to this URL:
http://www.cisco.com/tac
All customers, partners, and resellers who have a valid Cisco service contract have complete access to
the technical support resources on the Cisco TAC website. Some services on the Cisco TAC website
require a Cisco.com login ID and password. If you have a valid service contract but do not have a login
ID or password, go to this URL to register:
http://tools.cisco.com/RPF/register/register.do
If you are a Cisco.com registered user, and you cannot resolve your technical issues by using the Cisco
TAC website, you can open a case online at this URL:
http://www.cisco.com/tac/caseopen
If you have Internet access, we recommend that you open P3 and P4 cases online so that you can fully
describe the situation and attach any necessary files.
Cisco TAC Escalation Center

The Cisco TAC Escalation Center addresses priority level 1 or priority level 2 issues. These
classifications are assigned when severe network degradation significantly impacts business operations.
When you contact the TAC Escalation Center with a P1 or P2 problem, a Cisco TAC engineer
automatically opens a case.
To obtain a directory of toll-free Cisco TAC telephone numbers for your country, go to this URL:
http://www.cisco.com/warp/public/687/Directory/DirTAC.shtml
Before calling, please check with your network operations center to determine the Cisco support services
to which your company is entitled: for example, SMARTnet, SMARTnet Onsite, or Network Supported
Accounts (NSA). When you call the center, please have available your service agreement number and
your product serial number.

Information about Cisco products, technologies, and network solutions is available from various online
and printed sources.
• The Cisco Product Catalog describes the networking products offered by Cisco Systems, as well as
ordering and customer support services. Access the Cisco Product Catalog at this URL:
http://www.cisco.com/en/US/products/products_catalog_links_launch.html
• Cisco Press publishes a wide range of networking publications. Cisco suggests these titles for new
and experienced users: Internetworking Terms and Acronyms Dictionary, Internetworking
Technology Handbook, Internetworking Troubleshooting Guide, and the Internetworking Design
Guide. For current Cisco Press titles and other information, go to Cisco Press online at this URL:
http://www.ciscopress.com

xiv 78-16119-01, Cisco MDS SAN-OS Release 1.3
Preface
• Packet magazine is the Cisco quarterly publication that provides the latest networking trends,
technology breakthroughs, and Cisco products and solutions to help industry professionals get the
most from their networking investment. Included are networking deployment and troubleshooting
tips, configuration examples, customer case studies, tutorials and training, certification information,
and links to numerous in-depth online resources. You can access Packet magazine at this URL:
http://www.cisco.com/go/packet
• iQ Magazine is the Cisco bimonthly publication that delivers the latest information about Internet
business strategies for executives. You can access iQ Magazine at this URL:
http://www.cisco.com/go/iqmagazine
• Internet Protocol Journal is a quarterly journal published by Cisco Systems for engineering
professionals involved in designing, developing, and operating public and private internets and
intranets. You can access the Internet Protocol Journal at this URL:
http://www.cisco.com/en/US/about/ac123/ac147/about_cisco_the_internet_protocol_journal.html
• Training—Cisco offers world-class networking training. Current offerings in network training are
listed at this URL:
http://www.cisco.com/en/US/learning/le31/learning_recommended_training_list.html

78-16119-01, Cisco MDS SAN-OS Release 1.3 xv
Preface

xvi 78-16119-01, Cisco MDS SAN-OS Release 1.3
C H A P T E R 1
SVC Product Overview
The Cisco MDS 9000 Family SAN Volume Controller User Guide provides information on how to set up
and configure the SAN Volume Controller (SVC) storage software using a Cisco MDS 9000 Family
Caching Services Module (CSM).
For more information about IBM TotalStorageTM SAN Volume Controller (SVC) Storage Software, refer
to the IBM TotalStorage SAN Volume Controller Storage Software for Cisco MDS 9000 Configuration
Guide or the IBM TotalStorage SAN Volume Controller Storage Software for Cisco MDS 9000
Command-Line Interface User's Guide.
This chapter describes the SVC and explains the supported features. It includes the following sections:
• About SVC, page 1-1
• SVC Features, page 1-2
About SVC
IBM and Cisco combine the IBM TotalStorageTM SAN Volume Controller (SVC) Storage Software with
the Cisco MDS 9000 Caching Services Module (CSM) to reduce complexity and to reduce the cost of
managing SAN-based storage. This solution implements a cache-based, clustered architecture and
provides a highly available, scalable alternative that is necessary in today’s demanding storage
environments.
The combined SVC storage software along with the Cisco MDS 9000 Series CSM is delivered as a
feature of the Cisco MDS 9000 Family. The SVC software runs on a clustered pair of CSMs within the
switch.
Based on virtualization technology, this solution is designed to support a virtualized pool of storage from
the storage subsystems attached to a SAN. It manges the combined storage volumes from a central point,
avoids downtime for planned outages, increases capacity utilization, and implements copy services from
a single license across multiple storage devices.
This storage pool helps tap unused storage capacity by increasing efficiency. It is designed as an
integrated solution supporting high performance and continuous availability in open-system
environments. Storage volumes are represented to applications as virtual disks (VDisks). These VDisks
are created from the pool of managed disks residing behind the storage engines. Storage administrators
can scale performance by adding storage engines and scale capacity by adding disks to the managed
storage pool.

78-16119-01, Cisco MDS SAN-OS Release 1.3 1-1
Chapter 1 SVC Product Overview
SVC Features
The IBM—Cisco SVC storage solution offers the following benefits and advantages:
• Reduces complexity
• Lowers the cost of managing SAN-based storage
• Creates a single pool of storage from disparate storage devices to increase capacity utilization
• Easier to manage
• Implementing a cache-based, clustered architecture to provide a highly available solution.
• Provides the scalability and performance required in today’s demanding storage environments.
• Provides simple migration of storage.
• Provides a single set of copy services.
SVC Features
This section explains the features provided by the combined SVC storage software and the Cisco MDS
9000 Series CSM solution.
• A central point for volume management control
Through virtualization, the Cisco MDS 9000 Family’s SAN-OS software helps create pools of
managed disks spanning multiple storage subsystems. These managed disks are mapped to virtual
disks used by server applications thus making better use of existing storage. This simple interface
incorporates the Storage Management Initiative Specification (SMIS) application programming
interface (API), and further demonstrates IBM’s focus on open standards.
• Dynamic data migration
The Cisco MDS 9000 SAN-OS software includes a dynamic data-migration function that helps
administrators migrate storage from one device to another, without taking it offline. This allows
administrators to reallocate and scale storage capacity without disrupting applications. And the
solution supports both local area network (LAN) free and server-free backups while a clustered
configuration designed to support high availability allows for non-disruptive software upgrades.
SVC for Cisco MDS 9000 also leverages the IBM TotalStorage Enterprise Storage Server ™
multipathing software.
• Improved resource utilization
This solution enables more efficient use of personnel and technology resources. It helps increase
administrator productivity by empowering central management of volumes under disparate storage
controllers from a single user interface. It also increases the amount of available storage capacity by
pooling storage across multiple devices. Designed to manage up to two petabytes (PB) of total
usable storage capacity, SVC for Cisco MDS 9000 will support even higher performance by adding
storage engine pairs. All storage engines within a cluster jointly manage the entire capacity of a
storage pool.
• Advanced copy services
With conventional SAN disk arrays, copy operations are limited to in-box or like-box-to-like-box
environments. But the SVC software moves copy services from individual storage controllers to the
SAN. Administrators can apply copy services across disparate storage devices within the network.
Advanced copy services —such as FlashCopy ® and Peer-to-Peer Remote Copy (PPRC)— are
supported across the managed storage.

1-2 78-16119-01, Cisco MDS SAN-OS Release 1.3
SVC Features
• Cisco MDS 9000 CSM

The CSM integrates two high performance processing nodes which, when combined with the Cisco
MDS 9000 SAN-OS software, delivers network hosted virtualization and replication services. Each
CSM includes 8 GB of local cache used to hold recently accessed data blocks. On-board dual
redundant batteries and hard disk drives protect cached data in the event of a power failure. To
further ensure data availability and integrity, nodes are paired with nodes on other CSMs in high
availability clusters.
• Distributed cache-based virtualization
The Cisco MDS 9000 CSM virtualization architecture overcomes the inherent bottlenecks
associated with other virtualization architectures. Virtualization performance can be easily scaled to
the level required by adding CSMs. Application I/O response time is improved over other
virtualization architectures through the use of local caching of disk blocks. Because Cisco MDS
9000 virtualization is switch-based, any host can access any virtual volume from anywhere in the
fabric, independent of the host’s attachment point in the SAN. In addition to virtualization and
replication services, the Cisco MDS 9000 CSM leverages all of the advanced SAN-OS features
available on the Cisco MDS 9000 platform, simplifying security, diagnostics and management.
• Fabric-based virtualization using intelligent networking services
Cisco MDS 9000 fabric-based virtualization provides a level of integration with intelligent SAN
services unavailable to host or virtualization appliance-based solutions. The Cisco MDS 9000
platform with integrated CSM delivers the intelligence and advanced features required to make
multilayer intelligent storage area networks a reality including hardware-enabled innovations
designed to dramatically improve scalability, availability, security, and manageability of storage
networks, resulting in increased utility and lower total cost of ownership (TCO).
• Virtual SAN
Virtual SANs (VSANs) allow more efficient SAN utilization by creating hardware-based isolated
environments within a single SAN fabric. Each VSAN maintains its own fabric services for added
scalability and resilience. Zoning within a SAN allows additional configuration flexibility. VSANs
allow the cost of SAN infrastructure to be shared among more users, while assuring absolute
segregation and security of traffic and retaining independent control of configuration on a
VSAN-by-VSAN basis. VSANs provide a protective barrier between application hosts and physical
storage, enhancing data integrity in a virtualized storage environment.
• Comprehensive security
The Cisco CSM and the SAN-OS software integrate seamlessly to apply extensive security measures
at all possible points of attack. SSH, RADIUS, SNMPv3, and role-based access control are used to
block unauthorized access. To guard against compromising control traffic, Fibre Channel Security
Protocol (FC-SP) provides confidentiality, data origin authentication, and connectionless integrity
across the fabric. Data plane traffic is secured with VSANs, guaranteeing segregation of traffic
across shared fabrics, and with zoning to satisfy traffic segregation requirements within a VSAN.
Hardware-based ACLs provide further granularity for advanced security options.
• High availability
Like all other Cisco MDS 9000 Family modules, the CSM is hot-swappable and fully integrates into
the Cisco MDS 9000 high availability architecture. The Cisco MDS 9000 SAN-OS software
architecture offers an unparalleled level of availability including automatic restart of failed
supervisor processes and fabric level availability via Cisco PortChannel capability which allows 16
physical links to be aggregated into one logical interface. The logical interface remains active in the
event of a port, ASIC, or module failure, and can sustain the failure of any physical link without
causing a reset. Additionally, Fabric Shortest Path First (FSPF) multipathing provides the

SVC Features
intelligence to load balance across up to 16 equal cost paths and, in the event of a switch failure, to
dynamically reroute traffic. When deployed in clustered pairs and combined with SAN-OS software,
availability is extended to the volume level ensuring maximum uptime.
• Management options
The Cisco MDS 9000 Family CSM provides three principal modes of management of your virtual
storage environment: Cisco MDS 9000 Family Command Line Interface (CLI), IBM’s SVC for
Cisco MDS 9000 CLI, and IBM’s ICAT management GUI. For users who prefer a common interface
for both SAN and Volume management, the Cisco SAN-OS CLI includes the full suite of
capabilities necessary to manage your virtual storage environment from the SAN-OS command line.

C H A P T E R 2
Getting Started
The chapter explains the tasks required to set up and configure each building block of your system to run
SVC for Switches. You will be performing these tasks using one or more Cisco MDS 9000 Family
switches for setup and configuration. You may wish to have your Cisco and IBM documentation handy
for reference to more detailed procedures.
This chapter includes the following sections:
• Preparing the Cisco MDS Switch, page 2-2
• Setting Up the Cisco MDS Switch, page 2-2
• Understanding SVC Terminology, page 2-5
• Separating Hosts and Storage Devices, page 2-6
• Verifying Interface Connectivity, page 2-8
• Assigning VSAN Numbers, page 2-9
• Multiple Initiators and Targets, page 2-9

Chapter 2 Getting Started
Preparing the Cisco MDS Switch
Preparing the Cisco MDS Switch

To prepare for both hardware and software setup, be sure you have the following in place before getting
started:
• Installed and functioning Cisco MDS 9000 Family switch fabric. Each Cisco MDS 9000 Family
switch needs a switch IP (mgmt0, one for each switch).
Note Refer to the Cisco MDS [9216 Switch or the Cisco MDS 9500] Series Hardware Installation
Guides, and the Cisco MDS 9000 Family Configuration Guide.
• One IP address for each SVC cluster for management purposes. This should be in the same subnet
as the management IP address of the switches across which the cluster spans.
Setting Up the Cisco MDS Switch

This section explains the process to set up a new, or update an existing MDS 9000 Family switch in
preparation for the SVC.
Before setting up your hardware, ensure you have correctly installed the Cisco MDS chassis and its
components as specified in “Chapter 2: Installing the Cisco MDS 9000 Family Switch” in the Cisco MDS
[9216 Switch or the Cisco MDS 9500] Series Hardware Installation Guides.
Caution Be sure to save your work frequently using the copy running-config startup-config command.
Warning The SVC functionality is only available when CSM modules are present in the switch. If you issue a
copy running startup command when all CSM modules are removed (or powered-down), then SVC
supervisor configurations associated with the CSM module, including world wide name (WWNs), may
be discarded.
Setting Up a New Cisco MDS Switch

To set up a new Cisco MDS 9000 Family switch, follow these steps.
Step 1 Follow instructions for preinstallation, installing the chassis in the rack, grounding the chassis, installing
modules, installing CompactFlash cards, installing Power Supplies, and installing the Fan Assembly as
specified in the Cisco MDS [9216 Switch or the Cisco MDS 9500] Series Hardware Installation Guides.
Step 2 Connect to the supervisor module as specified in “Chapter 3: Connecting the Cisco MDS 9000 Family
Switch” in the Cisco MDS [9216 Switch or the Cisco MDS 9500] Series Hardware Installation Guides.
Step 3 Login to the Cisco MDS 9000 Family switch using the Cisco MDS 9000 Family CLI.
Step 4 Configure the switch as specified in “Chapter 3: Initial Configuration” in the Cisco MDS 9000 Family
Configuration Guide.
a. Perform the initial setup routine.
b. Assign a switch name.

Setting Up the Cisco MDS Switch
c. Access the switch.

After reviewing the default configuration, you can change it or perform other configuration or
management tasks. The initial setup can only be performed at the Command-Line Interface (CLI).
However, you can continue to configure other software features, or access the switch after initial
configuration by using either the CLI, or the Fabric Manager GUIs.
Tip The Cisco Fabric Manager provides an alternative to the CLI for most switch configuration commands.
To use the Cisco MDS 9000 Fabric Manager, refer to the Cisco MDS 9000 Family Fabric Manager User
Guide. To use the CLI, refer to the Cisco MDS 9000 Family Configuration Guide.
Note The rest of this procedure uses the Cisco CLI to configure the switch.
Step 5 Verify the module status as specified in “Chapter 3: Initial Configuration” in the Cisco MDS 9000 Family
Configuration Guide.
Step 6 Configure the management port as specified in “Chapter 3: Initial Configuration” in the Cisco MDS 9000
Family Configuration Guide.
Updating an Existing Cisco MDS Switch
Note These steps assume you have a remote FTP, TFTP, SFTP, or SCP server that contains the SVC image.
Caution Be sure to save your work frequently using the copy running-config startup-config command.
To update a existing Cisco MDS switch, follow these steps.
Step 1 Login to the Cisco MDS switch using the Cisco MDS 9000 Family CLI.
Note Refer to the www.cisco.com web site to verify compatibility issues, or the appropriate MDS 9000 Family
release notes to ensure your system and setup meets the minimum requirements, or images will not
install properly.
Step 2 Install the new image on each SVC node in each module in the fabric. Each module has two nodes.
switch# install module 2 node 1 image svc-system
ftp://171.71.188.111/m9000-ek9-csm-svc-mz.1.3.1.bin
For ftp://171.71.188.111, please enter user name:user
For ftp://user@171.71.188.111, please enter password:
SVC reimage going on. Please wait
m9000-ek9-csm-svc-mz.1.3.1.bin 100% |*****************************| 45408 KB 00:53
svc 2/1 software reimage succeeded
switch# install module 2 node 2 image svc-system

ftp://171.71.188.111/m9000-ek9-csm-svc-mz.1.3.1.bin
For ftp://171.71.188.111, please enter user name:user

SVC Role Authorization
For ftp://user@171.71.188.111, please enter password:

SVC reimage going on. Please wait
m9000-ek9-csm-svc-mz.1.3.1.bin 100% |*****************************| 45408 KB 00:55
svc 2/2 software reimage succeeded
All prior information on the node is lost with each install.

Step 3 Verify that the CSM node booted successfully, by issuing the show module command.
switch# show module
Mod Ports Module-Type Model Status
--- ----- ------------------------------- ------------------ ------------
2 0 Caching Services Module DS-X9560-SMC ok <---------- CSM
4 8 IP Storage Services Module DS-X9308-SMIP ok
5 0 Supervisor/Fabric-1 DS-X9530-SF1-K9 active *
6 0 Supervisor/Fabric-1 DS-X9530-SF1-K9 ha-standby
Mod Sw Hw World-Wide-Name(s) (WWN)

--- ----------- ------ --------------------------------------------------
2 1.3(1) 0.3 --
4 1.3(1) 0.206 20:c1:00:05:30:00:a7:9e to 20:c8:00:05:30:00:a7:9e
5 1.3(1) 0.0 --
6 1.3(1) 0.0 --
Mod Application Image Description Application Image Version

-------- ----------------------------- -------------------------
2 svc-node1 1.3(1) <-------Node one on the CSM
2 svc-node2 1.3(1) <-------Node two on the CSM
Mod MAC-Address(es) Serial-Num

--- -------------------------------------- ----------
2 00-05-30-00-93-e2 to 00-05-30-00-93-e6 JAB06xxxx10
4 00-05-30-00-9d-de to 00-05-30-00-9d-ea JAB064605aa
5 00-05-30-00-52-f2 to 00-05-30-00-52-f6
6 00-05-30-00-53-3e to 00-05-30-00-53-42
* this terminal session
Tip The initial setup can only be performed at the CLI. You can continue to configure other software
features, or access the switch after initial configuration by using either the CLI or the Device Manager
and Fabric Manager GUIs. To use the Cisco MDS 9000 Fabric Manager, refer to the Cisco MDS 9000
Family Fabric Manager User Guide.
SVC Role Authorization

By default, two roles exist in all Cisco MDS switches:
• SVC administrator (svc-admin)— Has permission to view the entire configuration and make
SVC-specific configuration changes within the switch(svc) prompt.
• SVC operator (svc-operator)—Has permission to view the entire configuration. The operator
cannot make any configuration changes.
The two default roles cannot be changed or deleted.

Understanding SVC Terminology
Understanding SVC Terminology

This section defines frequently-used SVC terms that are necessary to proceed with the configuration
process (see Figure 2-1).
Figure 2-1 Graphical Representation of SVC Terms
Node 1 CSM 1 Node 2

I/O Group I/O Group
Node 1 CSM 2 Node 2
Vdisk 1 Vdisk 2 Vdisk 3
Mdisk group Mdisk group
105298
Node
An instance of SVC that is running in the CSM. A CSM consists of two completely independent nodes.
Each node is represented by an interface.
I/O group
SVC nodes are always used in pairs. A pair of SVC nodes is called an I/O group. The nodes in a given
I/O group must reside in two separate CSMs. An I/O group acts as a storage controller in the fabric.
Cluster
A cluster can contain multiple I/O groups. All nodes in the cluster must run the same SVC image version.
The IP address for a cluster must be assigned in the same subnet as the management interface.
MDisk
A representation of back end storage. Each cluster is configured so all the nodes in a cluster see the same
set of MDisks.
VDisk
A virtual representation of a LUN that is exposed by the cluster to the hosts in a SAN. Each VDisk is
independently associated with a single I/O group.

Separating Hosts and Storage Devices
MDisk Group
A set of MDisks form a MDisk group. Storage for a VDisk originates from MDisks in a single MDisk
group.
Host
One of more initiator Fibre Channel ports (pWWNs) form a host. A host is mapped to one or more
VDisks. Hosts cannot directly access a MDisk.

In order for SVC to virtualize the back-end storage to hosts, you must ensure that the hosts do not
directly access the storage. You can separate hosts or disks using the concept of VSANs. The VSAN
feature is specific to Cisco MDS 9000 Family switches. Alternatively, you can also use zones to separate
hosts or disks.
Refer to the Cisco MDS 9000 Family Configuration Guide for further information on VSANs or zones.
Each SVC interface is assigned a nWWN. A SVC interface consists of three N-ports:
• Target N-ports: used for traffic between SVC nodes and hosts.
• Initiator N-ports: used for traffic between SVC nodes and disks
• Management N-ports: used for traffic to and from other SVC nodes.
Each N-port within an SVC interface is assigned a port World Wide Name (pWWN). The pWWNs and
node World Wide Names (nWWNs) are preserved across switch reboots (see Figure 2-2).
Figure 2-2 Logical Representation of an SVC Interface
VSAN 1 VSAN 2 VSAN 3

Node 1
Host H Node 2
Node 3
Host T Node 4
Node 5
Node 6
Target Mgmt Initiator
N-Port B N-Port C N-Port A
105299
Node 2
In Figure 2-3, provides a logical view of four SVC nodes in a SAN. These nodes are configured so the
hosts do not have direct access to the disks.

By default, all N-ports reside in VSAN 1. You must explicitly remove them when necessary.
Figure 2-3 Logical Representation of a SAN
Zone 2
Zone 1
Host VSAN
VSAN 1
Target Target Target Target

N-Port N-Port N-Port N-Port
Mgmt Mgmt Mgmt Mgmt
Node 3 Node 1 Management VSAN Node 2 Node 4
VSAN 2
Initiator Initiator Initiator Initiator
Disk VSAN
VSAN 3
105303
To configure a SVC interface and N-port VSANs in a Cisco MDS switch, follow these steps:
Command Purpose
Step 1 switch# config t Enters configuration mode.
switch(config)#
Step 2 switch(config)# interface svc 2/1 Enters the configuration mode for SVC interface 2/1.
switch(config-if)#
Step 3 switch(config-if)# initiator vsan 3 Configures the initiator VSAN 3 for disks.
switch(config-if)# no initiator vsan 1 Removes VSAN 1
Step 4 switch(config-if)# target vsan 1 Configures the target VSAN 1 for hosts.
Step 5 switch(config-if)# mgmt vsan 2 Configures the management VSAN 2 for nodes.
Step 6 switch(config-if)# no mgmt vsan 1 Removes VSAN 1.
Step 7 switch(config-if)# no shutdown Enables the SVC interface.

Verifying Interface Connectivity
Verifying Interface Connectivity

This section lists the show commands that help verify the SVC interface state and the N-port association
with the SVC interface.
Example 2-1 Displays Information for A Specified SVC Interface
switch# show interface svc 2/1

svc2/1 is up
Node WWN is 2e:ab:00:05:30:00:1a:e0
Fabric WWN is 20:01:00:05:30:00:1a:de
Target N-port WWN is 2e:a5:00:05:30:00:1a:e0, vsan is 1, FCID is 0xe80003
Initiator N-port WWN is 21:2e:00:05:30:00:00:21, vsan is 3, FCID is 0xea0004
Mgmt N-port WWN is 2f:af:00:05:30:00:1a:e0, vsan is 2, FCID is 0xe80000
5 minutes input rate 2392 bits/sec, 299 bytes/sec, 0 frames/sec
5 minutes output rate 2240 bits/sec, 280 bytes/sec, 0 frames/sec
272 frames input, 89764 bytes
0 discards, 0 errors
232 frames output, 84176 bytes
Example 2-2 Displays N-port Connections for a Specified SVC Interface
switch# show svc session svc 2/1

svc2/1:
pWWN 21:01:00:e0:8b:31:20:31, nWWN 20:01:00:e0:8b:31:20:31, FCID 0xe80200
pWWN 21:00:00:e0:8b:11:29:31, nWWN 20:00:00:e0:8b:11:29:31, FCID 0xe80300
pWWN 50:05:07:63:00:c8:9c:f9, nWWN 50:05:07:63:00:c0:9c:f9, FCID 0xea0000
pWWN 50:05:07:63:00:c8:9c:fa, nWWN 50:05:07:63:00:c0:9c:fa, FCID 0xea0001
pWWN 2f:b9:00:05:30:00:1a:e0, nWWN 2f:b7:00:05:30:00:1a:e0, FCID 0xe80002
pWWN 2f:ba:00:05:30:00:1a:e0, nWWN 2e:b3:00:05:30:00:1a:e0, FCID 0xe80003
pWWN 2f:b8:00:05:30:00:1a:e0, nWWN 2e:ac:00:05:30:00:1a:e0, FCID 0xe80001
Example 2-3 Displays the FCNS Database
switch# show fcns database

VSAN 1:
--------------------------------------------------------------------------
FCID TYPE PWWN (VENDOR) FC4-TYPE:FEATURE
--------------------------------------------------------------------------
0xe80000 N 2e:a8:00:05:30:00:1a:e0 (Cisco) scsi-fcp:target svc
0xe80003 N 2e:a5:00:05:30:00:1a:e0 (Cisco) scsi-fcp:target svc
0xe80200 N 21:01:00:e0:8b:31:20:31 (QLogic) scsi-fcp:init
0xe80300 N 21:00:00:e0:8b:11:29:31 (QLogic) scsi-fcp:init
Total number of entries = 4
VSAN 2:
--------------------------------------------------------------------------
--------------------------------------------------------------------------
0xe80000 N 2f:af:00:05:30:00:1a:e0 (Cisco) scsi-fcp:both svc
0xe80001 N 2f:b8:00:05:30:00:1a:e0 (Cisco) scsi-fcp:both svc
VSAN 3:
--------------------------------------------------------------------------

Assigning VSAN Numbers
--------------------------------------------------------------------------
0xea0000 N 50:05:07:63:00:c8:9c:f9 (IBM) scsi-fcp:target fc..
0xea0001 N 50:05:07:63:00:c8:9c:fa (IBM) scsi-fcp:target fc..
0xea0002 N 21:2e:00:05:30:00:00:23 (Cisco) scsi-fcp:init svc
Assigning VSAN Numbers

When configuring SVC interfaces and N-port VSANs, the VSAN number can be any number from 1 to
4096. Only 64 VSANs for all initiator/mgmt/target (in total) are allowed—you can have initiator in
VSANs 1-30, target in VSANs 31-60, and mgmt in VSANs 61-64). If the target, inititator, and mgmt
overlap in any VSAN, each overlap is also included in the total VSAN count. A mgmt N-port can only
exist in 4 of these 64 VSANs.
Multiple Initiators and Targets

You can create multiple N-ports for any SVC interface in different VSANs (see Figure 2-1).
Figure 2-4 Graphical Representation of SVC Terms
Host VSAN Host VSAN

VSAN 1 VSAN 5
Target Target Target Target Target Target Target Target

N-Port N-Port N-Port N-Port N-Port N-Port N-Port N-Port
Mgmt Mgmt Mgmt Mgmt
VSAN 2
Disk VSAN
VSAN 3
105744

Multiple Initiators and Targets

C H A P T E R 3
Creating and Managing Clusters
This section explains the steps required to create clusters. To configure other SVC features or to access
the switch after initial configuration you can use one of the following CLI or Graphical User Interface
(GUI) options:
• Cisco MDS 9000 Family CLI—to use the Cisco MDS CLI, follow the procedure specified in this
guide.
Note The rest of this procedure uses the Cisco CLI to configure the Cisco MDS switch.
• IBMTM SAN Volume Controller CLI—to use the IBM CLI, refer to the IBM TotalStorage SAN
Volume Controller Storage Software for Cisco MDS 9000 Command-Line Interface User's Guide.
• IBM SAN Volume Controller GUI—to use the IBM web-based GUI, refer to the IBM TotalStorage
SAN Volume Controller Storage Software for Cisco MDS 9000 Configuration Guide.
• About CSM Nodes, page 3-2
• About Clusters, page 3-2
• Selecting Nodes for a Cluster, page 3-3
• Isolating Management Traffic, page 3-4
• Creating a Cluster, page 3-5
• Adding Nodes to Clusters, page 3-6
• Verifying Nodes in a Cluster, page 3-7
• Deleting a node from a Cluster, page 3-8

Chapter 3 Creating and Managing Clusters
About CSM Nodes
About CSM Nodes

A node provides virtualization, caching, migration and copy services to the SAN. Nodes are deployed
in pairs with each pair of nodes forming an I/O group. The nodes belonging to the same I/O group MUST
have different power domains. This entails that nodes of the same I/O group should come from two
different CSMs. When a node fails within an I/O group then the other node in the I/O group will take
over the responsibilities of the failed node. Data loss during a node failure is prevented by mirroring the
IO read/write cache info across both the nodes in the I/O group.
About Clusters
Nodes are grouped into clusters of up to 2 pairs of nodes. These nodes are managed as a set (cluster),
and present a single point of control for the user for configuration and service activity. For I/O purposes,
so as to avoid a single point of loss of availability nodes will be grouped into pairs (I/O groups), with a
single pair being responsible for serving I/O on a given VDisk. I/O traffic for a particular VDisk is, at
any one time, handled exclusively by the nodes in a single I/O group. Thus, although a cluster may have
many nodes within it, the nodes handle i/o in independent pairs. This means that the i/o capability of a
cluster scales well, since additional throughput can simply be obtained by adding additional I/O Groups.
There are some circumstances when all the nodes in the cluster do act together rather than in pairs. At
any one time, a single node in the cluster is used to manage configuration activity. The node at which
the cluster got created will start off as the configuration node. This configuration node manages a cache
of the configuration information that describes the cluster configuration and provides a focal point for
configuration commands. Similarly, at any one time, a single node acts as the managing node for overall
management of the cluster. If the configuration node or managing node fails, another node in the cluster
will take over their responsibilities. The nodes also act together to implement the data migration function
described in Chapter 7, “Configuring Copy Services.”
There are several advantages to managing a set of nodes as a cluster.
• All the cluster related configuration operations happen on the config node.
• Individual node operations like node addition, deletion, shutdown, can be done at the config node.
• All the nodes in the cluster run the same software version. Software upgrade can be initiated for the
whole cluster instead of having to do this on a per-node basis.
Physical Topology
In Figure 3-1, CSMs reside in slots 3 and 7 in a Cisco MDS 9500 Series switch. CSM 3 has two nodes
identified as interface svc 3/1 and interface svc 3/2. CSM 7 has two nodes identified as interface svc
7/1 and interface svc 7/2. These four interfaces are configured to form a 4-node cluster.
• I/O group 1 includes interface svc 3/1 (Node1) and interface svc 7/1 (Node3).
• I/O group 2 is made up of interface svc 3/2 (Node 2) and interface svc 7/2 (Node 4).
These two I/O groups form a SVC cluster. So SVC interfaces 3/1, 3/2, 7/1, and 7/2 belong to one cluster
Figure 3-1 also shows two hosts and a back-end storage device. This physical topology serves as an
example in the following sections to understand SVC configurations.

Selecting Nodes for a Cluster
Figure 3-1 Sample SVC Configuration Using Cisco MDS Switches
Host 1 Host 2
ESS
Cisco MDS 9500 Switch
Node 1 CSM 3 Node 2
Node 3 CSM 7 Node 4
105301
Selecting Nodes for a Cluster
To configure a 4-node sample configuration, follow these steps.
Step 1 Change to SVC configuration mode.

switch1# svc-config
Step 2 Display the available nodes in the local switch (switch 1).
switch1(svc)# show nodes local
-------------------------------------------------------------------------------
Node cluster config cluster node sw
node status status version
-------------------------------------------------------------------------------
svc3/1 No unconfigured free 1.3(1)

Isolating Management Traffic
Isolating Management Traffic

Management traffic is isolated for CSM nodes that are part of the same cluster using a traffic domain
separator (for example VSANs or zones).
To create a separate traffic domain using VSANs, follow these steps.
Step 1 Exit the SVC configuration mode and enter the configuration mode.
switch1(svc)# exit
switch1# config t
switch1(config)#
Step 2 Configure VSAN 2 for CSM management traffic

switch1(config)# vsan database
switch1(config-vsan-db)# vsan 2
switch1(config-vsan-db)# exit
switch1(config)#
Step 3 Configure the management N-ports on VSAN 2 for all 4 SVC nodes.
switch1(config)# interface svc3/1
switch1(config-if)# mgmt vsan 2
switch1(config-if)# no mgmt vsan 1 <--- You have to explicitly remove from vsan 1
switch1(config-if)# no shut
switch1(config-if)# exit
switch1(config)#
Step 4 Show the FC name server database

switch1# show fcns database vsan 2
VSAN 2:
--------------------------------------------------------------------------
--------------------------------------------------------------------------
0x770000 N 22:32:00:05:30:00:11:69 (Cisco) scsi-fcp:both svc
Step 5 Default zone configuration is permit for VSAN 2.

switch1(config)# zone default-zone permit vsan 2
switch1(config)#

Creating a Cluster
Creating a Cluster
Create a cluster called SampleCluster using one node for the cluster creating This example uses
interface svc3/1 to begin the cluster creation process. It also uses the 10.1.1.100 IP address that is in the
same subnet as switch 1’s management’s IP network.
Note If the cluster spans multiple switches, the switch management IP address should be in the same subnet
as the cluster IP address, because the cluster IP address can move to any switch (based on the SVC config
node).
To create a cluster, follow these steps.
Step 1 Create a cluster using the cluster add command in SVC configuration mode.
switch1# svc-config
switch1(svc)# cluster add SampleCluster ip 10.1.1.100 node svc3/1
Cluster creation going on. Please wait....---> This process takes a few seconds.
Step 2 Verify the newly-created cluster for the configured node.

-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
svc3/1 SampleCluster Yes active active 1.3(1)
Step 3 Display the configured cluster.

switch1(svc)# show cluster SampleCluster nodes
Node node1 is online(3)
Node WWN is 22:26:00:05:30:00:11:69
Serial number is JAB072006AQ
Unique id is 01:00:07:20:30:36:41:51
Node is in config mode
Node is part of iogroup id 1 name io_grp0
The configured node is the only node in this cluster.

Adding Nodes to Clusters
Adding Nodes to Clusters

Once the initial node is used to create a cluster, you can add other required nodes to the same cluster.
You can determine which nodes are available by issuing the show cluster cluster-name nodes candidate
command.
To add other nodes to a cluster, follow these steps.
Step 1 Add the other nodes to the cluster.

a. Verify which nodes can be added to the cluster. This step is important if you have several SVC nodes
in the SAN,.
switch1(svc)# show cluster SampleCluster nodes candidate
-----------------------------------------------------------------------------
NODE NWWN
-----------------------------------------------------------------------------
switch1.7.2 21:28:00:05:30:00:11:69
switch1.7.1 21:26:00:05:30:00:11:69
switch1.3.2 21:2a:00:05:30:00:11:69
This example has 3 other SVC nodes in the SAN that are candidate nodes for this cluster. The node
name is an encoding of the <switch-name>.<slot-number>.<node-ID>. For example: switch1.7.2 is
in the switch named switch1 at slot 7 node 2.
Caution Do not add two nodes from the same CSM to the same I/O group of a cluster. Cisco MDS SVC does not
allow this configuration as both nodes will be contained in one power domain. If both nodes are
configured in the same I/O group of one cluster and a power failure occurs, both nodes will fail.
b. Add more nodes (switch 1-slot 3-node 2, switch 1-slot 7-node 1, and switch 1-slot 7-node 2) to the
newly-created cluster by entering the configuration submode for the selected cluster
(SampleCluster)
switch1(svc)# cluster config SampleCluster
switch1(svc-cluster)# node nwwn 21:28:00:05:30:00:11:69 iogroup 1
switch1(svc-cluster)# node nwwn 21:26:00:05:30:00:11:69 iogroup 2
switch1(svc-cluster)# node nwwn 21:2a:00:05:30:00:11:69 iogroup 2
switch(svc-cluster)# exit

Verifying Nodes in a Cluster
Verifying Nodes in a Cluster

After the cluster is created with the required nodes, you can verify the status of each node in the cluster,
status of each node, and the associated I/O group to ensure the configuration is functioning as desired.
The node state transitions from adding to pending to online during the cluster creation process.
To verify the nodes in a cluster, follow these steps.
Step 1 Verify the nodes in the cluster.

switch1(svc)# show cluster SampleCluster nodes
Node WWN is 22:26:00:05:30:00:11:69
Unique id is 01:00:07:20:30:36:41:51
Node is in config mode

Node WWN is 21:28:00:05:30:00:11:69
Serial number is JAB076607H8
Unique id is 01:00:07:66:30:37:48:38
Node is in non config mode
Node node3 is pending(2)

Node WWN is 21:26:00:05:30:00:11:69
Serial number is JAB071007H8
Unique id is 01:00:07:10:30:37:48:38
Node node4 is adding(6)

Node WWN is 00:00:00:00:00:00:00:00
Unique id is 01:00:07:66:30:36:41:51
Step 2 Verify the I/O group status

switch1(svc)# show cluster SampleCluster iogroup
ID Name Node count VLUN count
--- --------------- ---------- ---------
1 io_grp0 2 0
2 io_grp1 2 0
3 io_grp2 0 0
4 io_grp3 0 0
5 recovery_io_grp 0 0
Note The recovery_io_group is an internal iogroup used during cluster recovery.

Deleting a node from a Cluster
Step 3 Display the status of all nodes in the switch.

switch(svc)# show nodes local
Config Cluster Node Sw
Node Cluster node status status version
-------- ---------------- ------ ------- ------ -----------
svc3/1 SampleCluster Yes Active Active 1.3(1)
svc3/2 SampleCluster No Active Active 1.3(1)
svc7/1 SampleCluster No Active Active 1.3(1)
svc7/2 SampleCluster No active Active 1.3(1)
The 4-node cluster is now created with the nodes communicating with each other in VSAN 2. All nodes
in the switch are active and are part of cluster named SampleCluster. The SVC config node is svc3/1 (see
Figure 3-2).
Figure 3-2 Creating a 4-Node Cluster

Mgmt Mgmt Mgmt Mgmt
VSAN 2
105304

A node has to be removed from a cluster using the no node command in the cluster config mode.
To delete a node that is online, follow these steps.
Step 1 Enter the cluster configuration mode for the required cluster.
switch1# svc-config
Step 2 Issue the no node nwwn command.

switch1(svc-cluster)# no node nwwn 21:28:00:05:30:00:11:69 iogroup 1
To delete a node that is in an offline state, follow these steps.
Step 1 Enter the cluster configuration mode for the required cluster.
switch1# svc-config
Step 2 Issue the no node name command.

switch1(svc-cluster)# no node name node3

When you delete a node in a cluster, the node is removed from the cluster state. In addition, the local
state of the deleted node is also updated to indicate that it is no longer a part of any cluster.
If the node is offline, the local state of the deleted node should be explicitly updated using the node svc
x/y delete command.
Deleting a Cluster
The MDS CLI does not use an explicit command to delete a cluster. The cluster is automatically deleted
when the last node in the cluster is deleted.


C H A P T E R 4
Managing Back-End Storage
The nodes in a cluster view the back-end storage controllers as individual disks, known as managed disks
(MDisks).
• About Managed Disks, page 4-2
• About MDisk Groups, page 4-2
• Extents, page 4-2
• MDisk Modes, page 4-2
• Isolating Back-end Storage Traffic, page 4-3
• Verifying Traffic Isolation, page 4-4
• Configuring LUN Masking, page 4-4
• Configuring LUN Masking, page 4-4
• Configuring MDisk Groups, page 4-5
• Configuring Quorum Disks, page 4-7

Chapter 4 Managing Back-End Storage
About Managed Disks
About Managed Disks

SVC nodes see the back-end storage controllers presented to SANs as a number of individual disks,
known as Managed Disks (MDisks). SVC does not attempt to provide recovery from physical disk
failures within the back-end controllers. For this reason, a MDisk is usually, but not necessarily, a RAID
array.
All nodes in a cluster must be able to see the same set of back-end storage ports on each back-end
controller. Operation in a mode where two nodes see a different set of ports on the same controller is
degraded and the system logs errors requesting a repair action. This occurs if inappropriate zoning is
configured in the fabric or if inappropriate LUN masking is used. This rule has important implications
for back-end storage such as FastT which impose exclusivity rules on the HBA WWNs that a storage
partition can be mapped to. For such storage devices, the initiator N-port pWWNs of all nodes in a
cluster must be mapped to the same set of LUNs
About MDisk Groups

MDisk Groups are collections of MDisks. A MDisk is contained within exactly one MDisk Group.The
MDisk Group acts as a container into which MDisks contribute Extents (Chunks of disk blocks) and
from which VDisk (logical disks associated with nodes in an I/O group) consume these extents of
storage.
Extents
An extent is the unit of allocation of storage in a MDisk. Each MDisk is broken up logically into a
number of extents. A MDisk does not need to be a integer multiple of extent size. SVC supports a partial
extent at the end of the MDisk.
However, a VDisk occupies an integer number of extents even if the VDisk size is not an integer multiple
of the extent size. The remaining space at the last extent in the VDisk remains unused.
MDisk Modes
The three MDisks modes are Image, Managed, or Unmanaged.
• Image mode
Image Mode provides a direct block-for-block translation from a MDisk to a VDisk virtualization.
This mode allows virtualization of MDisks which already contain data. It allows a customer to insert
SVC into the data path of an existing storage configuration with minimal downtime. Once SVC is
inserted into the data path using image mode, you can use SVC’s migration facilities to migrate the
data to managed mode and re-arrange the data while an application is accessing the data.
• Managed mode
Disks operating in managed mode allow an arbitrary relationship between the VDisk extents and the
MDisk extents. The actual mapping of the extent is based on the VDisk creation policy. The unused
extents in a MDisk are available for use in creating new VDisks data migration.
• Unmanaged mode
Mdisks in this mode do not belong to any Mdisk group

Isolating Back-end Storage Traffic
Isolating Back-end Storage Traffic

Setup the traffic isolation for all initiator N-ports in the SampleCluster into the same VSAN or zone as
the back-end storage ports.
To isolate initiator N-ports, follow these steps.
Step 1 Create the target N-ports that this cluster can and should access. In this example, the N-ports to be
accessed by the SampleCluster are 20:12:00:05:30:00:8d:e0 (interface fc 1/7) and
20:22:00:05:30:00:8d:e0 (interface fc 1/8).
Step 2 Create VSAN 3 for the target traffic for the SampleCluster.
switch# conf t
switch(config)# vsan database
switch(config-vsan-db)# vsan 3
Step 3 Add the two Fibre Channel N-ports connected to targets into VSAN 3.
switch(config-vsan-db)# vsan 3 interface fc1/7
switch(config-vsan-db)# vsan 3 interface fc1/8
switch(config-vsan-db)# exit
switch(config)#
Step 4 Add the CSM node's initiator N-port in the SampleCluster in VSAN 3.
Note Each initiator must be explicitly removed from VSAN 1.
switch(config)# interface svc3/1

switch(config-if)# initiator vsan 3
switch(config-if)# no initiator vsan 1
switch(config-if)# exit
switch(config)# interface svc 7/2
Step 5 Configure the default zone configuration as permit for VSAN3.

switch(config)# zone default-zone permit vsan 3

Verifying Traffic Isolation
You have now created a disk VSAN (see Figure 4-1).
Figure 4-1 Creating Traffic Isolation Using a Target VSAN

Mgmt Mgmt Mgmt Mgmt
VSAN 2
Disk VSAN
VSAN 3
105302
Verifying Traffic Isolation
To verify that the target initiator ports are configured and traffic isolation has been implemented issue
the show fcns database command for VSAN 3. switch# show fcns database vsan 3
VSAN 3:
--------------------------------------------------------------------------
--------------------------------------------------------------------------
0x730000 N 50:05:07:63:00:c5:9c:f9 (IBM) scsi-fcp:target fc..
0x730001 N 50:05:07:63:00:cf:9c:f9 (IBM) scsi-fcp:target fc..
0x730002 N 22:36:00:05:30:00:11:69 (Cisco) scsi-fcp:init svc
Configuring LUN Masking

The initiator N-port pWWN of all four nodes in the cluster must be used to configure LUN masking in
the backend storage controller.
switch# show interface svc 3/1
svc2/1 is up
Node WWN is 2e:ab:00:05:30:00:1a:e0
Fabric WWN is 20:01:00:05:30:00:1a:de

Identifying MDisks
5 minutes input rate 2392 bits/sec, 299 bytes/sec, 0 frames/sec

5 minutes output rate 2240 bits/sec, 280 bytes/sec, 0 frames/sec
272 frames input, 89764 bytes
232 frames output, 84176 bytes
Identifying MDisks
To identify MDisks, follow these steps:
Step 1 Display the details of each MDisk using the show cluster cluster-name mdisk command. This command
lists the back-end controller ports that can be accessed by the MDisk. The MDisk still do not belong to
any MDisk group.
switch# svc-config
switch(svc)# show cluster SampleCluster mdisk
-------------------------------------------------------------------------------
id nwwn mdisk-grp capacity status
-------------------------------------------------------------------------------
1 50:05:07:63:00:c0:9c:f9 953 MB online
2 50:05:07:63:00:c0:9c:f9 1.86 GB online
3 50:05:07:63:00:c0:9c:f9 1.86 GB online
4 50:05:07:63:00:c0:9c:f9 1.86 GB online
5 50:05:07:63:00:c0:9c:f9 1.86 GB online
6 50:05:07:63:00:c0:9c:f9 953 MB online
7 50:05:07:63:00:c0:9c:f9 1.86 GB online
8 50:05:07:63:00:c0:9c:f9 953 MB online
9 50:05:07:63:00:c0:9c:f9 1.86 GB online
10 50:05:07:63:00:c0:9c:f9 953 MB online
Step 2 Display details of the required MDisk using the show cluster cluster-name mdisk id command.
switch(svc)# show cluster SampleCluster mdisk id 1
mdisk id 1 is online
Is unmanaged
Controller node WWN is 50:05:07:63:00:c0:9c:f9 --> IBM ESS storage device's nWWN
Controller port WWN is 50:05:07:63:00:cf:9c:f9, LUN 00:00:00:00:00:00:00:00
Controller port WWN is 50:05:07:63:00:c5:9c:f9, LUN 00:00:00:00:00:00:00:00
Controller serial number is 07B24417
Capacity is 953 MB
Configuring MDisk Groups

To configure MDisk groups, follow these steps:
Step 1 Obtain a list of candidate MDisks using the show cluster cluster-name mdisk candidate command.
Select the group(s) to add the configured MDisk.
switch(svc)# show cluster SampleCluster mdisk candidate
----------------------------------------------------
id nwwn capacity
----------------------------------------------------
1 50:05:07:63:00:c0:9c:f9 953 MB

Configuring MDisk Groups
2 50:05:07:63:00:c0:9c:f9 1.86 GB
3 50:05:07:63:00:c0:9c:f9 1.86 GB
4 50:05:07:63:00:c0:9c:f9 1.86 GB
5 50:05:07:63:00:c0:9c:f9 1.86 GB
6 50:05:07:63:00:c0:9c:f9 953 MB
7 50:05:07:63:00:c0:9c:f9 1.86 GB
8 50:05:07:63:00:c0:9c:f9 953 MB
9 50:05:07:63:00:c0:9c:f9 1.86 GB
10 50:05:07:63:00:c0:9c:f9 953 MB
Step 2 Create a MDisk group called finance.

switch(svc)# cluster config SampleCluster
switch(svc-cluster)# mdisk-grp add finance extent 16
switch(svc-cluster)# mdisk-grp name finance
switch(svc-cluster-mdisk-grp)#
Step 3 Add MDisks 1 to 5 to the finance MDisk group.

switch(svc-cluster-mdisk-grp)# mdisk id 1
Step 4 Exit to the SVC cluster configuration mode.

switch(svc-cluster-mdisk-grp)# exit
switch(svc-cluster)#
Step 5 Create a MDisk group called marketing.

switch(svc-cluster)# mdisk-grp name marketing
switch(svc-cluster)# mdisk-grp add marketing extent 32
switch(svc-cluster-mdisk-grp)#
Step 6 Add MDisks 6 to 10 to the marketing MDisk group.

Step 7 Exit to the SVC configuration mode.

switch(svc-cluster-mdisk-grp)# exit
switch(svc)#
Step 8 Verify that all configured MDisks are allocated to each MDisk group.
switch(svc)# show cluster SampleCluster mdisk
-------------------------------------------------------------------------------
id nwwn mdisk-grp capacity status
-------------------------------------------------------------------------------
1 50:05:07:63:00:c0:9c:f9 finance 953 MB online
2 50:05:07:63:00:c0:9c:f9 finance 1.86 GB online
6 50:05:07:63:00:c0:9c:f9 marketing 953 MB online
7 50:05:07:63:00:c0:9c:f9 marketing 1.86 GB online
9 50:05:07:63:00:c0:9c:f9 marketing 1.86 GB online

Configuring Quorum Disks
Step 9 Verify the details of each MDisk group and confirm that each MDisk group has 5 MDisks. VDisks have
not been assigned at this point.
switch(svc)# show cluster SampleCluster mdisk-grp
-------------------------------------------------------------------------------
name Capacity free extent number number status
size(MB) of mdisks of vdisks
-------------------------------------------------------------------------------
finance 7.56 GB 7.56 GB 16 5 0 online
marketing 6.48 GB 6.48 GB 16 5 0 online
Note The storage capacity (953 MB) is not an integer multiple of the extent size, the last partial extent is
unused.
Step 10 To identify the number of free extents for MDisk 1, use the show cluster cluster-name mdisk id
command.
switch(svc)# show cluster SampleCluster mdisk id 1
mdisk id 1 is online
Is member of mdisk-grp finance
Controller node WWN is 50:05:07:63:00:c0:9c:f9
Controller port WWN is 50:05:07:63:00:cf:9c:f9, LUN 00:00:00:00:00:00:00:00
Controller port WWN is 50:05:07:63:00:c5:9c:f9, LUN 00:00:00:00:00:00:00:00
Controller serial number is 07B24417
Capacity is 953 MB
Number of free extents is 42
Is quorum disk number 2
Note MDisk ID 1 (one) is a quorum disk and SVC nodes reserve some storage on the quorum disks for cluster
management.

Quorum disks are used when the cluster loses half of the nodes that make up the cluster. The half of the
cluster that contains the majority of the quorum disks continues to process the I/O transactions. The other
half stops processing the I/O transactions. By establishing quorum disks, you can ensure that both halves
of the cluster do not continue to operate.
Quorum disks are set automatically by the software. You can use the quorum command to overwrite the
system-assigned quorum disk and pick a particular set of managed disks to be a quorum disk. The
managed disk that is currently assigned the quorum index number is automatically set to be a
non-quorum disk.
You can assign one of 3 possible quorum index numbers in any desired order. The quorum index number
ranges from 1 to 3.
Tip We recommend that you set quorum disks on multiple controllers to avoid the possibility of losing all of
the quorum disks with a single failure.

To configure quorum disk allocation, follow these steps:
Step 1 Create a cluster called SampleCluster.

Step 2 Sets the quorum disk ID for the specified MDisk in this cluster.
switch(svc-cluster)# quorum disk 2 mdisk 1
When you issue this command, the Mdisk that was previously-assigned the quorum index 2 will no
longer be a quorum disk.
Caution If the quorum command fails due to the lack of sufficient extents in the new quorum disk, the old
quorum disk may no longer be operational.

C H A P T E R 5
Managing Virtual Disks
A VDisk is a virtual representation of a LUN that is exposed by the cluster to the hosts in a SAN. Each
VDisk is independently associated with a single I/O group.
• Virtualization Policies, page 5-2
• Licensing Requirements, page 5-2
• Configuring VDisks, page 5-3

Chapter 5 Managing Virtual Disks
Virtualization Policies
Virtualization Policies
Virtualization is the process of creating a pool of storage that can be split into VDisks. VDisks are visible
to the host systems that use them and provide a common way to manage SAN storage. VDisks in the
Cisco MDS 9000 Family use one of three virtualization policies: striped, sequential, or image.
Striped
When a VDisk is created using a striped policy its extents are allocated from the specified ordered list
of MDisks. The allocation algorithm starts with the first MDisk in the ordered list and attempts to
allocate an extent to it and then moves on to the next disk.
MDisk in turn process allocation—if a specified MDisk has no free extents then it misses its turn and
the turn passes to the next MDisk in the list. When the end of the list is reached the turn loops back to
the first disk in the list. The disk allocation proceeds until all required extents are allocated.
A specific MDisk can appear more than once in the list. This causes two extents to be allocated from the
disk on each pass of the list. This might be useful when striping across MDisks of different sizes.
Sequential
When a VDisk is created using a sequential policy its extents are allocated from a single specified
MDisk. The target MDisk is searched for regions containing free extents which are sequential such that
the region is large enough to allocate the VDisk from completely sequential extents. If it finds more than
once such region, it chooses the smallest region which satisfies this condition. If it finds no such regions,
the VDisk creation fails.
Image
Image mode provides support to import existing data from a disk that was previously not managed by
SVC.
Licensing Requirements
The total virtualized capacity that is licensed is the number of Gigabytes (GB) of VDisk capacity that
are exported by the cluster. By default, this capacity is set to zero (0). The required amount of
virtualization capacity must be licensed and configured using the feature enable command before
creating any VDisk.
When you reach 90% capacity, any attempt to create or extend VDisks results in a warning messages.
The software does not stop you from creating and expanding VDisks. Instead, errors are placed in the
featurization log when your usage reaches or exceeds 100%.

Configuring VDisks
Configuring VDisks
To configure VDisks, follow these steps.
Step 1 Create and identify three (3) VDisks from the marketing MDisk group and one (1) VDisk for the finance
group.
switch1(svc-cluster)# vdisk add crm-log iogroup 1 mdisk-grp marketing capacity 2 gb
Warning: licensed virtualisation capacity has been exceeded
Tip The official purchased virtualization capacity must be configured before any VDisk is created.
This warning is used if you are exceed the amount of virtualization for which you have a license. Use
the feature enable capacity command to configure the amount of purchased virtualization capacity.
Step 2 Configure the licensed virtualization capacity to be 200GB and continue to create the VDisks for the
marketing and finance MDisk groups.
switch(svc-cluster)# vdisk add crm-data iogroup 1 mdisk-grp marketing capacity 2 gb clean
switch(svc-cluster)# vdisk add crm-idx iogroup 1 mdisk-grp marketing capacity 1 gb
switch(svc-cluster)# vdisk add fn-1 iogroup 2 mdisk-grp finance capacity 2 gb
Note The clean option initializes the entire VDisk to 0. Until the cleaning is done, the VDisk stays in the
offline state.

switch(svc)#
Step 4 Verify the VDisk configuration using the show cluster cluster-name vdisk command.
switch(svc)# show cluster SampleCluster vdisk
-------------------------------------------------------------------------------
name capacity iogroup mdisk-grp name policy status
-------------------------------------------------------------------------------
crm-idx 1.00 GB 1 marketing striped online
crm-log 1.00 GB 1 marketing striped online
crm-data 2.00 GB 1 marketing striped offline
fn-1 2.00 GB 2 finance striped online
Tip The crm-data VDisk is offline due to the use of the clean option during the VDisk creation process in
Step 2. Clearing a disk takes time—please wait for this process to complete.
Step 5 Reissue the show cluster cluster-name vdisk command to ensure that all VDisks are online.
-------------------------------------------------------------------------------
name capacity iogroup mdisk-grp name policy status
-------------------------------------------------------------------------------
crm-idx 1.00 GB 1 marketing striped online
crm-log 1.00 GB 1 marketing striped online
crm-data 2.00 GB 1 marketing striped online
fn-1 2.00 GB 2 finance striped online

Configuring VDisks
Note The online status for each VDisk indicates that formatting is complete.
Step 6 Use the show cluster cluster-name mdisk-group command to verify the number of VDisks created in
each MDisk.
switch(svc)# show cluster SampleCluster mdisk-grp
-------------------------------------------------------------------------------
name Capacity free extent number number status
size(MB) of mdisks of vdisks
-------------------------------------------------------------------------------
finance 7.56 GB 5.56 GB 16 5 1 online
marketing 6.48 GB 2.48 GB 16 5 3 online
Step 7 Use the show cluster cluster-name iogroup command to verify the number of VDisks available for each
I/O group.
switch(svc)# show cluster SampleCluster iogroup
------------------------------------------
ID Name Node count Vdisk count
------------------------------------------
1 io_grp0 2 3
2 io_grp1 2 1
3 io_grp2 0 0
4 io_grp3 0 0
5 recovery_io_grp 0 0
Note The recovery-io-group is an internal SVC I/O group created for cluster recovery processes.

C H A P T E R 6
Configuring Hosts
To continue configuring the using the SVC application for a Cisco MDS 9216 switch or for any switch
in the Cisco MDS 9500 Family, you must determine the number of hosts, isolate host traffic to VSAN
1, and map VDisks to hosts.
• About Hosts, page 6-1
• Isolating Host Traffic, page 6-1
• Creating Hosts, page 6-4
• Mapping VDisks to Hosts, page 6-4
• Configuring iSCSI Hosts in SVC, page 6-5
About Hosts
Hosts are identified to the cluster through user configuration. The LUN mapping feature controls which
VDisks are accessible by which hosts. A host may contain multiple ports that connect to the SAN. To
ease the configuration process, all host ports can be configured in a group. Host ports are referred to by
their pWWNs. The specific LUN number is optionally specified when the LUN map is configured.
Otherwise, the cluster chooses a LUN number automatically.
Isolating Host Traffic

The SVC target N-ports are in VSAN 1 along with the two hosts. To isolate hosts and targets, this
example uses the SAN-OS zoning feature.
To isolate hosts using the zoning feature, follow these steps.
Step 1 Issue the show fcns database vsan 1command to view the hosts and targets in VSAN 1.
switch# show fcns database vsan 1
VSAN 1:
--------------------------------------------------------------------------
--------------------------------------------------------------------------
0x6a0200 N 21:00:00:e0:8b:09:e7:04 (QLogic) scsi-fcp:init
0x6a0300 N 21:00:00:e0:8b:09:f0:04 (QLogic) scsi-fcp:init
0x6a0003 N 22:20:00:05:30:00:11:69 (Cisco) scsi-fcp:target svc

Chapter 6 Configuring Hosts

0x6a000c N 21:20:00:05:30:00:11:69 (Cisco) scsi-fcp:target svc
Step 2 Enter the MDS configuration mode and create a zone called host-finance in VSAN 1.
switch# config t
switch(config)# zone name host-finance vsan 1
switch(config-zone)#
Step 3 Assign members to the host-finance zone.

switch(config-zone)# member pwwn 21:00:00:e0:8b:09:e7:04 --> Host finance
switch(config-zone)# member pwwn 22:20:00:05:30:00:11:69 --> SVC target N-port
Step 4 Create a zone called host-marketing in VSAN 1.

switch(config)# zone name host-marketing vsan 1
Step 5 Assign members to the host-marketing zone.

switch(config-zone)# member pwwn 21:00:00:e0:8b:09:f0:04 --> Host marketing
Step 6 Exit to the MDS configuration mode and create a zone set called main-zset in VSAN 1
switch(config-zone)# exit
switch(config)# zoneset name main-zset vsan 1
switch(config-zoneset)#
Step 7 Assign the host-finance zone and the host-marketing zone as members of the main-zset zone set.
switch(config-zoneset)# member host-finance
switch(config-zoneset)# member host-marketing
Step 8 Exit to the MDS configuration mode.

switch(config-zoneset)# exit
switch(config-zone)# exit
switch(config)#
Step 9 Activate the main-zset zoneset in VSAN 1.

switch(config)# zoneset activate name main-zset vsan 1
Zoneset activation initiated. check zone status
Step 10 Exit to the MDS EXEC mode and verify the active zoneset in VSAN 1.
switch(config)# exit
switch# show zoneset active vsan 1
zoneset name main-zset vsan 1
zone name host-finance vsan 1
* fcid 0x6a0200 [pwwn 21:00:00:e0:8b:09:e7:04]
* fcid 0x6a0003 [pwwn 22:20:00:05:30:00:11:69]
* fcid 0x6a0006 [pwwn 21:23:00:05:30:00:11:69]
* fcid 0x6a0009 [pwwn 22:23:00:05:30:00:11:69]
* fcid 0x6a000c [pwwn 21:20:00:05:30:00:11:69]
zone name host-marketing vsan 1

* fcid 0x6a0300 [pwwn 21:00:00:e0:8b:09:f0:04]

* fcid 0x6a0003 [pwwn 22:20:00:05:30:00:11:69]
* fcid 0x6a0006 [pwwn 21:23:00:05:30:00:11:69]
* fcid 0x6a0009 [pwwn 22:23:00:05:30:00:11:69]
* fcid 0x6a000c [pwwn 21:20:00:05:30:00:11:69]
zone name $default_zone$vsan 1
You have now created the host VSAN and two zones—host-finance and host-marketing (see Figure 6-1).
Figure 6-1 Identifying the Target N-Ports
host-marketing
host-finance
Host VSAN
VSAN 1

Mgmt Mgmt Mgmt Mgmt
VSAN 2
Disk VSAN
VSAN 3
05300

Creating Hosts
Creating Hosts
To create a host, follow these steps.
Step 1 Use the show cluster cluster-name host candidate command to obtain a list of candidate hosts.
switch# svc-config
switch(svc)# show cluster SampleCluster host candidate
-------------------------------------------------------------------------------
id pwwn
-------------------------------------------------------------------------------
1 21:00:00:e0:8b:09:e7:04
Use the host candidate’s pWWN to identify the host to be mapped.

Step 2 Enter the cluster configuration submode.
Step 3 Create a host called Finance1 with the pWWN identified in Step 1.
switch(svc-cluster)# host add Finance1 hostport 21:00:00:e0:8b:09:e7:04

switch(svc)#
Step 5 Use the show cluster cluster-name host command to verify that the newly added host displays the
number of configured ports.
switch(svc)# show cluster SampleCluster host
-------------------------------------------
name number of ports
-------------------------------------------
Finance1 1
Mapping VDisks to Hosts

To map VDisks to hosts, follow these steps.
Step 1 Enter the cluster configuration submode.

Step 2 Enter the host configuration submode for Finance1.

switch(svc-cluster)# host name Finance1
switch(svc-cluster-host)#
Step 3 Map the three VDisks to Finance1

switch(svc-cluster-host)# map vdisk crm-data SCSI-lun 10
switch(svc-cluster-host)# map vdisk crm-idx
switch(svc-cluster-host)# map vdisk crm-log

Configuring iSCSI Hosts in SVC
Note The optional SCSI-lun 10 option allows the customer to specify the LUN value that is mapped to this
VDisk, otherwise the cluster automatically picks the lowest available value.

switch(svc-cluster-host)# exit
switch(svc)#
Step 5 Verify that Finance1 has one port with the configured pWWN
switch(svc)# show cluster SampleCluster host Finance1
host Finance1:
Number of port is 1
Port WWN is 21:00:00:e0:8b:09:e7:04
LUN0: vdik crm-idx
LUN1: vdisk crm-log
LUN10: vdisk crm-data

This section provides the configuration procedure to allow an iSCSI host to access VDisks exported by
an SVC cluster.
Tip Linux users should configure the /etc/iscsi.conf file using the following parameters:
Multipath=no
HostIPsforMP=<ip address of NIC1>,<ip address of NIC2>
ConnFailTimeout=50
Refer to the Cisco MDS 9000 Family Configuration Guide for further details on iSCSI concepts and
configuration options.
Note Before configuring the iSCSI Hosts, be sure to configure the required level of iSCSI authentication.
Refer to the Cisco MDS 9000 Family Configuration Guide for further information on the available iSCSI
authentication option.
The following example uses the null authentication option. It also displays the configuration for two
iSCSI hosts.
To configure two iSCSI hosts to access VDisks exported by an SVC cluster, follow these steps.
Step 1 Configure iSCSI to dynamically import all Fibre Channel targets into the iSCSI SAN using
auto-generated iSCSI target names.
switch# conf t
switch(config)# iscsi import target fc
Step 2 Configure the Gigabit Ethernet interface in slot 4 port 1 with an IP address and enable the interface.
switch(config)# int gigabitethernet 4/1

switch(config-if)# ip address 10.11.1.1 255.255.255.0

switch(config-if)# no shut
switch(config)#
Step 3 Configure the iSCSI interface in slot 4 port 1 and enable the interface.
switch(config)# int iscsi 4/1
switch(config-if)# no shut
switch(config)#
Step 4 Configure static pWWNs for each iSCSI initiator.

The first iSCSI initiator is identified using the IQN name—one nWWN and one pWWNs from the
switch's Fibre Channel WWN pool are allocated in the SVC target N-port VSAN (in this example, VSAN
1—See Figure 6-1) keep the mapping permanent:
switch(config)# iscsi initiator name iqn.1987-05.com.cisco:01.e41695d16b1a
switch(config-(iscsi-init))# vsan 1
switch(config-(iscsi-init))# static pWWN system-assign 1
switch(config-(iscsi-init))# static nWWN system-assign
switch(config-(iscsi-init))# exit
switch(config)#
The second iSCSI initiator is identified using the IP address—one pWWN from the switch’s Fibre
Channel WWN pool is assigned in the SVC target N-port VSAN (in this example, VSAN 1—See
Figure 6-1):
switch(config)# iscsi initiator ip address 10.15.1.11
switch(config-(iscsi-init))# vsan 1
switch(config-(iscsi-init))# static pwwn system-assigned 1
switch(config-(iscsi-init))# end
switch#
Step 5 View the configured initiators. The WWNs are automatically assigned by the system.
switch# show iscsi initiator configured
iSCSI Node name is iqn.1987-05.com.cisco:01.e41695d16b1a
Member of vsans: 1
Node WWN is 20:03:00:0b:fd:44:68:c2
No. of PWWN: 1
Port WWN is 21:00:00:e1:8b:09:e7:04
iSCSI Node name is 10.15.1.11

Member of vsans: 1
No. of PWWN: 1
Port WWN is 20:06:00:0b:fd:44:68:c2
Step 6 Add the host to the same zone as the SVC target N-Ports.
Zone membership for the iSCSI initiator can either be the iSCSI symbolic node name or the pWWN. In
this case, the pWWN can be used since it is statically created.
switch# conf t
switch(config)# zone name host-finance vsan 1
Step 7 Assign the membership.

The following example is based on the symbolic node name.
switch(config-zone)# member symbolic-nodename iqn.1987-05.com.cisco:01.e41695d16b1a
The following example is based on the persistent pWWN assigned to the initiator. You can obtain the
pWWN from the output of the show iscsi initiator command.

switch(config-zone)# member pwwn 20:06:00:0b:fd:44:68:c2

switch(config-zone)# end
switch#
Step 8 Create a SVC host using this pWWN

switch# svc-config
switch(svc)# show cluster SampleCluster host candidate
-------------------------------------------------------------------------------
id pwwn
-------------------------------------------------------------------------------
1 21:00:00:e1:8b:09:e7:04
2 20:06:00:0b:fd:44:68:c2

switch(svc-cluster)# host add iscsi1 hostport 21:00:00:e1:8b:09:e7:04
switch(svc-cluster)# host add iscsi2 hostport 20:06:00:0b:fd:44:68:c2
Step 9 Map VDisks to both hosts.

switch(svc-cluster)# host name iscsi1
switch(svc-cluster-host)# map vdisk crm-data
switch(svc-cluster-host)# exit
switch(svc-cluster)# host name iscsi2
switch(svc-cluster-host)# map vdisk crm-data
switch(svc-cluster-host)# end
switch#
Step 10 Start the iSCSI clients on both hosts and verify that the sessions come up using the show iscsi session
command.


C H A P T E R 7
Configuring Copy Services
The SVC copy services function available in all Cisco MDS 9216 switches and directors in the Cisco
MDS 9500 Family enables you to copy virtual Disks (VDisks). These copy services include data
migration, FlashCopyTM, and Remote Copy.
• Data Migration, page 7-2
• FlashCopy, page 7-4
• Remote Copy, page 7-8

Chapter 7 Configuring Copy Services
Data Migration
Data Migration
The Data Migration feature allows you to change the mapping of VDisk extents to MDisk extents
without interrupting a host’s access to that VDisk.
Data Migration Application

You can perform data migration on any VDisk managed by the SVC for the following purposes:
• Redistributing workload within a Cluster across back-end storage.
• Moving workload onto newly installed storage.
• Moving workload off old/failing storage
• Moving workload to rebalance a changed workload
• Migrating in data from legacy back-end storage to SVC managed storage
Data Migration Operation

After a data migration, the virtualisation policy of the VDisk is set to striped. When you request an
operation which requires some migration activity, SVC checks that sufficient suitable free extents exist
to complete the requested migration operation. If this is not the case then the migration will not be started
and the configuration command fails. However, once the operation starts, SVC does not reserve free
extents for the future use of the migration activity. This means that another migration activity can be
started to contend for free extents with existing migrations. If this happens, and no suitable free extent
is available to proceed with the migration, then the migration operation fails. The implications of this
are that migration between MDisk groups will normally be suspended and all other types of migration
is normally stopped.
Migrating VDisks between MDisk groups

To migrate VDisks from one managed MDisk group to another MDisk group, follow these steps.
Step 1 Create a cluster and the required VDisks (see Chapter 3, “Creating and Managing Clusters”).
Step 2 Display the configured VDisks in the newly-created cluster (called SampleCluster).
-------------------------------------------------------------------------------
Name Capacity iogroup mdisk-grp name Policy Status
-------------------------------------------------------------------------------
fn-data 1.00 GB 1 finance1 striped online
fn-log 1.00 GB 1 finance1 striped online
switch(svc)#
Step 3 Create the new MDisk group to migrate the VDisks (see Chapter 4, “Managing Back-End Storage”).
Step 4 Display the configured MDisk group (called finance2).
switch(svc)# show cluster SampleCluster mdisk-grp finance2
mdisk-grp finance2 is online
Total capacity is 17.08 GB
Free capacity is 17.08 GB
Extent size is 16 MB

Data Migration
Number of mdisks is 1
Number of vdisks using this group is 0
Tip In order to migrate a VDisk from its existing MDisk group to a new one MDisk group, both the
MDisk groups must have the same extent size.
Step 5 Enter the cluster configuration submode for the cluster called SampleCluster.
Step 6 Issue the migration command to the new MDisk group.

switch(svc-cluster)# migrate vdisk fn-data new-mdisk-grp finance2
switch(svc-cluster)# migrate vdisk fn-log new-mdisk-grp finance2
Step 7 Exit to the svc-config submode

switch(svc)#
Step 8 Check the progress of the migration by issuing the status command.
switch(svc)# show cluster SampleCluster status migration
migrating vdisk fn-data to mdisk grp finance2 : 68%
migrating vdisk fn-log to mdisk grp finance2 : 0%
switch(svc)#
Step 9 Verify that the migration has completed, by using the show VDisk command. The mdisk-grp name
column should have the new MDisk group name.
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
fn-data 1.00 GB 1 finance2 striped online
fn-log 1.00 GB 1 finance2 striped online
switch(svc)#

FlashCopy
FlashCopy
FlashCopy copies a set of source VDisks to a set of target VDisks. The original contents of the target
VDisks are lost. After the copy operation is completed, the target VDisks have the contents of the source
VDisks as they existed at a single point-in-time, when the FlashCopy was started. Although the copy
operation takes a finite amount of time to complete, the resulting data at the target appears as if the copy
was made instantaneously. FlashCopy is sometimes described as an instance of a Time-Zero copy (T0)
or point-in-time copy technology. This time is much less than the time required to copy the same data
using conventional techniques.
Point-in-time copy techniques are used to help solve the problem of making a consistent copy of a data
set which is being constantly updated. If a copy of a data set is taken using a technology which did not
provide point-in-time semantics and the data set changes during the copy operation then the resulting
copy may contain data which is not consistent with the latest version.
FlashCopy Applications
The following list provides some examples of using FlashCopy services:
• An important use of FlashCopy service is for taking consistent backups of changing data. In this
usage, a FlashCopy is created in order to capture a point-in-time and the resulting image is backed
up to tertiary storage such as tape. In this case the FlashCopy target is primarily treated as read only
although a few writes are occasionally involved.
• Another use of FlashCopy is for application testing. In a business setting it is often important to test
a new version of an application on real business data prior to updating the production copy of the
software. This reduces the risk of the software upgrade failing because it is incompatible with the
actual data in use at the time of the update. This application requires read/write access to the
FlashCopy target.
• Other uses of FlashCopy in the business environment include creating copies for auditing purposes
and for data mining.
• In the scientific and technical arena one way in which FlashCopy is employed is to create restart
points for long running batch jobs. This means that if a batch job fails many days into its run it may
be possible to restart the job from a saved copy of its data rather than re-running the entire multi-day
job.
FlashCopy Mapping
FlashCopy mapping is done between a source VDisk and a target VDisk. The VDisks must be the same
size. FlashCopying part of a VDisk is not supported. The source and target VDisks must both be
managed by the same SVC cluster but may be in different I/O groups within that cluster.
Each VDisk may be a member of only one FlashCopy mapping. VDisks participating in FlashCopy
mapping cannot have their size increased or decreased while they remain participants of the FlashCopy
mapping.

FlashCopy
FlashCopy Consistency Groups

A FlashCopy consistency group contains a number of FlashCopy mappings. A consistency group can
contain an arbitrary number of FlashCopy mappings up to the maximum number of FlashCopy mappings
supported by a SVC cluster. When the start command is issued for a consistency group, all of the
FlashCopy mappings in the consistency group are started at the same time, resulting in a point-in-time
copy which is consistent across all of the FlashCopy mappings which are contained in the consistency
group.
The FlashCopy function accepts one of two mode to copy the source VDisk contents to the destination
VDisk—full or copy-on-write.
• Full mode—The source VDisk is copied to the destination VDisk even if the source VDisk is not
changed after the FlashCopy operation is started. The copy process is implemented at the rate
specified by the rate parameter (see Table 7-1).
Table 7-1 Copy Rate Translation
Copy-rate KB/sec.
1-10 128
11-20 256
21-30 512
41to50 2048
91 to 100 64MB
• Copy-on-write mode—The source VDisk is only copied to the destination VDisk if the source
VDisk is changed (by a write operation) after the FlashCopy operation is started.
To configure consistency groups, follow these steps.
Step 1 Create the source, and target VDisks for FlashCopy (Chapter 5, “Managing Virtual Disks”).
Step 2 Verify the VDisk configuration.
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
fndata-snapshot 1.00 GB 1 finance striped online
fnlog 2.00 GB 1 finance striped online
fnlog-snapshot 2.00 GB 1 finance striped online
fndata 1.00 GB 1 finance striped online
Note The target VDisk, and the source VDisk in a FlashCopy mapping need to be of same size.
Step 3 Enter the cluster configuration submode

Step 4 Enable the FlashCopy feature.

switch(svc-cluster)# feature enable flash-copy

FlashCopy
Note The FlashCopy feature must be licensed and enabled before configuring any FlashCopy mappings.
Step 5 Create a FlashCopy consistency group.

switch(svc-cluster)# flash-copy add fcgrp
Step 6 Enter the FlashCopy consistency group submode

switch(svc-cluster)# flash-copy name fcgrp
switch(svc-cluster-flash-copy)#
Step 7 Map the source VDisk to the target VDisk.

switch(svc-cluster-flash-copy)# map src-vdisk fnlog dst-vdisk fnlog-snapshot
switch(svc-cluster-flash-copy)# map src-vdisk fndata dst-vdisk fndata-snapshot
Step 8 Refer to Table 7-1 and obtain the required copy rate. The copy rate specifies the rate of background copy.
It is expressed as a percentage, and its translation to bandwidth is given below. If the optional copy rate
is not configured, a default rate of 50 is configured. This example uses the full mode with a copy rate of
90.
Step 9 Configure the copy rate for the FlashCopy group.
switch(svc-cluster-flash-copy)# mode full rate 90
Step 10 Exit to the SVC configuration submode

switch(svc-cluster-flash-copy)# exit
switch(svc)#
Step 11 Verify the FlashCopy configuration.

switch(svc)# show cluster SampleCluster flash-copy fcgrp
Flash-copy mapping 1:
src vdisk is fnlog
dest vdisk is fnlog-snapshot
state is idle_or_copied
copy rate is 90
progress 0% done
Flash-copy mapping 2:
src vdisk is fndata
dest vdisk is fndata-snapshot
state is idle_or_copied
copy rate is 90
progress 0% done
Starting FlashCopy
Step 12 Refer to the “Starting the FlashCopy” section on page 7-6 to start the FlashCopy configuration.
Starting the FlashCopy

You can only start the FlashCopy process if you have configured a consistency group as specified in the
“FlashCopy Consistency Groups” section on page 7-5.
To start the FlashCopy process, follow these steps:

FlashCopy
Step 1 Check the status of the FlashCopy consistency group.

switch(svc)# show cluster SampleCluster flash-copy
-----------------------------------
Name Status
-----------------------------------
fccstgrp0 idle_or_copied
fcgrp idle_or_copied
Step 2 Issue the cluster name cluster-name flash-copy fcgrp prepare command to prepare the source and
target VDisks for FlashCopy. This will flush the cache of any data destined for the source VDisk and
force the cache into write through until the FlashCopy is started.
switch(svc)# cluster name SampleCluster flash-copy fcgrp prepare
Step 3 Check the status of the FlashCopy consistency group, to make sure that the prepare operation is
completed. If the prepare operation is completed, the status for the group will be prepared. If the prepare
operation is not completed, wait till it completes.
switch(svc)# show cluster SampleCluster flash-copy
-----------------------------------
Name Status
-----------------------------------
fccstgrp0 idle_or_copied
fcgrp prepared
Step 4 Once, the FlashCopy consistency group is prepared for FlashCopy, issue the cluster name cluster-name
flash-copy fcgrp start command to start the FlashCopy. This makes a point-in-time copy of the source
VDisk the moment the command is executed.
switch(svc)# cluster name SampleCluster flash-copy fcgrp start
Step 5 Check the progress of the FlashCopy operation.

switch(svc)# show cluster SampleCluster status flash-copy fcgrp
------------------------------------------
src vdisk dest vdisk progress
------------------------------------------
fnlog fnlog-snapshot 46%
fndata fndata-snapshot 6%
If the progress fields indicate 100%, for all the mappings in the FlashCopy consistency group, then the
FlashCopy is completed.
Stopping FlashCopy
A FlashCopy, once started, can be stopped by issuing the cluster name cluster-name flash-copy fcgrp
stop command. Once stopped, use the cluster name cluster-name flash-copy fcgrp prepare command
for the FlashCopy group before it is started.

Remote Copy
Remote Copy
Remote copy is a feature that seeks to maintain two copies of a data set. The relationship between the
two copies is not symmetric. One copy of the data set is considered the primary copy—or the source.
This copy provides the reference for normal run-time operation. Updates to this copy are shadowed to a
secondary copy—or the auxiliary. The secondary copy is not normally referenced for performing I/O.
This release of SVC supports synchronous remote copy. Synchronous remote copy ensures that updates
are committed at both primary and secondary before the application is given completion to an update.
This ensures that the secondary is fully up-to-date should it be needed in a failover. However, this means
that the application is fully exposed to the latency and bandwidth limitations of the communication link
to the secondary.
Disaster Recovery
If the primary copy fails, you can enable the secondary copy for I/O operation. A typical use of this
function may involve two sites where the first provides service during normal running and the second is
only activated when a failure of the first site is detected.
The secondary copy is not accessible for application I/O other than the I/Os that are performed for the
remote copy process itself.
Enabling the secondary copy for active operation require some SVC, operating system and possibly
application specific configuration.
Enabling the secondary copy needs to be performed as part of the entire failover process. The SVC
cluster at the secondary must be instructed to Stop the relationship which will have the affect of making
the secondary logical unit accessible for normal I/O access. The operating system might need to mount
file systems. The application might have some log of work to recover.
Remote Copy Relationships

A remote copy relationship consists of two VDisks—a source VDisk and an auxiliary VDisk. In the most
common use of remote copy the source VDisk contains a production copy of the data and is the VDisk
that normally used by an application. The auxiliary VDisk contains a backup copy of the data and is used
in disaster recovery scenarios.
Both VDisks in a remote copy relationship must be of the same size.
Both VDisks can be in the same cluster (and consequently, the same I/O group). Optionally, the VDisks
can come from two clusters that are configured to recognize each other and can be in any I/O group in
each of those two clusters.
Each VDisk in a remote copy relationship takes on a specific role, behaving as a primary or a secondary.
A primary VDisk contains a valid copy of application data and is accessible for application write I/O. A
secondary VDisk contains a valid copy of application data but is not available for application write I/O.
Remote Copy Consistency Groups

Certain uses of remote copy require the manipulation of more than one relationship. Remote copy
provides the ability to group relationships so that they are manipulated in unison. For some uses it might
be that the relationships share a loose association and that the grouping simply provides a convenience
for the administrator. But a more significant use arises where the relationships contain VDisks that have

Remote Copy
a tighter association. For example, if an application’s data is spread across more than one VDisk. A more
complex example is where multiple applications run on different host systems, where each application
has data on different VDisks, and these applications exchange data with each other.
A consistency group can contain zero or more relationship. All relationships in a consistency group must
have matching source and auxiliary clusters.
Configuring Remote Copy

To configure remote copy, follow these steps.
Note In this example, we choose separate local, and remote clusters, although they can be the same cluster.
Step 1 Create the local, and remote cluster (see Chapter 3, “Creating and Managing Clusters”).
Step 2 Create the Virtual Disks in the local, and remote cluster that form part of the remote copy relationship
(Chapter 4, “Managing Back-End Storage”).
local-switch(svc)# show cluster local-cluster vdisk
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
fndata-src 2.00 GB 1 finance striped online
fnlog-src 1.00 GB 1 finance striped online
remote-switch(svc)# show cluster remote-cluster vdisk

-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
fnlog-aux 1.00 GB 1 finance striped online
fndata-aux 2.00 GB 1 finance striped online
Step 3 Enter the cluster config submode of the local cluster

local-switch(svc)# cluster config local-cluster
local-switch(svc-cluster)#
Step 4 Enable the remote copy feature in the local cluster

local-switch(svc-cluster)# feature enable remote-copy
Note The remote copy feature must be licensed and enabled before configuring remote copy.
Step 5 Enter the cluster config submode of the remote cluster

remote-switch(svc)# cluster config remote-cluster
remote-switch(svc-cluster)#
Step 6 Enable the remote copy feature in the remote cluster

remote-switch(svc-cluster)# feature enable remote-copy
Step 7 Establish a remote copy partnership with the remote cluster at the local cluster.
local-switch(svc-cluster)# remote-copy cluster remote-cluster

Remote Copy
Step 8 Establish a remote copy partnership with the local cluster at the remote cluster.
remote-switch(svc-cluster)# remote-copy cluster local-cluster
Step 9 Create a remote copy consistency group in the local cluster.

local-switch(svc-cluster)# remote-copy add rcgrp cluster remote
Step 10 Enter the remote copy consistency group submode in the local cluster.
local-switch(svc-cluster)# remote-copy name rcgrp
local-switch(svc-cluster-remote-copy)#
Step 11 Create remote copy relationships between the source VDisks, and auxiliary VDisks, under the remote
copy consistency group.
local-switch(svc-cluster-remote-copy)# map src-vdisk fndata-src aux-vdisk fndata-aux
local-switch(svc-cluster-remote-copy)# map src-vdisk fnlog-src aux-vdisk fnlog-aux
local-switch(svc-cluster-remote-copy)#
Step 12 Return to the svc-config submode in the local switch.

local-switch(svc-cluster-remote-copy)# exit
local-switch(svc-cluster)# exit
local-switch(svc)#
Step 13 Verify the remote copy consistency group configuration.

local-switch(svc)# show cluster local-cluster remote-copy rcgrp
Remote-copy mapping 1:
master cluster is local-cluster
master vdisk is fndata-src
aux cluster is remote-cluster
aux vdisk is fndata-aux
status is inconsistent_stopped
progress 0% done
master vdisk is fnlog-src
aux vdisk is fnlog-aux
status is inconsistent_stopped
progress 0% done
local-switch(svc)#
Starting Remote Copy

Once the remote copy configuration is completed for a consistency group, start (activate) the remote
copy relationships by issuing the start command. This command triggers the background copy from the
corresponding source VDisks to the auxiliary VDisks respectively. Any subsequent writes issued to a
particular source VDisk are also mirrored in the corresponding auxiliary VDisk, in the remote copy
relationship.
To start remote copy relationships, follow these steps:

Remote Copy
Step 1 Enter the svc-config submode in the local switch.

local-switch# svc-config
local-switch(svc)#
Step 2 Issue the start command for the remote copy consistency group to activate the remote copy relationships
in the consistency group.
local-switch(svc)# cluster name local-cluster remote-copy rcgrp start
local-switch(svc)#
Step 3 Check the progress of the remote copy by issuing the Status command.
local-switch(svc)# show cluster local-cluster status remote-copy rcgrp
------------------------------------------
src vdisk dest vdisk progress
------------------------------------------
fndata-src fndata-aux 8%
fnlog-src fnlog-aux 16%
local-switch(svc)#
Step 4 Once, the background copy of the data in source VDisks to auxiliary VDisks are completed, the status
of all the relationships in the consistency group will be consistent_synchronized, and the auxiliary
VDisks will be up to date with the corresponding source VDisks.
local-switch(svc)# show cluster local-cluster remote-copy rcgrp
master vdisk is fndata-src
aux vdisk is fndata-aux
status is consistent_synchronised
progress 100% done
master vdisk is fnlog-src
aux vdisk is fnlog-aux
status is consistent_synchronised
progress 100% done
local-switch(svc)#
Stopping Remote Copy

Use the stop command to suspend the remote copy relationships in a consistency group. Subsequently,
the writes issued to a source VDisk, in a relationship, are not reflected at the corresponding auxiliary
VDisk.
local-switch(svc)# cluster name local-cluster remote-copy rcgrp stop
local-switch(svc)#

Remote Copy
Failover and Recovery Process

When remote copy relationships are active during a normal operation, the source VDisks in the
relationships are fully accessible, and the auxiliary VDisks are not available for write operations.
After a failure in the local (master) cluster, you must stop the remote copy relationship using the
aux-enable option, to allow write access to the auxiliary VDisks in the remote cluster.
remote-cluster(svc)# cluster name remote-cluster remote-copy rcgrp stop aux-enable
remote-cluster(svc)#
If the local cluster comes back up, the administrator can choose to resume the remote copy relationships
in one of two ways.
• Resume the remote copy relationships with the local cluster acting as the primary or master of the
relationships.
• Resume the remote relationships with the remote cluster acting as the primary or master of the
relationships. In either case, the force option is required when you resume the remote
copy—background copy is required to make the source VDisks, and auxiliary VDisks up to date.
The following command resumes the remote copy relationships in the consistency group with the local
cluster as the primary.
local-cluster(svc)# cluster name local-cluster remote-copy rcgrp start force
local-cluster(svc)#
The following commands resume the remote copy relationships in the consistency group with the remote
cluster as the primary, by enabling the aux option in the start command.
remote-cluster(svc)# cluster name remote-cluster remote-copy rcgrp start aux force
Verify the primary configuration for a remote copy consistency group using the show remote-copy
command. The Primary column indicate whether the auxiliary VDisks or source VDisks have the
primary (or master) role.
remote-cluster(svc)# show cluster remote-cluster remote-copy
----------------------------------------------------------
Name Remote Cluster Mappings Primary Status
----------------------------------------------------------
rcgrp remote 2 aux consistent_synchronised

C H A P T E R 8
Upgrading CSM Software
When CSMs are present in any switch in the Cisco MDS 9000 Family, several kinds of upgrade may be
performed as required—a cluster software upgrade, an automatic upgrade when nodes are added, a
service mode upgrade, or a switch software upgrade.
This section also explains the process to manage pWWNs and nWWNs when CSM modules are replaced
in any switch in the Cisco MDS 9000 Family switch.
• Upgrading Clusters, page 8-2
• Automatic Upgrade during Node Addition, page 8-5
• Upgrading in Service Mode, page 8-5
• Upgrading the Switch Software, page 8-7
• Replacing CSMs, page 8-9

Chapter 8 Upgrading CSM Software
Upgrading Clusters
Upgrading Clusters
Cluster software upgrades are done concurrently with user I/O operations and selected management
activities.
Note When software upgrades are in progress, not all nodes in the cluster are operational depending on the
upgrade phase. Consequently, the cache operates in write through mode.
Tip Configuration changes are disallowed from the time the upgrade is started until the upgrade operation
has terminated successfully. In case of failure, the upgraded and/or failed nodes must be reverted to the
original software version before configuration changes are permitted. If any configuration change is
attempted during this time, an error message is issued to indicate that an upgrade is in progress.
Cluster Upgrade Prerequisites

To prepare a switch for cluster software upgrade, follow these steps.
Step 1 Wait for all data migration to complete. The time taken for the data migration depends on the size of the
VDisks being migrated.
Caution Data migration, once started, cannot be stopped.
Step 2 Wait for all FlashCopy mappings to complete or stop them. If you choose to stop the FlashCopy
mapping, this will result in the FlashCopy targets going offline. The FlashCopy must be prepared and
started again in order to restart FlashCopy. This procedure results in the FlashCopy point-in-time being
lost.
Step 3 Stop all remote copy relationships.
Recognizing Failure Situations

Cluster software upgrade will fail if one of the following situations apply:
• Any node configured to be a member of the cluster is not present—in order to upgrade the software
the node must either be deleted from the cluster or must be brought online.
• All the nodes configured into the cluster do not currently have the same software level—may happen
as result of a failed back out or a service mode upgrade. This behavior cannot be overridden using
the force option.
• The new SVC software image is not compatible with the current switch or the SVC software level.
This behavior cannot be overridden using the force option.
• A node has been deleted from the cluster such that any I/O group has only one member—the upgrade
process would result in loss of access to data if this were allowed. The force option can be used to
override this restriction if you do not mind losing access to data during the upgrade.

Upgrading Clusters
Cluster Upgrade Guidelines

Be aware of the following guidelines when upgrading clusters:
• Nodes are updated one at a time. One node from each I/O group is updated before the second node
in any I/O group is updated.
• During the update of a node it does not participate in I/O activity in the I/O group. Thus all I/O
activity for the VDisks in the I/O group is directed to the other node in the I/O group by the host
multipathing software. During a node upgrade, the other node in the I/O group will attempt to flush
the cache and change operations to write-through mode. This flush is not guaranteed to be successful
or complete. Should the remaining node in an I/O group experience a failure when its’ partner is
being upgraded, then the only valid copy of the modified data in the cache could be lost. If this risk
is unacceptable, then the customer must take steps to ensure that no I/O activity takes place during
the software upgrade.
• A 30 minute delay is inserted into the procedure between updating the first and second nodes in each
I/O group. This allows time for the host multipathing software to rediscover paths to the nodes that
were updated first so that when the second set of nodes are updated loss of access does not result.
• The update is not committed until all nodes in the cluster have been successfully updated to the new
code level.
• When code upgrade starts an entry is made in the event log and another entry is made when the
upgrade completes or fails.
• The config node goes down to upgrade to the new software level. This will cause a different node to
become the new config node. At this point, you need to reconnect to the cluster IP address.
Performing the Cluster Upgrade

Step 1 Issue the cluster name cluster-name upgrade svc-system command at the configuration node of the
cluster.
Tip If the cluster contains an I/O group with a single node, use the optional force keyword at the end of this
command.
switch(svc)# cluster name SampleCluster upgrade svc-system

scp://userc@171.69.16.22/auto/andusr/userc/hkrel/VegasSW/build/gdb.sb-avanti/isan/targetfs
/m9000-ek9-csm-svc-mzg.1.3.x.bin
userc@171.69.16.22's password:
m9000-ek9-csm-svc-mz 100% |***************************************| 108 MB 01:23
This process may take few minutes as the config node performs the following checks:
• If the new SVC software image is compatible with the config node’s running switch software and
running SVC software. If any compatibility checks fail, cluster upgrade fails and an appropriate
error message is issued.
• If the SVC software image is compatible with the software running on each of the switches where
the other nodes in the cluster reside.
• The config node collects the compatibility check result from each node in the cluster and makes sure
that all of them have succeeded. If any of nodes report a check failure, then an error message is
issued with a table displaying all the check failures across all the nodes in the cluster.

Upgrading Clusters
• If all the nodes in the cluster are running the same level of code and all nodes are present.
• If the new SVC software image has been successfully transmitted to all the nodes in the cluster.
If all the checks succeed in Step 1 the upgrade process begins.
Step 2 Verify that cluster upgrade has succeeded by issuing one of two commands in SVC configuration mode:
• the show node local command—Ensure that the node software version corresponds to the new node
software version.
• the show cluster SampleCluster nodevpd command—Ensure that the software upgrade complete
event message is in the supervisor syslog.
Sample Cluster Upgrade

switch(svc)# show node local
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
svc3/1 SampleCluster No active active 1.3(1)
switch(svc)# cluster name SampleCluster upgrade svc-system

userc@171.69.16.22's password:
m9000-ek9-csm-svc-mz 100% |***************************************| 108 MB 01:23
Note The cluster upgrade process has initiated successfully when the prompt returns.
switch(svc)# show node local

-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
svc3/1 SampleCluster No active active 1.3(x)
In this example, node svc4/1 has completed upgrade to the new version of the svc software.

Automatic Upgrade during Node Addition
Automatic Upgrade during Node Addition

If a node is running a SVC software version which is different from the cluster version, and that node is
added to the cluster, the config node automatically downloads the cluster software to the new node. If
the cluster software is not compatible with the software running on the switch that contains the new node,
the add node operation will fail.
Upgrading in Service Mode

Service mode software upgrade provides a recovery alternative if the cluster cannot accept a normal
software upgrade. A single node is placed in service access mode to perform a service mode software
upgrade. This is in contrast to the normal method for upgrading all the nodes in the cluster.
Caution This upgrade process is only to be used by experienced switch administrators under the care of customer
support representatives.
To upgrade a single node in service mode, follow these steps.
Step 1 Issue the node svc slot/node servicemode command on the required node to place the node in service
mode.
switch(svc)# node svc 7/2 servicemode
switch(svc)#
Step 2 Issue the show node local command in SVC configuration mode to verify that the required node is in
servicemode.
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
svc7/2 SampleCluster No unconfigured servicemode 1.3(1)
Step 3 Issue the node svc command to begin the upgrade.'

switch(svc)# node svc 7/2 upgrade svc-system
This command checks if the new SVC software image is compatible with the running switch software
and the running SVC software.
If any of these compatibility checks fail, then node upgrade fails with the appropriate error message to
the user.
Step 4 Issue the show node local command in SVC configuration mode to verify that the required node is
running with new node software.
-------------------------------------------------------------------------------

Upgrading in Service Mode
-------------------------------------------------------------------------------
Note The node automatically exits the service mode when the upgrade is completed45.
Sample Service Mode Upgrade

-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
switch(svc)# node svc 7/2 servicemode

-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
svc7/2 SampleCluster No unconfigured servicemode 1.3(1)
switch(svc)# node svc 7/2 upgrade svc-system


-------------------------------------------------------------------------------
-------------------------------------------------------------------------------

Upgrading the Switch Software

This upgrade requires a dual-supervisor MDS 9500 director. This procedure only upgrades the switch
software—not the SVC software that is running on the nodes in the cluster.
Use the install all command to do switch software upgrade irrespective of the modules present in the
switch.
Performing the Switch Upgrade

To perform an automated software upgrade on any switch, follow these steps:
Step 1 Log into the switch through the console, Telnet, or SSH port of the active supervisor.
Step 2 Perform the upgrade by issuing the install all command.
switch# install all system bootflash:system-image kickstart bootflash:kickstart-image
At this point, the following events happen:

• The installer checks if the new switch system software is compatible with the SVC software for both
nodes in a CSM and for all CSMs in the switch.
• You will receive the following descriptive information on the intended changes to your system:
– a compatibility assessment for each module in the switch
– a upgrade assessment
– a module version comparison
See the “Sample Switch Upgrade” section on page 8-7 for an example of this information.
• Once the effects of the command are presented, you can choose to continue or cancel when you see
this question (the default is no):
Do you want to continue y/n? [n] :y
Step 3 Select y if you choose to continue or n if you choose to cancel the upgrade.
• y = each CSM module is upgraded—one at a time, with a gap of 30 minutes between each to ensure
that only one node in the I/O group is down at any time. This allows time for the host multipathing
software to rediscover paths to the modules containing nodes that were upgraded first.
– If incompatibility warnings exist, the incompatible CSM nodes are shut down after the upgrade.
– If incompatibilities do not exist, the CSM nodes are upgraded.
• n = the installation process is aborted.
Step 4 Exit the switch console and open a new terminal session to view the upgraded supervisor module using
the show module command.
Sample Switch Upgrade

switch# install all system
scp://usery@171.69.16.22/auto/vwsvkd/usery/m9500-sf1ek9-mzg.1.3.x.bin
For scp://usery@171.69.16.22, please enter password:

Copying image from scp://usery@171.69.16.22/auto/vwsvkd/usery/m9500-sf1ek9-mzg.1.3.x.bin

to bootflash:///m9500-sf1ek9-mz
.1.3.x.bin.
[####################] 100% -- SUCCESS
Verifying image bootflash:///b96d

[####################] 100% -- SUCCESS
Verifying image bootflash:///m9500-sf1ek9-mzg.1.3.x.bin

[####################] 100% -- SUCCESS
Extracting “svclc” version from image bootflash:///m9500-sf1ek9-mzg.1.3.x.

in.
[####################] 100% -- SUCCESS
Extracting “slc” version from image bootflash:///m9500-sf1ek9-mzg.1.3.x.bi

.
[####################] 100% -- SUCCESS
Extracting “system” version from image bootflash:///m9500-sf1ek9-mzg.1.3.x

bin.
[####################] 100% -- SUCCESS
Extracting “kickstart” version from image bootflash:///b96d.

[####################] 100% -- SUCCESS
Extracting “loader” version from image bootflash:///b96d.

[####################] 100% -- SUCCESS
Compatibility check is done:

Module bootable Impact Install-type Reason
------ -------- -------------- ------------ ------
2 yes disruptive reset To be installed sys image is incompatible with
node 1 running image
2 yes disruptive reset To be installed sys image is incompatible with
node 2 running image
Images will be upgraded according to following table:

Module Image Running-Version New-Version Upg-Required
------ ---------- -------------------- -------------------- ------------
1 slc 1.3(1) 1.3(x) yes
1 bios v1.0.8(08/07/03) v1.1.0(10/24/03) yes
2 svclc 1.3(1) 1.3(x) yes

2 svcsb 1.3(3) 1.3(3) no
2 svcsb 1.3(3) 1.3(3) no
2 bios v1.0.8(08/07/03) v1.1.0(10/24/03) yes
3 svclc 1.3(1) 1.3(x) yes

3 svcsb 1.3(1) 1.3(1) no
3 svcsb 1.3(1) 1.3(1) no
3 bios v1.0.8(08/07/03) v1.1.0(10/24/03) yes
5 system 1.3(1) 1.3(x) yes

5 kickstart 1.3(1) 1.3(1) no
5 bios v1.0.8(08/07/03) v1.1.0(10/24/03) yes
5 loader 1.2(2) 1.2(2) no
6 system 1.3(1) 1.3(x) yes

6 kickstart 1.3(1) 1.3(1) no
6 bios v1.0.8(08/07/03) v1.1.0(10/24/03) yes
6 loader 1.2(2) 1.2(2) no

Replacing CSMs
7 svclc 1.3(1) 1.3(x) yes

7 svcsb 1.3(1) 1.3(1) no
7 svcsb 1.3(1) 1.3(1) no
7 bios v1.0.8(08/07/03) v1.1.0(10/24/03) yes
Do you want to continue with the installation (y/n)? [n] y
Caution If you type yes at this point, the switch upgrade will proceed. Since the software running on the nodes
2/1 and 2/2 are not compatible, after the switchover the nodes 2/1, 2/2 are shutdown. Nodes 3/1, 3/2, 7/1,
and 7/2 on the other hand will be up and running since there were no incompatibility warning messages.
Tip If the cluster spans multiple switches, we recommend that all switches run the same version of the switch
software. When upgrading switch software in a multi-switch environment, be sure to update one switch
at a time.
Replacing CSMs
When you replace the CSM, you must ensure that the new CSM is using valid nWWNs and pWWNs.
You may choose to install the new CSM in a different slot or in the same slot. The process to replace the
CSM differs based on this decision.
Tip To avoid the need to reconfigure servers and controllers, we recommend that you configure the
replacement nodes with the same nWWNs and pWWNs as the replaced nodes. The procedure provide
in this section follows this recommendation.
Caution If nodes being replaced are given the same nWWNs or pWWNs as previous nodes that were participating
in a cluster, they must be added to same I/O group and the same cluster as the nodes being replaced.
Adding nodes with the same nWWNs or pWWNs (as the replaced nodes) to a different I/O group or
cluster, can result in data corruption. Refer to the IBM TotalStorage Subsystem Device Driver User's
Guide for more information.
If the nodes being replaced are given new nWWNs or pWWNs, then perform the following additional
steps after adding the nodes back to the cluster:
Step 1 At the end of the recovery process, follow the SDD procedure to discover the new paths and to check
that each VPath is presenting the correct number of paths. Refer to the IBM TotalStorage Subsystem
Device Driver User's Guide for more information.
Step 2 You may also need to modify the configuration of your disk controllers. If your controller uses a mapping
technique to present its RAID arrays or partitions to the cluster, you must modify the port groups that
belong to the cluster because the nWWN or pWWN's of the node have changed.

Replacing CSMs
Prerequisites to Replacing a CSM

Step 1 Use the show nodes local and show interface svc slot/node commands to obtain the following
information for each of the nodes in the CSM:
• The slot number in which the CSM is located
• The node number within the CSM (1 or 2)
• The node name—If the default name was used, then you cannot keep the same node name.
• The cluster (name) to which this node belongs
• The I/O group to which this node belongs
• The applicable nWWN and pWWNs.
• The current software version on the node
Document this information in an easily accessible location—you will be using this information to
upgrade the software after replacing a CSM.
Step 2 Verify that the node is not functional.
Step 3 Verify that the other node in its I/O group is operational before deleting this node.
Step 4 Delete the nodes to be replaced from their cluster(s).
Replacing a CSM in the Same Slot

If a CSM is replaced by another CSM in the same slot in any switch in the Cisco MDS 9500 Series or in
a Cisco MDS 9216 Switch, the same nWWNs and pWWNs are automatically assigned for both interfaces
on the new CSM. No further configuration is required.
Caution If the replacement nodes are assigned the same nWWNs and pWWNs as the replaced nodes, be sure to
assign the nodes to the same I/O group and cluster as before. Otherwise, data corruption may occur. If
the information on which I/O group and cluster the previous nodes were part of is not available, contact
your reseller (if applicable) or customer service for assistance.
If a CSM is replaced by another CSM in the same slot on the same chassis, and you do not wish to retain
the same nWWNs and pWWNs, follow this process.
Step 1 Remove the CSM from the slot as directed in the Cisco MDS 9216 Switch or the Cisco MDS 9500 Series
Hardware Installation Guide.
Step 2 Use the svc purge-wwn module command to erase the nWWNs and pWWNs from the original slot.
Issue the command after the module has been removed or when it is in the powered-down state.
switch# svc purge-wwn module <slot-num>

Replacing CSMs
The old nWWNs and pWWNs will be lost from the system and never reassigned for any purpose in that
chassis.
Step 3 Replace the new CSM in the same slot as directed in the Cisco MDS 9216 Switch or the Cisco MDS 9500
Series Hardware Installation Guide. The interfaces on this new CSM will have brand new nWWN and
pWWNs.
Replacing a CSM in a Different Slot

If a CSM is replaced by another CSM in a different slot on the same or different chassis, and the same
nWWNs and pWWNs are to be retained, follow this process.
Step 1 Document the nWWN and pWWN values for the CSM before removing it.
Step 2 Remove the CSM from the slot as directed in the Cisco MDS 9216 Switch or the Cisco MDS 9500 Series
Hardware Installation Guide.
Step 3 Use the following command to erase the nWWNs and pWWNs from the original slot. Issue the command
when the module has already been removed or when it is in the powered-down state.
switch# svc purge-wwn module <slot-num>
The old nWWNs and pWWNs will be lost from the system and never reassigned for any purpose in that
chassis.
Step 4 Replace the new CSM in the desired slot as directed in the Cisco MDS 9216 Switch or the Cisco MDS
9500 Series Hardware Installation Guide.
Step 5 Wait for the module to initialize.
Step 6 Configure the VSAN for the initiator, target, and management N-ports to match the replaced nodes:
switch# config t
switch (config)# interface svc new-slot-num / node-num
switch (config-if)# initiator vsan vsan-id
switch (config-if)# target vsan vsan-id
switch (config-if)# mgmt vsan vsan-id
Step 7 Set the nWWN and pWWN values for the two interfaces within the module with the following
commands:
switch (config-if)# shutdown
switch (config-if)# nwwn saved-nwwn-value
switch (config-if)# initiator vsan vsan-id pwwn saved-pwwn-value
switch (config-if)# target vsan vsan-id pwwn saved-pwwn-value
switch (config-if)# mgmt vsan vsan-id pwwn saved-pwwn-value
Tip The related SVC interface must be shut down before setting the WWNs.
Step 8 Reload the CSM using the reload module slot-number command for the new nWWNs to take effect.

Replacing CSMs
Post-Replacement Verification
To perform a post-replacement verification, follow these steps.
Step 1 Use the show nodes local command to verify that the nodes in the CSM have initialized without errors.
If node does not initialize, the software version of the nodes on the replacement CSM may not be
compatible with the version running on the switch.
Step 2 Wait for the nodes to initialize.
Step 3 Add the nodes back to their clusters (see the Chapter 8, “Upgrading CSM Software”).
Tip If the previous node used the default name, you cannot reassign the default name to the new node. If you
assigned a name to the old node, the new node can be assigned the same name.
Note The Node ID of the replacement node is different from the node ID of the replaced node.
Step 4 Use the SDD management tool on the host systems to verify that all paths are online.

C H A P T E R 9
Configuring SPAN on SVC Interfaces
Effective SAN-OS Release 1.3(x) Switched Port Analyzer (SPAN) capabilities are also available on the
Caching Services Module (CSM) module.
The SPAN feature is specific to switches in the Cisco MDS 9000 Family. It monitors network traffic
though a interface. Traffic through any SVC interface can be replicated to a special port called the SPAN
destination port (SD port).
• About SVC as a SPAN Source, page 9-2
• Configuring SVC Interfaces as SPAN Sources, page 9-2

Chapter 9 Configuring SPAN on SVC Interfaces
About SVC as a SPAN Source
About SVC as a SPAN Source

SPAN sources refer to the interfaces from which traffic can be monitored. You can also specify CSM
nodes as a SPAN source, in which case, the SVC virtual interfaces (N-ports) are included as SPAN
sources. You can choose the SPAN traffic in the ingress direction, the egress direction, or both directions
for any source interface (see Figure 9-1).
Figure 9-1 SVC Interfaces as SPAN Sources
Initiator
Host VSAN Management VSAN Disk VSAN
rx tx
FC traffic that
can be spanned rx tx tx rx
Target Mgmt Initiator

N-Port N-Port N-Port
105738
Caching Services Node (svc1/1)
You can also specify the traffic type (initiator traffic, target traffic, or management traffic) in both the
ingress and egress directions or in either direction. To specify this option, use the traffic-type parameter.
Refer to the “Multiple Initiators and Targets” section on page 2-9 for additonal information on
configuring multiple N-port VSANs.
Configuring SVC Interfaces as SPAN Sources

To create SVC interfaces as SPAN sources, follow these steps.
Step 1 Enter the MDS configuration mode.

switch# config t
switch(config)#
Step 2 Configure a SPAN session.

switch(config)# span session 2
switch(config-span)#
If the session does not exist, it will be created.

Step 3 Configure the destination interface for the SPAN session.
switch(config-span)# destination interface fc1/9
Step 4 Configure the source interface svc1/1 for the initiator, target, and mgmt traffic-types in both directions.

Using VSAN Filters to Specify N-Port Traffic
switch(config-span)# source interface svc1/1
Note The traffic-type option is specific to SVC interfaces and cannot be used with any other SPAN source
interfaces type.
Step 5 Configure the source interface svc2/1 for all initiator and mgmt traffic in the rx direction, and for all
target traffic in the tx direction.
switch(config-span)# source interface svc2/1 rx traffic-type initiator
switch(config-span)# source interface svc2/1 rx traffic-type mgmt
switch(config-span)# source interface svc2/1 tx traffic-type target
Step 6 Configure the source interface in the rx direction for the initiator, target, and mgmt traffic-types in the
rx direction.
sw(config-span)# source interface svc1/1 rx
Step 7 Display the configured SPAM session to verify the configuration.

switch# show span session 2
Session 2 (admin suspended)
Destination is fc1/9
No session filters configured
Ingress (rx) sources are
svc1/1(inititator), svc1/1 (target), svc1/1 (mgmt)
svc2/1(initiator), svc2/1(mgmt)
Egress (tx) sources are
svc1/1(inititator), svc1/1 (target), svc1/1 (mgmt)
svc2/1(target)
...
You have now configured SVC interfaces as SPAN sources. Refer to the Cisco MDS 9000 Family
Configuration Guide for further details on the SPAN feature.

By default, traffic on all N-ports are spanned. You can monitor traffic on a specific N-port using the
VSAN filtering SPAN feature.
For example, if a SVC interface (3/1) with three initiator N-ports. Each N-port is in VSAN 3, 4, and 5
respectively. The following procedure configures the steps to SPAN the initiator N-port in VSAN 3.
Refer to the Cisco MDS 9000 Family Configuration Guide for further details on using VSAN filters.
To configure a SPAN filter, follow these steps:
Step 1 Enter the MDS configuration mode.

switch# config t
switch(config)#
Step 2 Configure a SPAN session.

switch(config)# span session 3
switch(config-span)#
Step 3 Configures the source svc 3/1 interface in the egress (tx) direction for all initiator traffic.

switch(config-span)# source interface svc 3/1 tx traffic-type initiator
Step 4 Configures VSAN 2 as a session filter to only span traffic in VSAN 2. This will specifically monitor
traffic on the initiator N-port traffic in VSAN 2.
switch(config-span)# source filter vsan 2
Step 5 Configures VSAN 4 and 5 as a session filter to additionally span initiator N-port traffic in these two
VSANS.
switch(config-span)# source filter vsan 4-5

C H A P T E R 10
Configuring a Dual Fabric SAN Environment
Dual fabric SAN environments are an important configuration requirement. You can use CSM modules
in combination with the Inter-VSAN Routing (IVR) feature to operate across two isolated fabrics.
• Overview, page 10-2
• Basic SVC Requirements, page 10-2
• Dual Fabric Prerequisites, page 10-2
• Sample Configuration, page 10-2

Chapter 10 Configuring a Dual Fabric SAN Environment
Overview
Overview
Redundant isolated fabrics (fabrics without an ISL connecting them) are used to ensure that disruptions
in one fabric do not affect the other. Hosts and disk subsystems in such an environment can be configured
to ensure that at least one port remains attached to each fabric.
Basic SVC Requirements

SVC has the following requirements:
• All SVC nodes must be able to communicate with each other in the management VSAN.
• All SVC nodes must be able to use the same preferred target controller port for a given Mdisk.
In a redundant, dual fabric configuration, the Cisco MDS Inter-VSAN Routing (IVR) feature must be
used in order to satisfy these basic SVC requirements. IVR provides the same benefit of keeping the two
fabrics separate while providing the connectivity required to implement SVC. Refer to Cisco MDS 9000
Family Configuration Guide for more information on IVR.
Dual Fabric Prerequisites

To configure a dual fabric SAN environment using IVR and SVC, verify the following requirements:
• Every VSAN must be assigned a unique number. This number should not be used for any other
VSAN in both fabrics. This ensures that the IVR VSAN is not carried across any other
non-dedicated dual-fabric E port.
• The same IVR configuration must be configured in each switch in each fabric.
• The default zone in the transit VSAN should be set to deny.
• The transit VSAN must only be dedicated for the dual fabric purpose.
• New interfaces should not be added to this VSAN once the configuration is implemented.
• Trunk Mode must be enabled for the transit VSAN.
• Route advertisement must be disabled for the transit VSAN.
• The Domain ID for the transit VSAN must be static and domain merges must be disabled on this
VSAN.
• Topology discovery must be disabled for the transit VSAN.
Sample Configuration
Figure 10-1 displays a dual fabric SAN environment example used in this configuration. The procedure
to configure this sample scenario is provided after the illustration.

Figure 10-1 Traffic Isolation and Connectivity Using SVC and IVR
Host 1
fc8/3 fc7/3
Initiator Initiator
VSAN 3 VSAN 13
Management Transit Management
fc8/17 fc7/17
VSAN 4 VSAN 20 VSAN 14
Switch Switch
Target Target
VSAN 2 VSAN 12
fc8/2 fc7/2
120127
IBM controller
To configure a dual fabric SAN environment in the first fabric, follow these steps:
Step 1 Get the switch WWN information for both switches.

switch1# show wwn switch
Switch WWN is 20:00:00:05:30:00:41:de
switch2# show wwn switch

Switch WWN is 20:00:00:05:30:00:07:1e
Step 2 Create the VSANs in Switch1.

switch1# configure terminal
switch1(config-vsan-db)# vsan 2 interface fc8/2 <-- target port
switch1(config-vsan-db)# vsan 3 interface fc8/3 <-- initiator port
switch1(config)#
Step 3 Set the static domains in Switch 1.

Sw1(config)# fcdomain domain 2 static vsan 2
Step 4 Configure the SVC interface into the correct VSANs in Switch 1.
switch1(config)# int svc 1/1 - 2
switch1(config-if)# no initiator vsan 1
switch1(config-if)# initiator vsan 2
switch1(config-if)# no target vsan 1
switch1(config-if)# target vsan 3
switch1(config-if)# no mgmt vsan 1

Step 5 Set the default zone configuration for the transit VSAN in Switch 1.
switch1(config)# no zone default-zone permit vsan 20
Step 6 Configure the target and SVC initiators in a zone in Switch 1.

switch1(config)# zone name z_2 vsan 2
switch1(config-zone)# member pwwn 20:06:00:a0:b8:0f:6c:34 <-- target
switch1(config-zone)# member pwwn 27:8d:00:05:30:00:33:2a <-- svc initiator
switch1(config-zone)# member pwwn 27:89:00:05:30:00:33:2a <-- svc initiator
switch1(config-zone)# exit
switch1(config)# zoneset name zs_2 vsan 2
switch1(config-zoneset)# member z_2
switch1(config-zoneset)# exit
switch1(config)# zoneset activate name zs_2 vsan 2
Zone set activation is now initiated. Check zone status.
Step 7 Configure the initiator and SVC targets in a zone in Switch 1.

switch1(config-zone)# member pwwn 21:01:00:e0:8b:29:f0:04 <-- initiator
switch1(config-zone)# member pwwn 27:8e:00:05:30:00:33:2a <-- svc target
switch1(config-zone)# member pwwn 27:8a:00:05:30:00:33:2a <-- svc target
Step 8 Configure the SVC management N ports in a zone in Switch 1.

switch1(config-zone)# member pwwn 27:90:00:05:30:00:33:2a <-- svc mgmt
switch1(config-zone)# member pwwn 27:8f:00:05:30:00:33:2a <-- svc mgmt
Step 9 Enable and configure IVR using the switch WWNs (see Step 1 for the required WWNs).
switch1(config)# ivr enable
switch1(config)# ivr vsan-topology database
switch1(config-ivr-topology-db)# autonomous-fabric-id 1 switch-wwn 20:00:00:05:30:00:41:de
vsan-ranges 2,4,20
switch1(config-ivr-topology-db)# autonomous-fabric-id 1 switch-wwn 20:00:00:05:30:00:07:1e
vsan-ranges 12,14,20
switch1(config-ivr-topology-db)# exit
switch1(config)# ivr vsan-topology activate
Step 10 Set up the IVR zone to allow SVC initiators in Switch 1 to talk to the target port in Switch 2.
switch1(config)# ivr zone name ivr_2_12
switch1(config-ivr-zone)# member pwwn 27:8d:00:05:30:00:33:2a vsan 2
switch1(config-ivr-zone)# member pwwn 27:89:00:05:30:00:33:2a vsan 2
switch1(config-ivr-zone)# member pwwn 20:07:00:a0:b8:0f:6c:34 vsan 12
switch1(config-ivr-zone)# exit
Step 11 Set up the IVR zone to allow SVC initiators in Switch 2 to talk to target port in Switch 1.
switch1(config-ivr-zone)# member pwwn 23:30:00:05:30:00:8d:e2 vsan 12


Step 12 Set up the IVR zone to allow SVC management on both switches to talk to each other.
switch1(config-ivr-zone)# member pwwn 27:8f:00:05:30:00:33:2a vsan 4
Step 13 Create the IVR zone set with the IVR zones and activate the IVR zone set.
switch1(config)# ivr zoneset name ivr_zs
switch1(config-ivr-zoneset)# member ivr_2_12
switch1(config-ivr-zoneset)# exit
switch1(config)# ivr zoneset activate name ivr_zs
Zone set activation is now initiated. Check inter-VSAN zoneset status.
If any VSANs are configured to permit the default-zone, then you must use the force option to activate
the IVR zone set
switch1(config)# ivr zoneset activate name ivr_zs force
Zone set activation is now initiated. Check inter-VSAM zoneset status.
Step 14 Bring up all interfaces, including the E-port, for the IVR transit VSAN.
switch1(config)# interface fc8/2, fc8/3
switch1(config-if)# no shutdown
switch1(config)# interface fc8/17
switch1(config-if)# switchport trunk mode on
switch1(config-if)# switchport mode E
switch1(config-if)# switchport trunk allowed vsan 20
You have now configured the first fabric for the dual fabric SAN environment displayed in Figure 10-1.
To configure a dual fabric SAN environments in the second fabric, follow these steps:
Step 1 Get the switch WWN information for both switches (see Step 1 in the previous procedure).
Step 2 Create the VSANs in Switch2.
switch2# configure terminal
switch2(config-vsan-db)# vsan 12 interface fc7/2 <-- target port
switch2(config-vsan-db)# vsan 13 interface fc7/3 <-- initiator port
Step 3 Set the static domains in Switch 2.

switch2(config)# fcdomain domain 12 static vsan 12


Step 4 Configure the SVC interface into the correct VSANs in Switch 2.
switch2(config)# int svc 2/1 - 2
switch2(config-if)# no initiator vsan 1
switch2(config-if)# initiator vsan 12
switch2(config-if)# no target vsan 1
switch2(config-if)# target vsan 13
switch2(config-if)# no mgmt vsan 1
Step 5 Set the default zone configuration for the transit VSAN in Switch 2.
switch1(config)# no zone default-zone permit vsan 20
Step 6 Configure the target and SVC initiators in a zone in Switch 2.

switch2(config-zone)# member pwwn 20:07:00:a0:b8:0f:6c:34 <-- target
switch2(config-zone)# member pwwn 23:30:00:05:30:00:8d:e2 <-- svc initiator
switch2(config-zone)# member pwwn 23:31:00:05:30:00:8d:e2 <-- svc initiator
Step 7 Configure the initiator and SVC targets in a zone in Switch 1.

switch2(config-zone)# member pwwn 21:00:00:e0:8b:09:f0:04 <-- initiator
switch2(config-zone)# member pwwn 23:32:00:05:30:00:8d:e2 <-- svc target
switch2(config-zone)# member pwwn 23:33:00:05:30:00:8d:e2 <-- svc target
switch2(config-zone)# zoneset name zs_13 vsan 13
switch2(config-zoneset)# zoneset activate name zs_13 vsan 13
Step 8 Configure the SVC management N ports in a zone in Switch 1.

switch2(config-zone)# member pwwn 23:34:00:05:30:00:8d:e2 <-- svc mgmt
switch2(config-zone)# member pwwn 23:35:00:05:30:00:8d:e2 <-- svc mgmt
switch2(config-zone)# zoneset name zs_14 vsan 14
switch2(config-zoneset)# zoneset activate name zs_14 vsan 14
Step 9 Enable and configure IVR using the switch WWNs (see Step 1 in the previous procedure).
switch2(config)# ivr vsan-topology database
switch2(config-ivr-topology-db)# autonomous-fabric-id 1 switch-wwn 20:00:00:05:30:00:41:de
vsan-ranges 2,4,20
switch2(config-ivr-topology-db)# autonomous-fabric-id 1 switch-wwn 20:00:00:05:30:00:07:1e
vsan-ranges 12,14,20
switch2(config-ivr-topology-db)# exit
switch2(config)# ivr vsan-topology activate
Step 10 Set up the IVR zone to allow SVC initiators in Switch 1 to talk to the target port in Switch 2.

switch2(config-ivr-zone)# member pwwn 27:8d:00:05:30:00:33:2a vsan 2

Step 11 Set up the IVR zone to allow SVC initiators in Switch 2 to talk to target port in Switch 1.
Step 12 Set up the IVR zone to allow SVC management on both switches to talk to each other.
switch2(config-ivr-zone)# member pwwn 27:8f:00:05:30:00:33:2a vsan 4
Step 13 Create the IVR zone set with the IVR zones and activate the IVR zone set.
switch2(config)# ivr zoneset name ivr_zs
switch2(config-ivr-zoneset)# exit
switch2(config)# ivr zoneset activate name ivr_zs
If any VSANs are configured to permit the default-zone, then you must use the force option to activate
the IVR zone set
switch1(config)# ivr zoneset activate name ivr_zs force
Step 14 Bring up all interfaces, including the E-port, for the IVR transit VSAN.
switch2(config)# interface fc7/2, fc7/3
switch2(config)# interface fc7/17
switch1(config-if)# switchport trunk mode on
switch1(config-if)# switchport mode E
switch2(config-if)# switchport trunk allowed vsan 20
You have now configured the second fabric for the dual fabric SAN environment displayed in
Figure 10-1.



Cisco MDS 9000 Family SAN Volume Controller Configuration Guide

Hochgeladen von

Dokumentinformationen

Copyright

Verfügbare Formate

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

Verfügbare Formate

Cisco MDS 9000 Family SAN Volume Controller Configuration Guide

Hochgeladen von

Copyright:

Verfügbare Formate

Cisco MDS 9000 Family

SAN Volume Controller

Customer Order Number: DOC-7816119=

Cisco MDS 9000 Family SAN Volume Conroller Configuration Guide

Date Description of Change Where Changed