Student Guide - CFP 270 - Brocade 8 GB/s Introduction To Administration and Theory

CFP270:
Brocade 8 Gbit/sec
Introduction to
Administration and Theoryy
Student Guide
Revision 1008
CFP270
Corporate Headquarters - San Jose, CA USA

T: (408) 333-8000
info@brocade com
info@brocade.com
European Headquarters - Geneva, Switzerland
T: +41 22 799 56 40
emea-info@brocade.com
Asia Pacific Headquarters - Singapore
T: +65-6538-4700
apac info@brocade.com
apac-info@brocade.com
2008 Brocade Communications Systems, Inc. All Rights Reserved.

Brocade, the Brocade B-weave logo, Fabric OS, File Lifecycle Manager, MyView, Secure Fabric OS,
SilkWorm, and StorageX are registered trademarks and the Brocade B-wing symbol and Tapestry are
trademarks of Brocade Communications Systems, Inc., in the United States and/or in other countries.
FICON is a registered trademark of IBM Corporation in the U.S. and other countries. All other brands,
products, or service names are or may be trademarks or service marks of, and are used to identify,
products or services of their respective owners.
Notice: This document is for informational purposes only and does not set forth any warranty,
expressed or implied, concerning any equipment, equipment feature, or service offered or to be offered
by Brocade. Brocade reserves the right to make changes to this document at any time, without notice,
and assumes no responsibility for its use. This informational document describes features that may not
b currently
be
tl available.
il bl C
Contact
t taB
Brocade
d sales
l office
ffi for
f information
i f
ti on feature
f t
and
d product
d t availability.
il bilit
Export of technical data contained in this document may require an export license from the United
States government.
Revision: October, 2008
Table of Contents
Module 1 Course Introduction
CFP270 Course Overview ......................................................................................................................... 2
Brocade Certification Program ...................................................................................................................... 3
What is a Brocade Certification Worth? ........................................................................................................ 4
CFP270 Course Objectives........................................................................................................................... 5
CFP270 Course Objectives (cont.) ............................................................................................................... 6
CFP270 Course Prerequisites ...................................................................................................................... 7
CFP270 Instructor-Led Course Agenda........................................................................................................ 8
Registering for the BCFP Exam .................................................................................................................... 9
The Fulfillment Process............................................................................................................................... 10
Training Facility and Training Policies ........................................................................................................ 11
Introductions ................................................................................................................................................ 12
Module 2 - Hardware Product Features
Objectives...................................................................................................................................................... 2
The Brocade Family of Products ................................................................................................................... 3
Brocade Switches ......................................................................................................................................... 4
Condor2 / Condor ASIC Comparison ........................................................................................................... 5
Virtual Channels (VCs)1................................................................................................................................ 6
Condor2 ASIC ............................................................................................................................................... 7
Condor2 ASIC (cont.) .................................................................................................................................... 8
GoldenEye2 ASIC Features ....................................................................................................................... 9
GoldenEye2 vs. GoldenEye ASIC Comparison .......................................................................................... 10
Brocade 300 vs. Brocade 200E .................................................................................................................. 11
Brocade 5100 vs. Brocade 5000 ................................................................................................................. 12
Brocade 5300 vs. Brocade 4900 ................................................................................................................. 13
Brocade Backbone and Director Family ..................................................................................................... 14
DCX Data Center Backbone ....................................................................................................................... 15
DCX Port Side .......................................................................................................................................... 16
DCX Non-Port Side .................................................................................................................................. 17
CP8 Blade / CR8 (Core) Blade ................................................................................................................... 18
CP8 Blade Overview ................................................................................................................................ 19
CR8 (Core) Blade Overview..................................................................................................................... 20
ICLs Double the Core.................................................................................................................................. 21
ICL Details ................................................................................................................................................... 22
ICL Connections .......................................................................................................................................... 23
ICL Cabling ................................................................................................................................................. 24
Other ICL Cabling Supported Configurations ............................................................................................. 25
ICL Inter Chassis Link .............................................................................................................................. 26
DCX Dual Core Fabric ................................................................................................................................ 27
ICL Ports ..................................................................................................................................................... 28
Brocade 48000 Director .............................................................................................................................. 29
Director CP4 Blades.................................................................................................................................... 30
CP Redundancy .......................................................................................................................................... 31
DCX vs. 48000 ..................................................................................................................................... 32
Chassis: Dimensions / Weight .................................................................................................................... 33
FC8-16 Blade (DCX and 48000) ................................................................................................................. 34
FC8-32 Blade (DCX and 48000) ................................................................................................................. 35
FC8-48 Blade (DCX and 48000) ................................................................................................................. 36
FC4-16 Blade (48000 Only) ........................................................................................................................ 37
FC4-32 Blade (48000 Only) ........................................................................................................................ 38
FC4-48 Blade (48000 Only) ........................................................................................................................ 39
FC10-6 Blade (DCX and 48000) ................................................................................................................. 40
FR4-18i Blade (DCX and 48000) ................................................................................................................ 41

FC4-16IP Blade (48000 Only) ..................................................................................................................... 42
Director Blade Support Matrix ..................................................................................................................... 43
FC Port Blade Frame Flow.......................................................................................................................... 44
DCX Architecture 16-Port Blade .............................................................................................................. 45
Mi10K Director ............................................................................................................................................ 48
The Mi10K 4Gbps Blade (LMQ) .................................................................................................................. 49
Line Module Cards (LIMs) ........................................................................................................................... 50
Optical Paddles ........................................................................................................................................... 51
M6140 Director ............................................................................................................................................ 52
M6140 Control Processor Card (CTP) ........................................................................................................ 53
Brocade 7500 FCR/FCIP ........................................................................................................................... 54
Brocade 7500E FCR/FCIP .......................................................................................................................... 55
7500 and 7500E Feature Matrix ................................................................................................................. 56
Brocade USD-X ........................................................................................................................................... 57
SFP and XFP Transceivers......................................................................................................................... 58
SFP Types................................................................................................................................................... 59
Optical Transceiver Capability .................................................................................................................... 60
Summary ..................................................................................................................................................... 61
Module 3 Fibre Channel Theory
Objectives...................................................................................................................................................... 2
Fibre Channel Networking Model .................................................................................................................. 3
Fibre Channel Classes of Service ................................................................................................................. 4
Fibre Channel Frame Format ........................................................................................................................ 5
Node WWN Name Format NWWN ............................................................................................................ 6
Port WWN Name Format PWWN .............................................................................................................. 7
NWWN and PWWN Example ....................................................................................................................... 8
Port Types ..................................................................................................................................................... 9
Fabric Initialization Process ........................................................................................................................ 10
Node Types and Port Types ....................................................................................................................... 11
Fabric Generic Services .............................................................................................................................. 12
Well-Known Addresses ............................................................................................................................... 13
Fibre Channel Network Addressing ............................................................................................................ 14
Fabric Login at FFFFFE .............................................................................................................................. 15
Name Server at FFFFFC ............................................................................................................................ 16
FFFFFC Port and Node Attributes .............................................................................................................. 17
Brocade B-Series Name Server Commands .............................................................................................. 18
nsshow [-rt] .................................................................................................................................................. 19
nscamshow [-t] ............................................................................................................................................ 20
nsallshow..................................................................................................................................................... 21
Fabric Controller at FFFFFD ....................................................................................................................... 22
Brocade RSCN Delivery.............................................................................................................................. 23
Device Communication Example ................................................................................................................ 24
Fabric Initialization Process ....................................................................................................................... 26
Buffer-to-Buffer (BB) Credits ....................................................................................................................... 29
Port Indexes - FCx-48 Blades ..................................................................................................................... 30
Port Indexes - Mixed Blades ...................................................................................................................... 31
Shared Area Numbers - FCx-48 Blades ..................................................................................................... 33
Shared Area Numbers - switchshow........................................................................................................... 35
N_Port ID Virtualization (NPIV) ................................................................................................................... 36
NPIV Commands ........................................................................................................................................ 38
Summary ..................................................................................................................................................... 39
ii
Module 4 Installation and Configuration

Objectives...................................................................................................................................................... 2
Environmental Concerns ............................................................................................................................... 3
Brocade Management Interfaces and Tools ................................................................................................. 4
Command Line Interface Shortcuts .............................................................................................................. 5
Initial Configuration ....................................................................................................................................... 6
Log in Through the Serial Port ...................................................................................................................... 7
Set the IP Address ........................................................................................................................................ 8
Log in Through the Ethernet Interface .......................................................................................................... 9
Set Command Line Session Timeout ......................................................................................................... 10
Set the Fabric-Wide Clock .......................................................................................................................... 11
Set Switch Time Zone ................................................................................................................................. 12
Set Login Banner ........................................................................................................................................ 13
Activate Licensed Features ......................................................................................................................... 14
Set the Switch Name................................................................................................................................... 15
Set syslog Server ........................................................................................................................................ 16
Initial Security Configuration ....................................................................................................................... 17
Disable Telnet ............................................................................................................................................. 18
Set Password Rules .................................................................................................................................... 19
Set Password Rules (cont.)......................................................................................................................... 20
Set Password Rules (cont.) Change Default Passwords............................................................................ 21
Set Password Rules (cont.)......................................................................................................................... 22
Set Role Based Access Control (RBAC) .................................................................................................... 23
Set LDAP to RBAC Roles ........................................................................................................................... 29
RADIUS Authentication ............................................................................................................................... 32
Initial Interoperability Configuration ............................................................................................................. 38
Fabric Interoperability Overview .............................................................................................................. 39
Verifying Switch Status ............................................................................................................................... 45
Switch Status Commands ........................................................................................................................... 46
switchstatuspolicyset................................................................................................................................... 48
Port Status................................................................................................................................................... 49
Port Speeds ................................................................................................................................................ 50
Port Settings & Port Setting Commands ..................................................................................................... 51
Verifying Switch Operation .......................................................................................................................... 53
Port Status LEDs ......................................................................................................................................... 54
Fabric Parameters....................................................................................................................................... 55
Configuration Parameters ........................................................................................................................... 56
Consistent Fabric Parameters .................................................................................................................... 57
Booting a Switch ......................................................................................................................................... 58
Summary ..................................................................................................................................................... 59
Appendix - Installation and Configuration ................................................................................................... 61
Initial Interoperability Configuration ............................................................................................................. 62
iii
Module 5 Zoning
Objectives...................................................................................................................................................... 2
Zoning Overview ........................................................................................................................................ 3
Process to Implement Zoning ....................................................................................................................... 4
Hierarchy of Objects ...................................................................................................................................... 5
Zone Management ........................................................................................................................................ 6
Zone Aliases ................................................................................................................................................. 7
Zoning Example ............................................................................................................................................ 8
Enabling Zoning .......................................................................................................................................... 12
Disabling Zoning ......................................................................................................................................... 14
Clearing Zoning ........................................................................................................................................... 15
Saving Zoning ............................................................................................................................................. 16
Maximum Zoning Database Size ................................................................................................................ 17
Zone Object Commands ............................................................................................................................. 19
Zoning Display Commands nsaliasshow .................................................................................................... 20
Zoning Display Commands (cont.) nodefind ............................................................................................... 21
Zoning Display Commands (cont.) nszonemember .................................................................................... 22
Zoning Display Commands (cont.) nszonemember -u ............................................................................... 23
Additional Zone Management Commands.................................................................................................. 24
Default Zoning ............................................................................................................................................. 25
Default Zoning defzone .............................................................................................................................. 26
Web Tools Zoning Administration ............................................................................................................ 30
Zoning Enforcement .................................................................................................................................... 31
Zoning Enforcement Command portzoneshow........................................................................................... 35
Implementation Considerations .................................................................................................................. 36
Zoning Best Practices ................................................................................................................................. 37
Adding a New Switch to a Zoned Fabric ..................................................................................................... 39
Ensure New Switch Has No Zoning ............................................................................................................ 40
Connect New Switch to Existing Fabric ...................................................................................................... 41
Merging Two Zoned Fabrics: Segmentation ............................................................................................... 42
Other Merge Fabric Considerations ........................................................................................................... 43
Fabric Manager Fabric Merge Check ......................................................................................................... 44
Fabric Selection .......................................................................................................................................... 45
Merging Check Results Successful ......................................................................................................... 46
Other Zoning Tools .................................................................................................................................... 47
SAN Health Sample Zoning Spreadsheet................................................................................................ 48
Summary .................................................................................................................................................... 49
Module 6 FSPF Routing

Objectives...................................................................................................................................................... 2
Routing Overview ....................................................................................................................................... 3
Fabric Terminology ....................................................................................................................................... 4
Principal Switch Path .................................................................................................................................... 6
Principal Switch Commands ......................................................................................................................... 7
Routing Terminology ..................................................................................................................................... 8
Over-Subscribing ISLs .................................................................................................................................. 9
Virtual Channels .......................................................................................................................................... 10
Virtual Channels 1/2/4 Gbit/s ASICs ........................................................................................................ 11
Virtual Channels 8 Gbit/sec ASICs .......................................................................................................... 12
Fabric Shortest Path First (FSPF) ............................................................................................................... 13
FSPF Link Cost ........................................................................................................................................... 14
FSPF Dynamic Load Sharing (DLS) ........................................................................................................... 15
FSPF In-Order Delivery............................................................................................................................... 16
Routing Policies ......................................................................................................................................... 17
Exchange-based Routing ............................................................................................................................ 18
iv
Exchange-based Routing and DLS............................................................................................................. 19

Exchange-based Routing and DLS (cont.) ................................................................................................. 20
Port-based Routing ..................................................................................................................................... 21
Port-based Routing and DLS ...................................................................................................................... 22
Routing Policy Selection ............................................................................................................................. 23
Display Routing Information Overview ..................................................................................................... 24
Display Routing Information fcping .......................................................................................................... 25
fcping Example ............................................................................................................................................ 26
Display Routing Information pathinfo ....................................................................................................... 27
Display Routing Information pathinfo (cont.) ............................................................................................ 28
Display Routing Information topologyshow Exchange-based Routing .................................................... 29
Display Routing Information topologyshow Port-based Routing.............................................................. 30
Interop Mode Considerations Routing ..................................................................................................... 31
What is Brocade B-Series ISL Trunking? ................................................................................................... 32
Trunking Requirements ............................................................................................................................. 33
Condor 4 Gbit/sec Trunking Overview ..................................................................................................... 34
GoldenEye 4 Gbit/sec Trunking Overview ............................................................................................... 35
8 Gbit/sec Trunking Overview .................................................................................................................. 36
Port-based Routing without Trunking: Unpredictable Performance........................................................... 37
Trunking Frame Allocation .......................................................................................................................... 38
Port-based Routing over Trunks ................................................................................................................ 39
Exchange-based Routing over Trunks........................................................................................................ 40
One Port Group with Multiple ISL Trunks ................................................................................................... 41
ASIC Trunk Pseudo-Master ....................................................................................................................... 42
The Deskew Counter .................................................................................................................................. 43
Trunking Related Commands Overview .................................................................................................. 44
trunkshow .................................................................................................................................................... 45
switchshow .................................................................................................................................................. 46
islshow......................................................................................................................................................... 47
portcfgtrunkport/portcfgshow....................................................................................................................... 48
trunkdebug .................................................................................................................................................. 49
Brocade M-series Open Trunking ............................................................................................................... 50
Summary ..................................................................................................................................................... 52
Module 7 Fibre Channel Long Distance
Objectives...................................................................................................................................................... 2
Long Distance Fabrics Overview ............................................................................................................... 3
Long Distance Cabling ................................................................................................................................. 4
Long Distance Connectivity Options ............................................................................................................. 6
Long Distance Connectivity Options Native FC over Dark Fiber .............................................................. 7
Long Distance Connectivity Options WDM ................................................................................................ 8
Long Distance Connectivity Options Time Division Multiplexing (TDM) .................................................... 9
Long Distance Connectivity Options SONET/SDH .................................................................................. 10
Long Distance Connectivity Options Extended Distance Solutions ......................................................... 11
FOS Extended Fabrics Feature .................................................................................................................. 12
Extended Fabric Configuration ................................................................................................................... 13
portshow ...................................................................................................................................................... 15
Distance Availability and Performance ....................................................................................................... 16
Extended Distance Availability Trunking ................................................................................................. 17
Extended Distance Availability Protection Switching .............................................................................. 18
Extended Distance Performance Buffer-to-Buffer Credit Allocation ........................................................ 19
E_Port Credit Recovery .............................................................................................................................. 20
portbuffershow ............................................................................................................................................ 26
Summary ..................................................................................................................................................... 27
Appendix - Fibre Channel Long Distance ................................................................................................... 29
Module 8 Administration and Maintenance

Objectives...................................................................................................................................................... 2
Centralized Management EFCM & Fabric Manager ..................................................................................... 3
Which Application to Use? ............................................................................................................................ 4
Fabric Manager Overview .......................................................................................................................... 5
EFCM Offerings ............................................................................................................................................ 6
Managing Individual Switches Web Tools and EFCM Basic ........................................................................ 7
EFCM Basic Edition ...................................................................................................................................... 8
Web Tools .................................................................................................................................................... 9
ESCM (Enterprise Server Connectivity Manager) ...................................................................................... 10
SNMP Overview ....................................................................................................................................... 12
SNMP Components .................................................................................................................................... 13
SAN Monitoring with SNMP ........................................................................................................................ 14
SNMP Version Authentication ..................................................................................................................... 15
Brocade SNMP Fabric OS Commands ....................................................................................................... 16
Brocade Single-CP Switch Firmware Download Internal Process.............................................................. 17
CP Differences for Directors and Backbones ............................................................................................. 25
Non-Disruptive CP Card Failover ................................................................................................................ 26
High Availability Commands .................................................................................................................... 27
Other HA Commands ................................................................................................................................. 28
Firmware Maintenance CP Cards ............................................................................................................ 29
Firmware Download Process ...................................................................................................................... 30
48000 Firmware Upgrade .......................................................................................................................... 31
DCX Firmware Upgrade .............................................................................................................................. 45
Firmware Maintenance Single-CP Mode ................................................................................................. 55
Firmware Maintenance Other Commands ............................................................................................... 56
Displaying Control Processor Status firmwareshow .................................................................................. 57
Downloading Firmware for M-Series ........................................................................................................... 58
Archiving Switch Configuration configupload .............................................................................................. 59
Archiving Switch Configuration configupload Example ............................................................................... 60
Restoring Switch Configuration configdownload ........................................................................................ 61
File Copy Protocols ..................................................................................................................................... 62
Summary ..................................................................................................................................................... 63
CFP270 Brocade 8 Gbit/sec Introduction to Administration and Theory ................................................ 64
Appendix - Administration and Maintenance More information on firmware download ............................ 65
Appendix A Error Handling....................................................................................................................... 66
Appendix B DCX Firmware Upgrade ....................................................................................................... 69
firmwareshow -v .......................................................................................................................................... 73
CFP270 Brocade 8 Gbit/sec Introduction to Administration and Theory ................................................ 74
vi
CFP270 Brocade 8 Gbit/sec

Introduction to Administration and Theory
Revision 1008
Course Introduction
1-1

Revision 1008
Course Introduction
1-2

Revision 1008
Course Introduction
1-3

Revision 1008
Course Introduction
1-4

Revision 1008
Course Introduction
1-5

Revision 1008
Course Introduction
1-6

Revision 1008
Course Introduction
1-7

Revision 1008
Course Introduction
1-8

Revision 1008
Course Introduction
1-9

Revision 1008
Course Introduction
1 - 10

Revision 1008
Course Introduction
1 - 11

Revision 1008
Course Introduction
1 - 12

Revision 1008
Course Introduction
1 - 13
This page is intentionally blank.
Revision 1008
1 - 14

Revision 1008
Hardware Product Features
2-1

Revision 1008
2-2

Revision 1008
2-3

Revision 1008
2-4

Footnote 1: On the 16 and 32-port blades, two of these port groups (2 x 8 = 16 total
per ASIC) are used for external ports and the other three port groups are used for
internal ports. On the 48 port line card three, of these port groups (3 x 8 = 24 total
per ASIC) are used for external ports and the other two port groups are used for
internal ports.
Footnote 2: Each Condor2 ASIC has 1420 user BB credits: Each front-end port is
allocated 8 credits (8 credits x 16 ports = 128 credits) which leaves 1292 credits
available per Condor2 ASIC. Note: The 48-port blade has 24 front-end ports per
ASIC (8 credits x 24 ports = 192 credits) which leaves 1228 credits available per
Condor2 ASIC. Use the p
portbuffershow command to see available credits.
Footnote 3: QoS: 16 Virtual Channels: Details on the next slide.
Revision 1008
2-5

Footnote 1: Low, Medium and High priority traffic refer to the QoS (Quality of
Service) feature which will be covered in a later module.
Revision 1008
2-6

Footnote 1: FL_Ports are not supported on the 48-port blade.

Footnote 2: In Fabric OS v6.0.0, Mirror ports are not supported on the Condor2
ASIC.
Footnote 3: In Fabric OS v6.0.0 8, Gbit/sec speed is only supported up to 10Km.
Footnote 4: Each Condor2 ASIC has 1420 user credits: Each front-end port is
available per Condor2 ASIC. Note: The 48-port blade has 24 front-end ports per
ASIC (8 credits x 24 ports = 192 credits) which leaves 1228 credits available per
Condor2 ASIC. Use the portbuffershow command to see available credits.
Revision 1008
2-7

Footnote 1: FICON support is on a platform / blade basis.
Revision 1008
2-8

Revision 1008
2-9

Footnote 1: The GoldenEye2 ASIC is architected for 32 ports but only 24 ports are
currently used (Brocade 300).
Footnote 2: The GoldenEye ASIC is architected for 24 ports but only 16 ports are
currently
tl used
d (B
(Brocade
d 200E)
200E).
Footnote 3: Each GoldenEye2 ASIC has 676 user BB credits: Each front-end port is
available per GoldenEye2 ASIC. Use the portbuffershow command to see
available credits.
Revision 1008
2 - 10

Both switches are capable of going into Access Gateway mode (covered in
CFP271).
Revision 1008
2 - 11

Footnote 1: Support for FCR on the Brocade 5100 requires an Integrated Routing
license and full Ports on Demand licenses.
Footnote 2: The power supply/fan FRUs in the Brocade 5100 are not compatible
with
ith th
those iin th
the B
Brocade
d 5000
5000.
Revision 1008
2 - 12

Footnote 1: Support for FCR on the Brocade 5300 requires an Integrated Routing
license.
Footnote 2: The fans in the Brocade 5300 are larger than the fans in the Brocade
4900.
4900
Revision 1008
2 - 13

Revision 1008
2 - 14

Footnote 1: Each power supply provides 2000 Watts if connected at 220 VAC or
1000 Watts if connected at 110 VAC.
Footnote 2: Fabric OS 6.0+ on blades that use the Condor2 ASIC (more information
on this
thi llater
t iin thi
this module)
d l ) will
ill check
h k tto make
k sure th
the SFP iis B
Brocade-branded.
d b d d If
not Brocade-branded, the port will not come up. 4 Gbit/sec blades that use the
Condor ASIC do not have this requirement.
Revision 1008
2 - 15

Footnote 1:
Slots 1-4 show 48-port blades
Slot 5 is Core blade 0
Slot 6 is CP blade 0
Slot 7 is CP blade 1
Slot 8 is Core blade 1
Slots 9 and 10 show 32-port blades
Slots 11 and 12 show 16-port blades
Revision 1008
2 - 16

If power supplies are using 220Vac input, only 2 supplies are required for N+1
protection. If power supplies are using 110Vac input, 3 power supplies would be
required for N+1 protection.
2 ffunctional
ti
l bl
blowers are required
i d tto cooll th
the DCX
DCX.
Revision 1008
2 - 17

Revision 1008
2 - 18

Footnote 1: The Service IP port is for future use.

USB port (must be a Brocade-Branded USB drive) can be used for:
firmwaredownload
configupload / configdownload
supportsave
LEDs:
Revision 1008
LED
Color
Location
Description
Power
Green
Front Panel
On = CP Power is operational
Off = CP Power if failed
Attention
Amber
Front Panel
On = If on for > 5 seconds, board is defective/faulted
Active
Blue
Front Panel
On = This CP is the Active CP

Off = This CP is either booting, negotiating to be Active or the
Standby CP
Ethernet
Link
Green
Front Panel
RJ45 Top
On = Ethernet Port MAC Link has been established at

100/1000Mbps
Off = no Link or 10Mbps
Ethernet
Activity
Green
Front Panel
RJ45 Bottom
On blinking = TX or RX frames activity present

Off = No TX or RX activity
USB Port
Green
External
On = USB port is enabled

Off = USB port is disabled
2 - 19

There are two LEDs on the top of the CR8 blade with the following behavior:
LED
Color
Description
p
Power
Green
On = CP Power is operational
Off = CP Power has failed
Attention
Amber
On = If on for > 5 seconds,

board is defective/faulted
Off = Normal Operation
The two ICL connectors have two LEDs each with the following behavior:
Revision 1008
2 - 20

ICL kit comes with the following:

Four ICL cables
Two ICL licenses (one per DCX)
Revision 1008
2 - 21

Revision 1008
2 - 22

Footnote 1: Does not require a Trunking license on the switch to trunk the ICL ports.
Revision 1008
2 - 23

Footnote 1: Cables and connectors are color coded to avoid misconnection.
Revision 1008
2 - 24

Footnote 1: All ICL cables are 2 meters in length.

These work just like other ISL connections. If pulling a cable would result in the
failover to other paths (ICL connections or external ISL connections between the
t
two
switches).
it h )
The only differences between ICLs and ISL are that ICLs do not use user ports, the
port speed is hard set, and Trunking is automatically enabled.
Revision 1008
2 - 25

Revision 1008
2 - 26

Footnote 1: For any-to-any topologies where any port connected to one chassis
may be connected to any port on the other chassis, ICLs are oversubscribed 4:1 at
8 Gbit/sec between the chassis (Core Blades in Chassis 1 to Core Blades in
Chassis 2).
2) If more inter-chassis bandwidth is required,
required nothing precludes the use of
front ports as ISLs in addition to ICLs. When each chassis is fully configured to 384
ports and any port may connect to any other port, then the two chassis are 6:1
oversubscribed (1.5x oversubscribed at each chassis multiplied by 4x
oversubscribed across ICLs) at 8 Gbit/sec.
ISL oversubscription is represented as a ratio of input ports to output ports.
Revision 1008
2 - 27

Here is what happens when you try to set the port speed on an ICL port:
DCX10:admin> portcfgspeed 5/1 4
Operation failed - Configuration change is not allowed on ICL
ports.
Revision 1008
2 - 28

The Brocade 48000 director extends the high-availability and performance features
introduced in prior Brocade director products. The CP and port cards are based on
the Condor ASIC, which provides double the port speed (up to 4 Gbit/sec) and
double the port count (up to 48 ports per port card
card, up to 384 ports per chassis)
chassis).
The Brocade 48000 continues the modular, redundant architecture with multiple
field-replaceable CP cards, port cards, power supplies, blower assemblies, and
WWN cards. All ports within the chassis are managed as a single domain. The
routing between the Brocade 48000 port cards is based on the same non-blocking
architecture used in prior Brocade director products.
A fully-loaded Brocade 48000 consumes less power than prior fully-loaded Brocade
director products.
The Brocade 48000 protects your existing investment by using the same power
supply, blower assembly, and WWN card as prior Brocade director products.
Revision 1008
2 - 29

Brocade director CP cards are installed in slot 5 (CP0) and slot 6 (CP1).
The Brocade 48000 CP card is also called the CP4.
Each CP card includes a single 1 GB Compact Flash card, that stores the Fabric
OS firmware and is managed as two equal-sized partitions. There is also 16 MBytes
of kernel flash memory (also managed as two equal-sized banks) for user
configurations.
Each CP card is powered by a CPU (800 MHz IBM Power PC 440GX). The kernel
is based on MontaVista, a Linux-based, industry-standard real-time OS that allows
adding new features. Fabric OS runs as a layered application on the Linux kernel.
Each CP card provides a dial-in modem port (identified as RS-232) for remote
management (requires an external modem); a serial port (10101) for local
management; and a 10/100 Mbit/sec Ethernet port (with an unique IP address) for
network management.
The Brocade 48000 CP card consumes 100 watts of power.
Revision 1008
2 - 30

The CP and Core sections draw power from separate power etches, and each
section can fail independent of the other section just as with each half of the
Brocade director WWN card.
The Brocade
Th
B
d 48000 CP card
d uses Condor
C d ASICs
ASIC in
i the
th Core
C
Section,
S ti
matching
t hi th
the
Condor ASIC deployed on the FC4-16 and FC4-32 port cards.
Revision 1008
2 - 31

Footnote 1: In the DCX, each slot can handle 32 x 8 Gbits/sec in both directions and
still be fully subscribed; where as the 48000 can only handle up to 16 x 4 Gbit/sec in
both directions and still be fully subscribed. Therefore, the DCX has 4 times the
bandwidth per slot than the 48000.
48000
Footnote 2: 384 user ports plus 64 ICL (ISL) ports for a total of 448 ports.
Revision 1008
2 - 32

Footnote 1: Fully loaded: 8 x 48-port blades, 2 Core blades, 2 CP blades, 3 blowers,

4 power supplies, 2 WWN cards and door.
Revision 1008
2 - 33

Footnote 1: EX_Port support is with the DCX only

Footnote 2: Blade LEDs:
Per-Blade LEDs
Per-Port LED
Revision 1008
2 - 34

Footnote 1: EX_Port support is in the DCX only

Per-Blade LEDs
Per-Port LED
Revision 1008
2 - 35

Footnote 1: EX_Ports only supported in the DCX

Per-Blade LEDs
Per-Port LED
Revision 1008
2 - 36

The FC4-16 port card provides sixteen 1, 2, or 4 Gbit/sec Fibre Channel ports that
may be connected to servers, storage, or other switches as needed.
Port hardware is based on the same ASIC technology used in the Brocade 4100
switch.
it h
Port interfaces are compatible with SWL, LWL, and ELWL transceivers.
Ports may be F_Port, FL_Port, or E_Port and can be pre-configured, or can selfconfigure.
Ports are numbered on the card from 0 to 15, start at the bottom of the card (in the
image
g above, the lower left-hand corner of the blade)) and move upwards.
On an FC4-16 port card, ports within the port card communicate directly.
Communications to other port cards are made through the backplane.
The Brocade 48000 optical slider, along with the thumb screw, are enhanced so that
is it more difficult to insert and remove the card, as well as power on the card,
without the thumbscrew fully in place.
The FC4-16
FC4 16 port card consumes 40 watts of power
power.
Revision 1008
2 - 37

The FC4-32 port card provides thirty-two 1, 2, or 4 Gbit/sec Fibre Channel ports that
may be connected to servers, storage, or other switches as needed.
Port hardware is based on the same ASIC technology used in the Brocade 4100
switch.
it h
Port interfaces are compatible with SWL, LWL, and ELWL transceivers.
Ports may be F_Port, FL_Port, or E_Port and can be pre-configured, or can selfconfigure.
Ports are numbered as follows:
Left-hand
Left hand column: ports are numbered on the card from 0 to 15
15, start from the
bottom of the card (in the image above, the lower-left corner) and move
upwards.
Right-hand column: ports are numbered on the card from 16 to 31, start from
the bottom of the card and move upwards.
The FC4-32 port card consumes 50 watts of power.
Revision 1008
2 - 38

The FC4-48 port blade is similar in architecture to the FC4-32 port blade. It contains
two Condor ASICs which interface to a total of 48 ports that are capable of 1/2/4
Gbit/sec speeds. It has many of the same capabilities as the FC4-32 port blade.
The FC4
Th
FC4-48
48 portt bl
blade
d h
has a new ttype off ejector
j t and
d ejector
j t h
handles.
dl
Wh
When th
the
board is inserted into an empty blade slot of a Brocade 48000, the handles are
pushed outward to lock it into the slot. They are pushed inward to eject it from the
chassis.
Revision 1008
2 - 39

FC10-6 ports may not be connected to 10 Gbit/sec ports in the Brocade Mi10K or
other vendors products.
The 8b/10b encoding scheme converts 8 bits of user data into 10 bits of data
t
transmitted
itt d over th
the Fib
Fibre Ch
Channell lilink.
k
The 64b/66b encoding scheme converts 64 bits of user data into 66 bits of data
transmitted over the Fibre Channel link.
The difference in encoding schemes means that one 10 Gbit/sec link can carry
almost as much data as three 4 Gbit/sec links:
10 Gbit/sec link: Bandwidth = 10 Gbit/sec * ((64/66)) = 9.70 Gbit/sec
Three 4 Gbit/sec links: Bandwidth = 3 * 4 Gbit/sec * (8/10) = 9.60 Gbit/sec
The XFP (10 Gbit/sec Small Form Factor Pluggable) is a hot-swappable optical
transceiver used by several 10 Gbit/sec protocols, including Fibre Channel.
The Brocade FC10-6 can be installed in a Brocade 48000 director with the following
blades: FC4-16, FC4-32, FC4-48, FR4-18i, FC4-16IP, and FA4-18.
Footnote
F
t t 1:
1 The
Th power supply
l requirement
i
t is
i for
f high-availability
hi h
il bilit purposes, and
d iis
not enforced by Fabric OS. Your switch provider may require four power supplies;
check switch provider supportability requirements.
Revision 1008
2 - 40

The FR4-18i blade provides the following features:

16 Fibre Channel ports (based on the Condor ASIC) that support the Fibre
Channel Routing Services
2 GbE ports supporting the FCIP and Fibre Channel Routing Services with link
speeds up to 1 Gbit/sec:
Each GbE port can support up to 8 FCIP tunnels.
Each FCIP tunnel is represented and managed as a virtual Fibre Channel
E_Port (VE_Port).
Each FCIP tunnel can support data compression and traffic shaping
shaping, with a
committed bandwidth.
Fibre Channel Routing Services can be used over each FCIP tunnel
(VEX_Ports).
You can learn more about the Brocade FR4-18i Blade by viewing the AFS175 webbased training class from Brocade.
Revision 1008
2 - 41

The FC4-16IP blade provides a director-based iSCSI gateway from the Fibre
Channel SAN to iSCSI initiators.
Both of the intelligent blades (FR4-18i and FC4-16IP) require 4 power supplies for
Hi h A
High
Availability.
il bilit
Revision 1008
2 - 42

Revision 1008
2 - 43

Footnote 1: Through the switch means the frame comes in on one blade goes
through core blade, and out another blade on the switch. The Condor ASIC latency
is about 800ns per ASIC and 2.4s through the switch.
Revision 1008
2 - 44

Each Condor2 ASIC has 40 ports; on the 16-port blade, only 32 are used: 16
external and 16 Internal.
If the initiator and target are on the same ASIC, the frame would not go through the
core and
d llocall switching
it hi would
ld b
be used.
d
If the frame coming out of the core blade is going out the ICL cable, each Condor2
ASIC on the core has 8 x 8 Gbit/sec Trunk going to the other DCX.
If the frame coming out of the core blade is going to a 16-port blade, then each
Condor2 ASIC on each core blade would have 2 x 8 Gbit/sec Trunk going to the
Condor2 ASIC on the 16-port blade.
If the frame coming out of the core blade is going to a 32 or 48-port blade, then
each Condor2 ASIC on each core blade would have 2 x 8 Gbit/sec Trunk going to
both of the Condor2 ASICs on the 32 or 48-port blade.
Revision 1008
2 - 45

Each Condor2 ASIC has 40 ports; on the 32-port blade, only 32 are used: 16
external and 16 Internal.
core and
d llocall switching
it hi would
ld b
be used.
d
Revision 1008
2 - 46

Each Condor2 ASIC has 40 ports; on the 48-port blade all 40 ports are used: 24
external and 16 Internal. This is why the 48-port blade is 24 to 16 oversubscribed.
core and
d llocall switching
it hi would
ld b
be used.
d
Revision 1008
2 - 47

Revision 1008
2 - 48

LMQ stands for Line Module Quad Speed.
Revision 1008
2 - 49

Revision 1008
2 - 50

Revision 1008
2 - 51

Revision 1008
2 - 52

The director is delivered with two CTP cards. The active CTP card initializes and configures
the director after power on and contains the microprocessor and associated logic that
coordinate director operation. The CTP card provides an initial machine load (IML) button
p
When the IML button is p
pressed, held for
and a RESET button ((recessed)) on the faceplate.
three seconds, and released, the director performs an IML that reloads the firmware from
FLASH memory. This operation is not disruptive to Fibre Channel traffic. When the RESET
button is pressed and held for three seconds, the director performs a reset.
A reset is disruptive and resets the:
Microprocessor and functional logic for the CTP card and reloads the firmware from
FLASH memory.
Ethernet LAN interface, causing the connection to the management server to drop
momentarily until the connection automatically recovers.
Ports, causing all Fibre Channel connections to drop momentarily until the connections
automatically recover. This causes attached devices to log out and log back in, therefore
data frames lost during director reset must be retransmitted.
A reset should only be performed if a CTP card failure is indicated. As a precaution, the
RESET button is flush mounted to protect against inadvertent activation.
Each CTP card also provides a 10/100Mbps RJ-45 twisted pair connector on the faceplate
that attaches to an Ethernet local area network (LAN) to communicate with the
management server or a Simple Network Management Protocol (SNMP) management
station. In addition, a CTP card provides nonvolatile memory for storing firmware, director
configuration information, persistent operating parameters, and memory dump files.
Director firmware is upgraded concurrently (without disrupting operation). The backup CTP
card takes over operation if the active card fails
fails. Failover from a faulty card to the backup
card is transparent to attached devices. Each card faceplate contains a green LED that
illuminates if the card is operational and active, and an amber LED that illuminates if the
card fails. Both LEDs are extinguished on an operational backup card. The amber LED
blinks if FRU beaconing is enabled.
Revision 1008
2 - 53

The Brocade 7500 is designed for stacked deployments

A stand-alone solution for enterprise and mid-range
16 port Fibre Channel Switch/Router
2 port Fibre Channel over IP Distance Extension
16 Front Panel 1, 2, or 4 Gbit/sec FC ports
Fibre Channel Routing Services for all ports including load balancing and long
distance support
16 4 Gbit/sec Internal Fibre Channel Router
2 Front Panel 1 Gbit/sec APs Ethernet Ports
Compression Hardware Assist
IPSEC Hardware Assist
8 FCIP tunnels per Port
Each FCIP Tunnel is represented and managed as a Virtual FC E_Port
Support for Fibre Channel Routing Services over FCIP link
Fabric OS v5.1.0+ required to operate blade
Fixed configuration
Non-disruptive firmware download (except GbE IP-ports where there can
be up to a 30sec outage)
1U by 24
24 deep
Redundant fans, redundant power supplies
Revision 1008
2 - 54

You can order and install a software upgrade license. After the Brocade 7500E
firmware detects the presence of the upgrade license, the Brocade 7500E will
enable functions equivalent to the Brocade 7500 as well as the high-performance
extension license,
license activation of all 16 Fibre Channel ports
ports, and full line-rate
performance.
The software upgrade license is a bundled package that includes:
Hardware-based encryption
Read and Write Tape pipelining
Fibre Channel-based extension with Fast Write
FICON (disk and tape) over metro distances
Local switching
Fibre Channel Routing between fabrics
Call Home feature
Full line rate speeds across IP WAN ports
Activation of 14 additional Fibre Channel ports
Seven additional tunnels per Gigabit Ethernet port
Revision 1008
2 - 55

Revision 1008
2 - 56

Revision 1008
2 - 57

Revision 1008
2 - 58

SWL Fiber Optic SFP Module - Based on short-wavelength 850 nm lasers supporting 2 and 4
Gbit/sec link speeds. This SFP module supports 50/62.5, multimode fiber optic cables, 50 cable is
recommended.
Note: Short-wavelength SPF modules may be labeled MM (for multimode), SW, or SWL (for shortwavelength
l
th llaser).
)
LWL Fiber Optic SFP Module Based on long-wavelength 1310nm lasers supporting 2 and 4
Gbit/sec link speeds. This SFP module supports 9 single-mode fiber optic cables.
ELWL Fiber Optic SFP Module Based on long-wavelength 1550nm lasers supporting 2 Gbit/sec
link speeds. This SFP module supports 9 single-mode fiber optic cables and distances over 50 km.
WDM Fiber Optic SFP Module Based on various wavelength lasers ranging from 1470nm to
1610nm supporting 2 Gbit/sec link speeds. Requires external xWDM mux/demux module to transmit
multiple wavelengths simultaneously over a single fiber optic cable.
cable WDM devices support longdistance connectivity to 100 km and beyond.
TST_ST01_B20_1:admin> sfpshow
Area
0: id (sw) Vendor: FINISAR CORP.
Serial No: U85066V
Area
1: id (sw) Vendor: FINISAR CORP.
Serial No: H11XSF9
. . . [truncated]
TST_ST01_B20_1:admin> sfpshow 0
Identifier:
SFP
Connector:
LC
Transceiver: 150c402001000000 100,200,400_MB/s M5,M6 sw Inter_dist

Encoding:
8B10B
Baud Rate:
42
(units 100 megabaud)
Length 9u:
(units km)
Length 9u:
(units 100 meters)
Length 50u:
15
(units 10 meters)
Length 62.5u:7
(units 10 meters)
Length Cu:
(units 1 meter)
Vendor Name: FINISAR CORP.

Vendor OUI:
00:90:65
Vendor PN:
FTRJ8524P2BNV
Vendor Rev:
Wavelength:
850
Options:
0032 Loss_of_Sig,Tx_Disable
BR Max:
BR Min:
Serial No:
U85066V
Date Code:
050729
(units nm)
Temperature: 28 Centigrade
Current:
6.678 mAmps
Voltage:
3319.4 mVolts
RX Power:
408.0 uWatts
TX Power:
311.9 uWatts
TST_ST01_B20_1:admin>
Revision 1008
2 - 59

If a non-branded SFP is used on blades running the Condor2 ASIC, the port will not
come online.
In Fabric OS v6.0, 8 Gbit/sec LW is not supported at 8 Gbit/sec. For long distance,
mustt use 4 Gbit/sec.
Gbit/
Revision 1008
2 - 60

Revision 1008
2 - 61

Revision 1008
2 - 62

The following table lists the switch type assigned to each switch and is displayed in
the switchshow command.
Revision 1008
Brocade Switch
Switch Type
ASIC
1000
Stitch/Flanel
2800
LOOM
2400
LOOM
20x0
LOOM
22x0
LOOM
3800
BLOOM
12000
10
BLOOM
3900
12
BLOOM
3200
16
BLOOM
24000
21
BLOOMII
3850
26
BLOOMII
3250
27
BLOOMII
4100
32
Condor
200E
34
GoldenEye
48000
42
Condor
4900
44
Condor
7500
46
Condor
7600
55
Condor
5000
58
Condor
DCX
62
Condor2
5300
64
GoldenEye2
5100
66
Condor2
300
71
GoldenEye2
2 - 63

Revision 1008
2 - 64

Revision 1008
Fibre Channel Theory
3-1

Revision 1008
3-2

The FC-0 and FC-1 layers specify physical and data link functions needed to
physically send data from one port to another.
FC-0 specifications include information about feeds and speeds.
FC-1 layer contains specifications for 1, 2, 4 and 8 Gbit/sec 8b/10b encoding,
ordered set and link control communication functions. 10 Gbit/sec communication
uses 64b/66b encoding.
FC-2 specifies content and structure of information along with how to control and
manage information delivery. This layer contains basic rules needed for sending
data across the network. This includes: (1) how to divide the data into frames, (2)
h
how
much
hd
data
t should
h ld b
be sentt att one ti
time b
before
f
sending
di more (fl
(flow control),
t l) and
d
(3) where the frame should go. It also includes Classes of Services, which define
different implementations that can be selected depending on the application.
FC-3 defines advanced features such as striping (to transmit one data unit across
multiple links) and multicast (to transmit a single transmission to multiple
destinations) and hunt group (mapping multiple ports to a single node). While the
FC 2 level concerns itself with the definition of functions with a single port
FC-2
port, the FC
FC-3
3
level deals with functions that span multiple ports.
FC-4 provides mapping of Fibre Channel capabilities to pre-existing protocols, such
as IP, SCSI, or ATM, etc.
Revision 1008
3-3

Different types of information have different delivery requirements. In order to

accommodate the different delivery requirements, Fibre Channel associates a set of
delivery characteristics into what is called a Class of Service. The characteristics
relate to the type of connection between the ports
ports, confirmation of delivery
delivery, flow
control mechanisms and how errors are handled.
Class-1 is a connection-oriented circuit that dedicates 100% of the bandwidth
between the sending and receiving ports. It also provides for a confirmation of
delivery (ACK).
Class-2 is a connectionless class with an acknowledgement (confirmation of
delivery) No bandwidth is allocated or guaranteed.
delivery).
guaranteed IP uses this class
class. Uses both
Buffer-to-Buffer (BB) credits and End-to-End (EE) credits for flow control.
Class-3 is a connectionless class without an acknowledgement (confirmation of
delivery). No bandwidth is allocated or guaranteed. FCP uses this class. Uses
Buffer-to-Buffer (BB) credits for flow control, does not use End-to-End (EE)
credits.
Class
Class-4
4 is a connection
connection-oriented
oriented class that uses virtual circuits and confirmation
of delivery. Unlike Class-1 that reserves the entire bandwidth, Class-4 can
allocate a requested amount of bandwidth.
Class-6 is a variation of Class-1 that provides a one-to-many multicast service
with a confirmation of delivery.
Class-F is a connectionless class with acknowledgements (confirmation of
delivery) between two switches
delivery).
switches.
Note: Brocade supports Class-2, Class-3, and Class-F only.
Revision 1008
3-4

A frame has a header and may have a payload. The header contains control and addressing
information associated with the frame. The payload contains the information being transported by the
frame on behalf of the higher level service or FC-4 upper level protocol. The Fibre Channel standards
allow bytes from the payload to be used for optional headers. There are many different payload
f
formats,
t based
b
d on the
th protocol.
t
l The
Th TYPE field
fi ld (Word
(W d 2,
2 bits
bit 31
31- 24) specifies
ifi which
hi h fformatt to
t use.
The Routing Control INFO bits (bits 27-24) determine how to interpret the payload.
Field Definitions:
Routing Control (R_CTL): first 8 bits of the header. They define the type of frame and its
content or function. The first 4 bits (Bits 31-28) identify the frame type. The second four bits
(Bits 27-24) define the contents of the frame or identify the function of the frame (22 = ELS, 02
= CT command).
D
Destination_ID
ti ti
ID (D
(D_ID):
ID) 24
24-bit
bit P
Portt Id
Identifier
tifi (PID) off the
th recipient.
i i t It could
ld also
l be
b a wellll
known address such as the Name Server FFFFFC.
Class Specific Control Field (CS_CTL): control necessary for the different classes of service.
This field is always zero for classes 2 and 3 per the standards. Classes 1 and 4 use it.
Source_ID (S_ID): 24-bit Port identifier (PID) of the source. It could be a well-known address.
Type: identifies the protocol of the frame content for Data Frames (i.e FC_CT, FCP, IPFC).
) contains miscellaneous control information regarding
g
g the frame such
Frame Control ((F_CTL):
as who owns initiative, first frame of the exchange, last frame of the exchange, etc.
Sequence ID (SEQ_ID): used to identify and track all of the frames within a sequence between
a source and destination port pair.
Data Field Control (DF_CTL): indicates if any optional headers are present at the beginning of
the data field of the frame. Optional headers are used for information that may be required by
some applications or protocol mappings.
q
Count ((SEQ_CNT):
_
) used to indicate the sequential
q
order of frame transmission
Sequence
within a sequence or multiple consecutive sequences within the same exchange. This is a
counter that increments as sequence of frames is transmitted.
Originator_ID (OX_ID): Exchange ID assigned by the originator port.
Responder_ID (RX_ID): Exchange ID assigned by the responder to the exchange.
Data Field/Payload: The maximum size is 2112 bytes.
Revision 1008
3-5

Footnote1: OUI is the Organization Unique Identifier.

Based on the IEEE Standard format, a Brocade Node WWN is:
10:00:00:05:1e:xx:xx:xx. Where:
The first 2 bytes are always 10:00
The next 3 bytes are vendor-specific. Brocade has the following OUI codes
registered:
00:00:88
00:01:0F
00 05 1E
00:05:1E
00:05:33
00:14:c9
00:60:69
00:60:DF
08:00:88
The last 3 bytes are derived from the Brocade main board serial number.
The 3-byte company ID found in the 64-bit IEEE Standard format WWN can be
searched at: http://standards.ieee.org/regauth/oui/oui.txt
Revision 1008
3-6

Port World Wide Name:

2p:pp:00:05:1e:xx:xx:xx
The three nibbles (p:pp) are used by Brocade to show the switch port number.
The 0:09 in the example above indicates this is port 9 on the switch.
Revision 1008
3-7

The Node WWN (NWWN) is like an umbrella that the port WWNs sit under like
spokes. Different vendors do different things with the WWNs.
The Port WWN (PWWN) sits under the Node WWN. The first two bytes are
assigned
i
db
by a vendor.
d
Example: Seagate uses Node and Port WWNs in a different manner than Brocade.
Seagate uses a format 2 (2x:) for both the Node and Port WWNs. In the PWWN,
the port is identified in the second nibble of the first byte. The zoning module will
show how the NWWN and PWWN are used.
Revision 1008
3-8

For a swtich port that goes through port initialization, it will arrive at an ending status
of F_Port, FL_Port or E_Port.
Revision 1008
3-9

A Universal Port (U_Port) is the initial state of a port. (State 1)

Is something connected (sending a light/electrical signal) to the port? If yes, continue.
(Transition 1)
U_Port
U
Port starts mode detection process by transmitting at least 12 LIP(F7) Primitive Sequences
Sequences.
(Transition 2)
If at least 3 consecutive LIP Primitive Sequences are received, then the port enters
OPEN_INIT state and attempts FC-AL loop initialization. (State 2)
If LIP Primitive Sequences are not received, the U_Port attempts OLD_PORT initialization
by taking the link down then transmitting NOS primitives. If Link Initialization Protocol fails
after 1 retry or LIP received after 1 second, go to FC-AL initialization. (Transition 2)
When operating in the FL_Port mode, a U_Port will try the loop initialization procedure
three times. If these fail, the port will be marked as faulty. To ensure N_Port, reinitialize
the port and the switch port will cut the laser forcing a loss of signal state for at least 20
s. Then the switch port will bring back the laser and issue NOSs. (Transition 2)
U_Port will attempt the OLD_PORT initialization (Link Initialization Protocol for point to point) by
taking the link down and then transmitting NOS Primitive Sequence if LIP timeout or any of the
loop initialization phases timeout or only one non-zero AL_PA is claimed in Loop Initialization
S
Sequences
(LOOP
(LOOP_EMPTY=false)
EMPTY f l ) or no non-zero AL_PAs
AL PA are claimed
l i d (LOOP
(LOOP_EMPTY=true).
EMPTY t )
If the ACTIVE state is reached, the port will operate in the G_Port mode. (State 3)
The normal E_Port or F_Port mode detection procedure follows. (Transition 3)
If ELP succeeds, the U_Port operates in the E_Port mode. (State 4)
If a valid FLOGI is received, the U_Port becomes an F_Port. (State 5)
If self loopback detected after ELP exchanges and LOOP_EMPTY = false, port exits
G P t and
G_Port
d reinitializes
i iti li
as FL_Port.
FL P t (State
(St t 2)
Note: The firmware will automatically attempt to reinitialize a faulty port every two seconds.
Revision 1008
3 - 10

Switch to Switch (ISL) connections use E_Ports.

Nodes that attach to the fabric can either be an N (Node) or NL (Node Loop).
Public NL nodes can communicate with any member of the same loop and have the
ability to send a frame to the fabric.
Fabric Nodes (N) can communicate with any other Fabric Node and can
communicate with private and public NL nodes on a loop.
Switched Fabrics: An extensive storage network in which a large number of servers
and storage systems are connected using Fibre Channel switches.
Revision 1008
3 - 11

In addition to specifying node port and topology behavior, Fibre Channel defines
several generic services that are used to manage a Fibre Channel network.
The specific address assigned to each generic service is of the format 0xFFFFFx.
The next slide identifies the services provided and the Well-Known Address for
each.
Revision 1008
3 - 12

Every switch has reserved 24-bit addresses known as Well Known Addresses. The services
residing at these addresses provide a service to either nodes or management applications in
the fabric.
FFFFF6 Clock Synchronization
y
Server: Clock Synchronization
y
over Fibre Channel is attained
through a Clock Synchronization Server that contains a reference clock. The Server
synchronizes clients clocks to the reference clock on a periodic basis, using either Primitive
Signals or ELS frames.
FFFFF7 Security Server: The security-key distribution service offers a mechanism for the
secure distribution of secret encryption keys.
FFFFF8 Alias Server: The Alias Server manages the registration and deregistration of Alias
IDs for both Hunt Groups and Multicast Groups. The Alias Server is not involved in the routing
of frames for any Group.
FFFFFA Management Server: The Management server provides a single point for managing
the fabric.
FFFFFB Time Server: The time server sends to the member switches in the fabric the time on
either the principal switch or the Primary FCS switch.
FFFFFC Directory(Name Server): The directory server/name server is where fabric/public
nodes register themselves and query to discover other devices in the fabric
fabric.
FFFFFD Fabric Controller: The fabric controller provides state change notifications to
registered nodes when a change in the fabric topology occurs.
FFFFFE F_Port(Fabric Server Login): Before a fabric node can communicate with services
on the switch or other nodes in the fabric, an address is assigned by the fabric login server.
Fabric addresses assigned to nodes are 3 bytes long and are a combination of the domain ID
plus the port area number of the port the node is attached to.
FFFFFF Broadcast
B
d
t Server:
S
Wh a frame
When
f
is
i transmitted
t
itt d tto thi
this address,
dd
the
th frame
f
is
i
broadcast to all operational N and NL ports.
Revision 1008
3 - 13

When a node attaches to the fabric, it must receive a unique 24-bit address. The
network address is a three-byte address based upon the Domain ID, the Area ID
and, if a loop device, its AL_PA. This address is the source address and is used for
routing data thru the fabric from one device to another
another.
Footnote1: XX will be 00 for Fabric OS switches and 13 for M-EOS switches.
Fabric-attached devices use an address format of DD AA 00. This is the address
of any Fabric-attached device that has logged into the fabric as point-to-point.
Public Loop attached devices use an address format of DD AA PP. The DD AA
bytes of the address come from the fabric login process and the PP byte is
assigned
i
dd
during
i FC
FC_AL
AL initialization.
i iti li ti
NPIV attached devices use an address format of DD AA PP. The DD AA bytes of
the address come from the fabric login process and the PP byte is assigned during
Login process. More information on NPIV at the end of this module.
Revision 1008
3 - 14

Footnote 1: A device can be an initiator or a target.

A device does a Fabric Login (FLOGI) to determine if a fabric is present and, if so,
exchange service parameters with the fabric. A successful Fabric Login sends back
th 24
the
24-bit
bit address
dd
for
f the
th device
d i in
i the
th fabric.
f b i The
Th device
d i mustt do
d a FLOGI before
b f
communicating with other devices in the fabric.
Note: Since the device does not know its 24-bit address until after it does the Fabric
Login, the SID (Source ID) in the frame header making the FLOGI request will be
zeros (0x000000).
Buffer-to-Buffer credits for the device and switch ports are exchanged in the FLOGI
-> Accept
A
t commands.
d More
M
on this
thi llater
t iin thi
this module.
d l
Revision 1008
3 - 15

Each Brocade switch contains a Name Server that maintains local information. The
Name Server provides the local devices with access to the Name Service. The
Name Server runs a low-level task that communicates with each switch. The local
information from each switch is shared with one another
another, and kept in the Name
Server cache. When a new device gets added to a switch, that information is
immediately propagated out to all switches in the Fabric. This behind-the-scenes
communication is transparent to the local device.
Name Server characteristics:
No single point of failure
Transparent distribution
Server-to-server protocol (based on FC-CT)
The Name Server, specified in the Fibre Channel Standard, is responsible for
directory information about Fabric-connected devices.
The Name Server maintains name and address information about Fabric-connected
d i
devices
iin a complex,
l
di
diverse environment
i
t supported
t db
by iinterconnected
t
t dB
Brocade
d
switches. Brocade adds value to the Name Service in two ways:
1. By distributing the Name Server throughout the Fabric.
2. By providing automatic registration of essential device information on behalf
of the devices, as well as deregistration.
The Name Server functions like a telephone directory.
The Name Server has no responsibility for the process of routing data among
devices.
During initialization, the Fabric Login allows the Fabric to discover devices.
Revision 1008
3 - 16

The Symbolic Port Name and Symbolic Node Name are free-form variables and are
not restricted by the Name Service. If no value is registered by the Nx_Port, then
the Symbolic Port Name and Symbolic Node name default to a null value.
Port
P t Id
Identifier
tifi - the
th 24
24-bit
bit address
dd
assigned
i
d by
b the
th switch
it h F
Fabric
bi
FC-4 Types - Most common are SCSI (8) and IP (5)
Port Type - N or NL
Symbolic Port/Node Name - a 256 character field that is used by the vendor
of the device.
Revision 1008
3 - 17

There is no single command to display the detailed Name Server information for all
devices in a multiple switch fabric. Use the nsshow command to display information
about the devices connected to the switch where this command is issued. Use
nscamshow to display information about devices connected to the remaining
switches in the fabric.
Use nsallshow to get a list of all the online node ports in the fabric.
Note: Each switch uses a cache to store the information about the devices
connected to the remaining switches in the fabric. It does not have to retrieve this
information from the other switches when the command is issued. The nscam
stands for Name Server Cache Manager.
Manager
Revision 1008
3 - 18

Displays:
Type: U for unknown, N for N_Port, NL for NL_Port

PID: 24-bit Fibre Channel address
COS: List of classes of service supported by device
PortName: Device port World Wide Name (WWN)
NodeName: Device node WWN
Permanent Port Name: Physical N_Port or NL_Port WWN
There might be additional lines if the device has registered any of the following information
(the switch automatically registers SCSI inquiry data for FCP target devices):
FC4s supported
IP address
Port and node symbolic names
Fabric Port Name. The WWN of the port on the switch to which the device is physically connected.
Hard address and/or port IP address
-r This value indicates what type of RSCN a device registers to receive.

SCR=0 Reserved
SCR=1 Fabric detected registration. Register to receive all RSCN requests issued by the fabric
controller for events detected by the fabric.
SCR=2
SCR 2 Nx_Port
N P t detected
d t t d registration.
i t ti
Register
R i t to
t receive
i allll RSCN requests
t iissued
d ffor events
t d
detected
t t d
by the affected Nx_Port.
SCR=3 Register to receive all RSCN request issued. The RSCN request returns all effected N_Port_ID
pages.
-t Displays the device type. The first part indicates the origination of the device.
Physical - Device connected to the Nx_Port, using FLOGI to login to the switch.
Virtual - Device contrived by the switch.
g FDISC to login
g to the switch.
NPV - Device connected to the Nx_Port,, using
iSCSI - Device connected to the iSCSI port.
Redirect: Frame redirection would be done by an application on the FA blade running some
SA/SAS. (eg. DMM or Invista)
Revision 1008
3 - 19

Use this command to display the local Name Server Cache information about the
devices discovered in the fabric by the Name Server Cache manager.
The message No Entry found! is displayed if the Name Server Cache Manager
h
hasn't
't discovered
di
d new switches
it h and
d any new d
devices
i
iin th
the ffabric.
bi
For each remote switch found, the output of this command shows the same
information as the nsshow command.
Note: The r option is not available with the nscamshow command.
Revision 1008
3 - 20

Displays global Name Server information

Use this command to display the 24-bit Fibre Channel addresses for all devices in
the fabric. If the operand type is supplied, only devices of specified FC-PH
FC PH type are
displayed. If type is omitted, all devices are displayed.
Below is the same command, only with the type 8 (SCSI):
sw2:admin> nsallshow 8
{
010100 0102e2 0102e4 0102e8 0102ef 020500 0206e2 0206e4
0206e8 0206ef
10 FCP Ports }
Revision 1008
3 - 21

A State Change Registration (SCR) is used by initiators to request notification from

the fabric when a state change occurs. When a state change occurs that the device
should know about, the fabric sends back a Registered State Change Notification
(RSCN) Only devices that do an SCR can receive an RSCN
(RSCN).
RSCN.
The Fabric Controller, with its well-known address at FFFFFD, handles this process.
RSL1_ST05_B200:admin> nsshow -r
{
Type Pid
N
COS
020000;
PortName
NodeName
SCR
2,3;10:00:00:00:c9:51:39:7d;20:00:00:00:c9:51:39:7d; 3
FC4s: FCP
NodeSymb: [52] "Emulex LP1150-F4 FV2.10A7 DV5-5.20A9 RSL1-ST05-W2K-1"
Fabric Port Name: 20:00:00:05:1e:02:1c:24
Permanent Port Name: 10:00:00:00:c9:51:39:7d
The Local Name Server has 1 entries }
Notice this device has registered for SCR 3 = Full Registration.

1 = Fabric RSCNs only
2 = Nx_Port RSCNs only
3 = Full (1 and 2) RSCNs
Revision 1008
3 - 22

Revision 1008
3 - 23

The above is an example of the frame communication between a host device and
the switch (fabric). Note: The HBA, among other things, is responsibility for framing
packets, physical addressing and link level error checking.
FLOGI
FLOGI: Fabric
F bi L
Login
i command:
d U
Used
d tto establish
t bli h a 24
24-bit
bit address
dd
for
f the
th
device logging in. Also establishes Buffer-to-Buffer credits, class of service
supported.
PLOGI: Port Login command: Device must login into the Directory (Name)
Server to Register its information as well as query for devices this device is
zoned with.
SCR
SCR: State
St t Change
Ch
R
Registration:
i t ti
D
Device
i needs
d tto register
i t ffor St
State
t Change
Ch
Notification so if there is a change is the fabric, such as a zoning change or a
change in the state of a device that this device has access to, the device will
receive an RSCN.
Registration: A device will exchange registration information with the Directory
(Name) Server.
Q
Query: Devices
D i
can query th
the Di
Directory
t
(Name)
(N
) Server
S
for
f information
i f
ti about
b t
the device it has access to.
Revision 1008
3 - 24

PLOGI: Port Login command: Initiator must login into the target.
PRLI: Process Login command: This established the operating (SCSI is the
most common) environment between the two N_Ports.
Inquiry: This example uses an INQ command, it could be something different
such as a report LUNs command for example. Which command is used is
determined by the driver on the initiator. The commands purpose is to get a list
of LUNs the initiator has access to.
Revision 1008
3 - 25

In this example, switch E is being added to the fabric.

For more information on this process, see the FC-SW4 Spec at www.t11.org
Revision 1008
3 - 26

Revision 1008
3 - 27

Revision 1008
3 - 28

Revision 1008
3 - 29

The port indexes are the same for both FC4-48 and FC8-48 blades.
For each blade, 24 ports per ASIC: Ports 0-7 and 24-39 (ports below red line) on
one ASIC, ports 8-23 and 40-47 (ports above red line) are on the other ASIC.
The grey boxes represent port Indexes 0-127.
The blue boxes represent port Indexes 128-255.
The yellow boxes represent port Indexes 256-383.
Revision 1008
3 - 30

The diagram above shows the port numbers that would exist for the domain in the
chassis depending on the port card type inserted in the slot.
Revision 1008
3 - 31

The diagram above shows the port numbers that would exist for the domain in the
chassis depending on the port card type inserted in the slot.
Revision 1008
3 - 32

The ability to address 384 ports in a single switch required a change to the 24-bit
addressing scheme (PID). The second byte of a PID is referred to as the Area ID.
With 8 bits, the Area ID can address ports 0-255. Brocade now uses the third byte
of the PID to address ports 256-383
256-383. The third byte of a PID is referred to as the
Node Address. The Node Address was used to identify the address (ALPA) for a
loop device. Since an FCx-48 port does not support loop devices, the Node Address
can be used to identify ports in the 256-383 range. This also requires the Area ID to
be shared (used twice).
Revision 1008
3 - 33

The grey boxes represent Port Indexes 0-127. The Area ID for these Indexes are
not shared.
The blue boxes represent the Port Indexes of one ASIC on each FCx-48 port card
thatt are shared
th
h d ((ports
t 16-23
16 23 share
h
the
th same Area
A
ID with
ith ports
t 40
40-47,
47 respectively).
ti l )
The yellow boxes represent the Port Indexes on the other ASIC on each FCx-48
port card that are shared (ports 24-31 share the same Area ID with ports 32-39,
respectively).
With shared Area IDs, the lower port number on the card has a Node Address of
0x00, while the higher port number on the card has a Node Address of 0x80.
The Fabric OS v5.2 Admin Guide section on Identifying Ports by Index also has a
representation of the information in this table.
Revision 1008
3 - 34

Example:
Ports 24-31 on the blade in Slot 3 share the same Area Id with ports 32-39 on the
same blade, respectively. Ports 24-31 will have a Node Address of 0x00 and ports
32 39 will
32-39
ill have
h
a Node
N d Address
Add
off 0x80.
0 80 This
Thi makes
k their
th i 24
24-bit
bit addresses
dd
unique.
i
Revision 1008
3 - 35

NPIV devices connected to the same switch port must have a unique 24-bit address
as well as a unique device PWWN.
Revision 1008
3 - 36

sw2:admin> nsshow
{
Type Pid
COS
PortName
NodeName
N
620800;
3;20:08:00:05:1e:07:c3:9f;10:00:00:05:1e:07:c3:9f;
Fabric Port Name: 20:08:00:05:1e:0c:ab:65
Permanent Port Name: 20:08:00:05:1e:07:c3:9f
Port Index: 8
Share Area: No
Device Shared in Other AD: No
Redirect: No
N
620801;
3;10:00:00:05:1e:42:86:88;20:00:00:05:1e:42:86:88;
FC4s: FCP
PortSymb: [46] "BRE042 A.2 L3-25014-01B FW:01.03.19 Port 0
"
Port Index: 8
Share Area: No
Redirect: No
N
620802;
3;10:00:00:05:1e:42:86:88;20:00:00:05:1e:42:86:88;
FC4s: FCP
PortSymb: [46] "BRE042 A
A.2
2 L3
L3-25014-01B
25014 01B FW:01
FW:01.03.19
03 19 Port 0
"
Port Index: 8
Share Area: No
Redirect: No
The Local Name Server has 3 entries }
Revision 1008
TTL(sec)
na
na
na
3 - 37

RSL1_ST05_B41:admin> portcfgnpivport 0 0
RSL1_ST05_B41:admin> portcfgshow
Ports of Slot 0
9 10 11
12 13 14 15
-----------------+--+--+--+--+----+--+--+--+----+--+--+--+----+--+--+-+ + + + +
+ + + +
+ + + +
+ + +
Speed
AN AN AN AN
AN AN AN AN
AN AN AN AN
AN AN AN AN
Trunk Port
ON ON ON ON
ON ON ON ON
ON ON ON ON
ON ON ON ON
Long Distance
.. .. .. ..
.. .. .. ..
.. .. .. ..
.. .. .. ..
VC Link Init
.. .. .. ..
.. .. .. ..
.. .. .. ..
.. .. .. ..
Locked L_Port
.. .. .. ..
.. .. .. ..
.. .. .. ..
.. .. .. ..
Locked G
G_Port
Port
.. .. .. ..
.. .. .. ..
.. .. .. ..
.. .. .. ..
Disabled E_Port
.. .. .. ..
.. .. .. ..
.. .. .. ..
.. .. .. ..
ISL R_RDY Mode
.. .. .. ..
.. .. .. ..
.. .. .. ..
.. .. .. ..
RSCN Suppressed
.. .. .. ..
.. .. .. ..
.. .. .. ..
.. .. .. ..
Persistent Disable.. .. .. ..
.. .. .. ..
.. .. .. ..
.. .. .. ..
NPIV capability
ON ON ON ON
ON ON ON ON
ON ON ON ON
Revision 1008
.. ON ON ON
3 - 38

Revision 1008
3 - 39

Revision 1008
3 - 40

Appendix A Fibre Channel Reference List

Title: Principles of SAN Design
Authors: Josh Judd & Dan Krueger
ISBN: 0-7414-2824-5
Title: Multiprotocol Routing for SANs
Author: Josh Judd
ISBN: 0-7414-2306-5
Title: Fibre Channel: A Comprehensive Introduction
Author: Robert W. Kembel
ISBN: 0-931836-84-0
Title: Fibre Channel Switched Fabric
Author: Robert W. Kembel
ISBN: 0-931836-71-9
Title: IP SANS: An Introduction to iSCSI, iFCP, and FCIP Protocols for Storage Area Networks
Author: Tom Clark
ISBN: 0-201-75277-8
Title: Fibre Channel, Gigabit Communications and I/O for Computer Networks
Author: Alan F. Benner
ISBN: 0-07-005669-2
Title: Designing Storage Area Networks: A Practical Reference for Implementing Fibre Channel SANs
(The Addison-Wesley Networking Basics Series)
Author: Tom Clark
ISBN: 0-201-61584-3
Title: A Practical Guide to SNMPv3 and Network Management
Author: David Zeltserman
ISBN: 0-13-021453-1
Title: Storage Area Networks Designing and Implementing a Mass Storage System
Authors: Ralph H. Thornburgh and Barry J. Schoenborn
ISBN: 0-13-027959-5
Title: Fibre Channel for SANs
Author: Alan F. Benner
ISBN: 0-07-137413-2
http://www.fibrechannel.com
Fibre Channel Industry Association a trade group promoting the use of fibre channel.
http://www.T11.org
Home page for the National Committee for Information Technology Standards (NCITS) T11 committee. This
group sets the fibre channel standards.
http://www.T10.org
Home page for the NCITS T10 committee. This group set the SCSI standards.
http://www.scsita.org
SCSI Trade Association - information about SCSI.
http://www.searchstorage.com
A storage-specific search engine.
http://www.SNIA.org
Storage Network Industry Association a trade group that promotes the use of storage area networks.
http://www.infostor.com
InfoStor a free monthly industry journal on storage. Apply at the website to qualify for a subscription.
http://compnetworking.about.com/cs/fibrechannel/index.htm
Revision 1008
3 - 41

Revision 1008
3 - 42

Revision 1008
Installation and Configuration
4-1

Revision 1008
4-2

Air flow for Brocade switches is from the non-cable side to the cable side because
the heat-generating components of the switch (ASICs and SFPs) are on the port
side.
IImproper cable
bl planning
l
i can cause problems
bl
and
d may iimpactt performance
f
iin th
the
SAN. Although patch panels are helpful in a cable management scenario, attempt to
minimize the number of connections as every fiber optic interconnection generates
a few dB of signal loss. Keep a manageable slack to minimize cable stress. Use
different color Velcro straps for trunk groups.
For more information on power supplies, fans, and temperature readings see the
hardware reference manual for the appropriate switch model
model.
Revision 1008
4-3

Footnote 1: The SMI Agent software provides a Common Information Model (CIM) agent for switch and director
products. The SMI Agent performs the functions of a general purpose server as defined in the Storage
Management Initiative Specification (SMI-S version 1.2.0) and enables a standard set of management functions
to be performed by third-party CIM clients.
Footnote 2: Secure Socket Layer (SSL) is part of base Fabric OS. SSL works by using a key to encrypt data
transferred over an SSL connection. By convention, URLs that require an SSL connection start with https:
instead of http: All Brocade supported Internet browsers support SSL.
Configuration of the SSL protocol involves obtaining, installing, and configuring PKI certificates:
Public Key Infrastructure (PKI) is a system of public key encryption using digital certificates from a
Certificate Authority (CA) and other registration authority to verify and authenticate the validity of each
party involved in an electronic transaction.
The CA works as part of a Public Key Infrastructure (PKI) and therefore checks with a registration
authority (RA) to verify digital certificate requestor information. Once RA verifies information CA can
issue a certificate. The information that the RA verifies depends on the CA, but includes items such as
owners public key; certificate expiration date; owners name and other public key owner information.
Secure access implementations that involve a Public Key Infrastructure (PKI) include three parts:
Part 1 Certificate Signing Request (CSR) management (SSL access CSRs have to be generated at the
CLI): Generate/delete/Display/Export CSR
Part 2 Certificate Authority (CA) management:
Send Certificate Signing Request (CSR) to CA and Get certificate(s) back from CA (SSL uses 3rd party
CAs such as Verisign, Entrust, InstantSSL, and GeoTrust.)
a t 3 Import
po t and
a d configure
co gu e ce
certificates
t cates SS
SSL requires
equ es 3 ce
certificates
t cates to be imported
po ted from
o tthe
eC
CLI:
Part
A Certificate Authority (CA) certificate that authenticates the switch certificate generated by a 3rd
party. CA is a chainroot certificate which means that the switch manufacturer (Brocade) does not
own the ROOT CA (it was obtained from a 3rd party). This CA certificate needs to be imported and
installed on each switch that will be accessed via HTTPS (Example:
ComodoSecurityServicesCA.crt).
A switch certificate that authenticates the switch needs to be imported and installed on each switch
that will be accessed via HTTPS (Example: 192_168_188_195.crt).
A server Root certificate needs to be installed on each server that will be used to access a switch
using HTTPS (Example: GTECyberTrustRoot.crt).
Note: The Root certificate, CA authentication certificate, and the switch certificate must come from the same
CA. One pair of CA and switch certificates are needed per switch. Each switch certificate will be unique. If the
same CA is used multiple switches could import the same CA certificate, but each switch will need to import it.
Revision 1008
4-4

When administrating Brocade switches using the command line interface, certain
key strokes can be helpful administrating the switch.
Footnote 1: The tab key will complete a command only when there is one available
command option
option. For e
example,
ample if you
o enter B51:admin>
B51 d i > cfgs
f
and then press
the tab key nothing will happen as there is more than one command that starts with
cfgs. However, B51:admin> cfgsa will complete the command cfgsave.
If more than one command is available, pressing the tab key twice will bring up a list
of remaining available commands.
B51:admin> cfgs
cfgsave
cfgshow
cfgsaveactivetodefined
cfgsize
B30:admin> cfgs
Revision 1008
4-5

Revision 1008
4-6

When a new switch has arrived for installation into a fabric, it is suggested to use a serial cable to
configure the switch with an IP address. After the IP address is configured, the serial connection to
the switch may be dropped and an SSH, telnet, or Web Tools session may be used for further switch
configuration because of its convenience and speed.
To configure the connection in a B-Series
B Series environment:
Bits per second: 9600
Data bits: 8
Parity: None
Stop bits: 1
Flow control: None
To configure the connection in an M-Series environment:
Installation steps
1. Insert the serial cable provided to an RS-232 serial port on the workstation
FOS switches use a straight-through cable
M-EOS switches use a null modem cable
2. Verify the switch has power and is past the POST stage
3. Invoke the ipaddrset command to set the IP address, subnet mask and default gateway
Revision 1008
4-7

B51:admin> ifmodeshow eth0

Link mode: negotiated 100baseTx-FD, link ok
B51:admin> ifmodeset eth0
Exercise care when using this command. Forcing
mode not supported by the network equipment to
result in an inability to communicate with the
interface. It is recommended that you only use
serial console port.
the link to an operating

which it is attached may
system through its Ethernet
this command from the
Are you sure you really want to do this? (yes, y, no, n): [no] yes
P
Proceed
d with
ith caution.
ti
Auto-negotiate (yes, y, no, n): [no]
Force 100 Mbps / Full Duplex (yes, y, no, n): [no]
Force 100 Mbps / Half Duplex (yes, y, no, n): [no]
Force 10 Mbps / Full Duplex (yes, y, no, n): [no]
Force 10 Mbps / Half Duplex (yes, y, no, n): [no]
You must select at least one link operating mode.
B51:admin> ipaddrset
Ethernet IP Address [10.255.248.35]:
Ethernet Subnetmask [255.255.255.192]:
Fibre Channel IP Address [0.0.0.0]:
Fibre Channel Subnetmask [0.0.0.0]:
Gateway IP Address [10.255.248.62]:
Issuing gratuitous ARP
ARP...Done.
Done
IP address is being changed...Done.
Committing configuration...Done.
Revision 1008
4-8

B51 login: admin

Password:
Please change your passwords now.
Use Control
Control-C
C to exit or press 'Enter' key to proceed
proceed.
Password was not changed. Will prompt again at next login
until password is changed.
B51:admin> killtelnet
[* abbreviated *]
__________________________________________________________________________
____
Session No
LOGIN@
USER
TTY
IDLE
FROM
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
~~~~
0
1:47pm
admin0
pts/0
1.00s
10.255.248.22
1
1:52pm
admin0
pts/1
9.00s
10.255.248.22
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
~~~~
Enter Session Number to terminate (q to quit) 1
Please Ensure (Y/[N]): Y
killing session....
Revision 1008
Done!
4-9

Default timeout on Linux-based switches is 10 minutes.
Revision 1008
4 - 10

B51:admin> date
Tue May 16 15:00:57 UTC 2006
B51:admin> tsclockserver
LOCL
B51:admin> tsclockserver 128.118.25.3
Updating Clock Server configuration...done.
128.118.25.3
B51:admin> date "0516073406"
External Time Synchronization in place. Cannot execute this
command.
B51:admin> tsclockserver LOCL
Updating Clock Server configuration...done.
LOCL
B51:admin> date "0516073406"
Tue May 16 07:34:00 UTC 2006
Revision 1008
4 - 11

B51:admin> tstimezone -5
Updating Time Zone configuration...done.
System Time Zone change will take effect at next reboot.
Revision 1008
4 - 12

B51:admin> bannerset
Please input content of security banner (press "." and RETURN
at the beginning of a newline to finis
h input):
Unauthorized access is prohibited.
Do not log in if you do not have the authorization to do so.
.
B51:admin> login
B51 login: admin
Password:
B51:admin> bannershow
B51:admin> bannerset ""
B51:admin> bannershow
Revision 1008
4 - 13

B51:admin> licenseidshow
10:00:00:05:1e:02:ab:21
b51:admin> licenseshow
bQSyeQbQcccRTzfu:
Fabric license
One feature per

license key
RzdczyebRdS0id0i:
Second Ports on Demand license - additional 8 port
upgrade license
WKAHHGEEMNrBEKC4NDWR4garNA7RNtDfBJFWE:
8 Gig FC license
SgYXFZAtaEK7GDLQJrRgfTKCa3ENMK4NB7RBN:
Multiple features
per license key
Extended Fabric license

Fabric Watch license
Performance Monitor license
Trunking license
FICON_CUP license
First Ports on Demand license - additional 8 port upgrade
license
Unknown license results when a license
Integrated Routing license
from a previous OS is no longer valid
Adaptive Networking license
Example: Web Tools is not longer a
licensed feature
Unknown30 license
Unknown31 license
Revision 1008
4 - 14

Having a well thought-out switch naming convention enables easy

identification of physical switches if a problem arises. Use a switch naming
convention that scales across the organization, keeping in mind that the SAN
might
i ht start
t t smallll b
butt can b
be extended
t d d enterprise-wide
t
i
id over titime. S
Switch
it h
names can be duplicated in the fabric. To see a list of the existing switch
names and their IP settings, use the command fabricshow.
Switch Name rules:
Up to 15 characters including letters, digits, hyphens, and underscore
characters
Must begin with a letter
No spaces
SW:admin> switchname "B51"
Committing configuration...
Done.
B51:admin> switchname
B51
B51:admin> fabricshow
Switch ID
Worldwide Name
Enet IP Addr
FC IP Addr
Name
------------------------------------------------------------------------1: fffc01 10:00:00:05:1e:02:12:a5
10.255.248.32
0.0.0.0
"B30"
2: fffc02 10:00:00:05:1e:02:ab:21
10.255.248.35
0.0.0.0
>"B51"
The Fabric has 2 switches
Revision 1008
4 - 15

B51:admin> syslogdipshow
No addresses configured
B51:admin>
B51:admin> syslogdipadd 10.255.248.2
B51:admin> syslogdipadd 10.255.248.3
syslog.IP.address.1
10.255.248.2
syslog.IP.address.2
10.255.248.3
B51:admin> syslogdfacility
Syslog facility: LOG_LOCAL7
B51:admin> syslogdfacility -l 6
Syslog facility changed to LOG_LOCAL6
B51:admin> syslogdipremove 10.255.248.3
syslog.IP.address.1
10.255.248.2
B51:admin> syslogdipremove 10.255.248.2

No addresses configured
Revision 1008
4 - 16

Revision 1008
4 - 17

Footnote 1: To create an IP Filter policy:

ipfilter --create <policyname> -type <ipv4 | ipv6>
<policyname> is a unique string composed of maximum 20 alpha,
numeric or underscore characters
The name default is reserved (default_ipv4 and default_ipv6)
The policy name is case insensitive but is always stored as
lo er case
lower
ipfilter --addrule <policyname> -rule <rule_number> -sip
<source_IP> -dp <dest_port> -proto <protocol> -act <permit |
deny>
Note: Previous to Fabric OS v5.3 use the configure command to disable telnet.
Revision 1008
4 - 18

Footnote 1: All printable punctuation characters except colon ":" are allowed.
Footnote 2: The minimum password length may be set from 8 to 40 characters in length. The
password length is the total number of lowercase, uppercase, digits, and punctuation characters. The
total number of these characters may not exceed 40. Keep this in mind as you specify the minimum
number of each type of character required.
Footnote 3: The password history policy is not enforced when an administrator sets a password for
another user, but the password set by the administrator is recorded in the user's password history.
swd77:admin> passwdcfg --set -lowercase 3 -uppercase 1 -digits 2 punctuation 2 -minlength 10 -history 3
swd77:admin> passwd
Changing password for admin
Enter old password:
Enter new password:
Password must be between 10 and 40 characters long.
Enter new password:
Insufficient number of upper case letters
p
Enter new password:
Insufficient number of lower case letters
[* abbreviated *]
Enter new password:
Insufficient number of digits in password
Enter new password:
yp new p
password:
Re-type
passwd: all authentication tokens updated successfully
Saving password to stable storage.
Password saved to stable storage successfully.
Revision 1008
4 - 19

Footnote 1: The password expiration policy is not enforced for root and factory accounts.
Footnote 2: The user will begin seeing warning messages when they login a number of
days prior to password expiration. They will be compelled to change their password when it
has expired.
Footnote 3: The account lockout policy is not enforced for root, factory, and admin role
accounts.
swd77:admin> passwdcfg --set -minpasswordage 20 -maxpasswordage 30 warning 5
swd77:admin> userconfig --show -a
[* abbreviated *]
Account name: root
Role: root
Description: root
Enabled: Yes
Password Last Change Date: Unknown
Pass ord E
Password
Expiration
piration Date
Date: Not Applicable
Locked: No
Account name: admin
Role: admin
Description: Administrator
Enabled: Yes
Password Last Change Date: Wed May 24 2006
Password Expiration Date: Fri Jun 23 2006
Locked: No
Revision 1008
4 - 20

B51 login: admin

Password:
Please change your passwords now.
Use Control
Control-C
C to exit or press 'Enter' key to proceed
proceed.
Warning:
proper
Access to
support
passwords are
of
the Root
and Factory accounts may be required
the switch. Please
ensure
the Root
and
for
Factory
documented in a secure location. Recovery of a lost Root
or Factory password will result in fabric downtime.

for user - root
Changing password for root
Enter new password:
Re-type new password:
Please change your passwords now
now.
for user - factory
for user - admin
for user - user

Saving passwords to stable storage.
Passwords saved to stable storage successfully
Revision 1008
4 - 21

Revision 1008
4 - 22

Revision 1008
4 - 23

Revision 1008
4 - 24

For each role there is a set of pre-defined permissions on the commands that can
be performed.
FOS v5.3 added the securityadmin role.
FOS v5.2 added the zoneadmin, basicswitchadmin, operator, and fabricadmin
roles.
FOS v5.0 added the switchadmin role.
Revision 1008
4 - 25

Each role is assigned a permission type for a category of commands. The

permission applies to all commands within the category.
Revision 1008
4 - 26

The categories above were chosen as an example to illustrate the permissions

assigned to each role. For a complete list of commands and role permissions,
see the Fabric OS Command Reference.
Revision 1008
4 - 27

B51:admin> userconfig --show -a

Account name: root
Role: root
Description: root
Enabled: Yes
Account name: factory
Role: factory
Description: Diagnostics
Enabled: Yes
Account name: admin
Role: admin
Enabled: Yes
B51:admin> userconfig --add jdoe -r admin d "Jane Doe"

Setting initial password for jdoe
Enter new password:
Re-type new password:
Account jdoe has been successfully added.
B51:admin> login
B51 login: jdoe
Password:
B51:jdoe> userconfig --show jdoe
Account name: jdoe
Role: admin
Description: Jane Doe
Enabled: Yes
B51:jdoe> userconfig --change admin -e no
Account name: user

Role: user
Description: User
Enabled: Yes
DANGER
Revision 1008
Your company policy may require you to disable default

accounts or group-access accounts such as the default
user and admin accounts. Before you disable the
default account admin, be certain you have created at
least one user-defined account assigned to the admin
role. Without an account with admin privileges, you will
not be able to manage your switch.
Broadcast message from root (pts/0) Wed May

17 09:14:48 2006...
Security Policy, Password or Account
Attribute Change: admin will be logged out
Attribute for account admin has been
successfully changed.
B51:jdoe> userconfig --show admin
Account name: admin
Role: admin
Enabled: No
B51:jdoe>
4 - 28

To implement a highly available solution that provides redundancy and minimizes

the effect of network outages:
Implement multiple LDAP servers on the network, configure all switches to
authenticate
th ti t with
ith allll LDAP servers, and
d configure
fi
allll switches
it h to
t use a local
l
l
database as secondary authentication.
Use the aaaconfig command to add a LDAP server.
Revision 1008
4 - 29

This command has the following operands:

--maprole - Maps an LDAP role to a specified switch role. The following operands
are required:
--ldaprole - Specifies the LDAP role to be mapped to a switch role. The role
must be a valid AD server role.
--switchrole - Specifies the switch role to which the LDAP role is mapped.
Valid switch roles include the following:
- admin
- user
- switchadmin
- zoneadmin
- fabricadmin
- basicswitchadmin
- operator
- securityadmin
--unmaprole - Removes the mapping between an LDAP role and a switch role.
Use the --show option for a listing of existing mappings. The following operand is
required:
--ldaprole - Specifies the LDAP AD sever role to be removed from the
mapping.
show - Displays a table of existing mappings between LDAP roles and their
--show
corresponding switch role.
--help - Displays command usage.
Revision 1008
4 - 30

LDAP roles are just names assigned to a switch role. They only apply to the
available RBAC roles and not the root account.
Revision 1008
4 - 31

The Remote Authentication Dial-In User Service (or RADIUS) is a protocol for
carrying Authentication, Authorization, and Accounting (AAA) information about
remote user access between a Network Access Server (which desires to
authenticate its links) and a shared Authentication Server.
Server RADIUS is an open
standard (IETF RFC 2865 and RFC 2866).
Client/server: The RADIUS client must pass user information to designated RADIUS
servers, and act on the returned response. The RADIUS server receives user
connection requests, authenticates the users, and then returns all configuration
information needed for the RADIUS client to deliver service. In this case, a Brocade
switch is configured
g
as a Network Access Server that acts as a RADIUS client.
Network Security: To ensure that user names and passwords remain private, all
client/server communication is encrypted, and authenticated with a shared secret
key.
RADIUS is focused on authenticating, authorizing, and accounting remote user
access in particular, logins and logouts. RADIUS does not perform these roles for
devices or switches entering
g a fabric these roles continue to be handled byy
existing Fibre Channel protocols.
In a fabric with switches running a mix of Fabric OS versions, the way a switch
authenticates users depends on whether a RADIUS server is set up for that switch.
For a switch with RADIUS support and configuration enabled, authentication
bypasses the local password database. On a RADIUS-enabled switch, logins
through the console port are not authenticated with the RADIUS server, but through
the local switch database.
For a switch with RADIUS support or configuration disabled, authentication uses
switch local account names and passwords.
Revision 1008
4 - 32

Footnote 1: Web Tools Switch Database is referred to as switchdb at the CLI.

If a denial (incorrect user name / password) is received from the RADIUS server
that is authenticating the login, then a secondary RADIUS server or authentication
d t b
database
llogin
i iis nott attempted.
tt
t d
If a configuration parameter is incorrect in either the RADIUS server or the switch
AND if the secondary database is Switch Database, then telnet or Web Tools Admin
access could be gained, after timeout(s), using a local account that authenticates
via the Switch Database.
Revision 1008
4 - 33

In the example above, the primary database is RADIUS, and the RADIUS servers
have been properly configured with user names and passwords. When a
management station attempts access, the following scenario occurs:
1 Th
1.
The managementt server attempts
tt
t to
t access the
th switch
it h (RADIUS client)
li t) using
i a
user name/password combination configured on the RADIUS servers.
2. The authentication request is sent to the first RADIUS server in the RADIUS
configuration (RADIUS Server 1).
3. If the response from RADIUS Server 1 is accept, management access is
achieved; if the response is deny, the management server does not get
access.
4. If there is a timeout from RADIUS Server 1, then the authentication request is
sent to the second RADIUS server in the RADIUS configuration (RADIUS Server
2).
5. If the response from RADIUS Server 2 is accept, management access is
achieved; if the response is deny, the management server does not get
access.
6. If there is a timeout from RADIUS Server 2 AND the switch database is
configured as a secondary database, then the user name/password is
authenticated on the local switch.
Note: Up to five RADIUS servers can be configured.
Revision 1008
4 - 34

Authentication protocols that can be configured include:

Authentication Protocol (PAP)
Challenge-Handshake Authentication Protocol (CHAP)
Protected Extensible Authentication Protocol (PEAP)
Revision 1008
4 - 35

1. Select Switch Admin

2. Select Show Advanced Mode
Revision 1008
4 - 36

3. Select the AAA Service tab

4. Select Add
5. Add RADIUS server information
(AAA = Authentication, Authorization, & Accounting)
Revision 1008
4 - 37

Revision 1008
4 - 38

Footnote 1: This is a generic definition of fabric interoperability. Fabric OS v6.0

fabric interoperability is only supported between classic Brocade (B-Series) and
classic McDATA (M-Series) switches. Direct E_Port connections are established
using ISLs with no FC Routing or Access gateway functionality
functionality.
Fabric OS Fabric OS: the firmware on B-Series switches and Directors (e.g. 5000,
48000, etc.).
M-EOS McDATA Enterprise OS, aka M-EOS: the firmware on B-Series switches
and Directors (e.g. M4700, M6140, etc.). M-EOS includes both M-EOSc for
Sphereon and M6000s and M-EOSn for the Mi10K.
Footnote
F
t t 2
2: E
Enables
bl th
the exchange
h
off ffabric
b i parameters,
t
allowing
ll i switches
it h tto merge
into one fabric with one principal switch with each switch having a unique domain
ID. Brocades goal is to provide E_Port interoperability between B-Series (classic
Brocade) and M-Series (classic McDATA) products at the physical, services, and
management layers. This approach will give customers the flexibility to expand their
existing SANs and to build new networks in the most seamless, efficient, and
supportable
pp
manner p
possible.
Footnote 3: AG connectivity capability uses Node Port Identifier Virtualization (NPIV)
functionality in Brocade embedded switches and the Brocade 200E in AG mode.
NPIV enables multiple hosts to connect through one port to any fabric switch that
supports NPIV. Fabric OS v6.0 enhancements to AG are discussed in another
instructor-led module associated with this course.
Footnote 4: FCR provides physical connectivity between hosts/storage in different
fabrics while keeping the fabrics isolated.
Revision 1008
4 - 39

Footnote 1: Fabric OS v6.0 introduces a new interopmode 2 built from the

ground up. Enhances Fabric OS v5.2.1_NI interopmode 2 functionality Interop
mode restrictions:
The 8 Gbit/sec Brocade DCX supports interop modes 2 or 3 starting in Fabric
OS v6.0.0a+, check release notes for current support.
The Brocade 4100 can load Fabric OS v6.0 but it will not run interop modes 2
or 3
Fabric OS Interop Mode
interopmode #
CLI Documentation Description
FOS Native Mode
interopMode off
FOS Interop Mode 1, obsolete
Not listed in the Fabric OS v6.0+ CLI

guide
FOS McDATA Native Mode
McDATA Fabric Mode
FOS McDATA
M DATA Open
O
Fabric
F b i Mode
M d
M DATA Open
McDATA
O
Fabric
F b i Mode
M d
Revision 1008
4 - 40

Revision 1008
4 - 41

Footnote 1: When zoning from a B-Series switch for the first time, use the
cfgsaveactivetodefined command to save the active config to the defined
config in order to make zoning changes. Zoning changes on B-Series switches are
performed and saved to the defined config
config. M-Series switches do not use a defined
config.
Revision 1008
4 - 42

Footnote 1: Zoning can NOT be enabled or configured from a Fabric OS switch in

McDATA Open Fabric Mode. Initiating zone activations and copying the M-EOS
fabric Effective zone configuration into the Define zone configuration database are
not permitted.
permitted
The following commands are not permitted in mode 3:
cfgenable/cfgdisable, cfgsaveactivetodefined, and cfgmcdtmode
These commands generate an error message when invoked in mode 3:
Error: This command cannot be executed in McDATA Open Fabric
mode.
Footnote 2: HCA may work but it is not supported. Please check release notes to
see if supported.
Footnote 3: Check the latest release notes for an up-to-date list of Fabric OS v6.x
features qualified in Open Fabric Mode (interopmode 3). For Fabric OS v6.0 most
proprietary Brocade features will be disabled. This includes but is not limited to:
Frame Level Trunking, Virtual Channels between switches, Administrative Domains,
and Frame Redirection
Redirection. Fabric OS v6
v6.1
1 added support for Frame level Trunking
between B-Series switches while in Open Fabric Mode (interopmode 3).
Revision 1008
4 - 43

Revision 1008
4 - 44

Revision 1008
4 - 45

B51:admin> switchstatusshow
Switch Health Report
05/06/2008 09:54:21 PM
Report time:
Switch Name:
B51
IP address:
192.168.176.59
SwitchState:
HEALTHY
Duration:
06:23
Power supplies monitor
HEALTHY
Temperatures monitor
HEALTHY
Fans monitor
HEALTHY
Flash monitor
HEALTHY
Marginal ports monitor
HEALTHY
Faulty ports monitor
HEALTHY
Missing SFPs monitor
HEALTHY
All ports are healthy
Revision 1008
4 - 46

switchstbatusshow will display the overall status of the switch that include
internal switch status, faulty ports, missing SFPs, power supplies, temperatures,
fans, portstatus, and ISLStatus. The status may be one of the following:
marginal/warning or down/failed
down/failed.
switchstatuspolicyshow: This command prints the current policy parameters
for calculating the overall status of the switch. The tolerances for calculating the
status of the switch can be configured with switchstatuspolicyset.
B300E:admin> switchstatusshow
S it h H
Switch
Health
lth R
Report
t
Report time: 05/21/2006 09:37:31
AM
Switch Name:
B300E
IP address:
10.255.248.32
SwitchState:
HEALTHY
Duration:
70:56
Power supplies monitor
HEALTHY
Temperatures monitor
HEALTHY
Fans monitor
HEALTHY
Flash monitor
HEALTHY
Marginal ports monitor
HEALTHY
Faulty ports monitor
HEALTHY
Missing SFPs monitor
HEALTHY
All ports are healthy

Revision 1008
4 - 47

B51:admin> switchstatuspolicyset
To change the overall switch status policy parameters
The current overall switch status policy parameters:
Down
Marginal
---------------------------------PowerSupplies
1
1
Temperatures
2
1
Fans
2
1
Flash
0
1
MarginalPorts
2
1
FaultyPorts
2
1
MissingSFPs
0
0
Note that the value
value, 0
0, for a parameter
parameter, means that it is
NOT used in the calculation.
** In addition, if the range of settable values in the prompt is (0..0),
** the policy parameter is NOT applicable to the switch.
** Simply hit the Return key.
The minimum number of
Bad PowerSupplies contributing to DOWN status: (0..1) [1]
Bad PowerSupplies contributing to MARGINAL status: (0..1) [1]
Bad Temperatures contributing to DOWN status: (0
(0..2)
2) [2]
Bad Temperatures contributing to MARGINAL status: (0..2) [1]
Bad Fans contributing to DOWN status: (0..3) [2]
Bad Fans contributing to MARGINAL status: (0..3) [1]
Out of range Flash contributing to DOWN status: (0..1) [0]
Out of range Flash contributing to MARGINAL status: (0..1) [1]
MarginalPorts contributing to DOWN status: (0..16) [2]
MarginalPorts contributing to MARGINAL status: (0..16) [1]
FaultyPorts contributing to DOWN status: (0
(0..16)
16) [2]
FaultyPorts contributing to MARGINAL status: (0..16) [1]
MissingSFPs contributing to DOWN status: (0..16) [0]
MissingSFPs contributing to MARGINAL status: (0..16) [0]
No change
Revision 1008
4 - 48

B51:admin> portshow 2
portName: Bay1
portHealth: No Fabric Watch License
Port Name
Authentication: None
portDisableReason: None
portCFlags: 0x1
portFlags: 0x20b03
PRESENT ACTIVE F_PORT G_PORT LOGICAL_ONLINE LOGIN NOELP ACCEPT FLOGI
portType: 11.0
POD Port: Port is licensed
portState: 1
Online
Port initialization from right to left and
portPhys: 6
In_Sync
portScn:
32
F_Port
current port type
port generation number:
0
portId:
010100
portIfId:
4302000d
302000
portWwn:
20:01:00:05:1e:04:88:7e
Port WWNs of Device
portWwn of device(s) connected:
10:00:00:00:c9:59:45:9d
Distance: normal
Distance: Normal buffering
portSpeed: N4Gbps
Port speed
LE domain: 0
FC Fastwrite: OFF
Interrupts:
Unknown:
Lli:
Proc_rqrd:
Timed_out:
Rx_flushed:
Tx_unavail:
Free_buffer:
Overrun:
Suspended:
y_err:
Parity
2_parity_err:
CMI_bus_err:
9
0
9
14
0
0
0
0
0
0
0
0
0
Link_failure:
Loss_of_sync:
Loss_of_sig:
Protocol_err:
Invalid_word:
Invalid_crc:
Delim_err:
Address_err:
Lr_in:
Lr_out:
Ols_in:
Ols_out:
0
3
0
0
115
0
0
0
3
0
0
3
Frjt:
Fbsy:
0
0
Port part of other ADs: No
Revision 1008
4 - 49

Footnote 1: On 8 Gbit/sec switches a Brocade-branded 4 Gbit/sec SFP is required

for a port to run at 1 Gbit/sec.
Revision 1008
4 - 50

Speed is displayed as 1G, 2G, 4G, 8G, or AN (when in Auto Speed Negotiation
mode). This value is set by the portcfgspeed command.
Trunk Port is displayed as ON (when port is set for trunking) or for OFF (when
tr nking is disabled on the port) as set b
trunking
by the portcfgtrunkport
t f t
k
t command.
command
Long Distance setting of the port is shown as blank when long distance mode
is L0 (normal) and will display modes depending on the distance mode setting:
LE (<= 10km), L0.5 (<=25km), L1 (<= 50km), L2 (<= 100km), LD (auto), LS
(static). This value is set by the portcfglongdistance command.
VC link init setting of the port is shown as blank when VC link init mode is
off
ff or ON
O when
h VC lilink
k iinitit mode
d iis on. Thi
This value
l iis sett b
by th
the
portcfglongdistance command.
Locked L_Port is displayed as ON when port is locked to L_Port only or when
L_Port lock mode is disabled (and it behaves as a U_Port). This value is set by
the portcfglport command.
Locked G_Port is displayed as ON when port is locked to G_Port only) or blank
when
h G_Port
G P t lock
l k mode
d iis di
disabled
bl d ((and
d it b
behaves
h
as a U_Port).
U P t) This
Thi command
d
is set by the portcfggport command.
Revision 1008
4 - 51

Disabled E_Port is displayed

p y as ON when p
port is not allowed to be an E_Port.
_
This command is set by the portcfgeport command.
Persistent disable is displayed as ON when the port is disabled across
reboots or power cycles or when the port is allowed to function normally. This
mode is set by the portcfgpersistentdisable command.
ISL R_RDY is displayed as ON when the port is set to R_RDY flow control. This
mode is set byy the p
portcfgislmode
g
command.
RSCN Suppressed is displayed as ON when RSCNs have been suppressed on
the port. This mode is set by the portcfg rscnsupr command.
Persistent Disable is displayed as ON when the port has been persistently
disabled. This mode is set by the portcfgpersistentdisable command
NPIV capability mode is displayed as ON when the port is configured to
perform N
N_port
port virtualization
virtualization. This mode is set by the portcfgnpivport
command.
QOS E_Port Displays (..)/OFF when Quality of Service (QoS) is disabled on the
port. Displays ON when QoS is enabled. By default, QoS is enabled by best
effort based on availability of buffers. This value is set by the portcfgqos
command.
Mirror Port Displays ON when Mirror Port is enabled on the port or (..)/OFF
( )/OFF
when disabled. This value is set by the portCfg mirrorport command.
FC Rate Limit Displays ON when ingress rate limit is set on the port or
(..)/OFF when the ingress rate limiting feature is disabled. This value is set by the
portCfgQos --setratelimit command. The default is OFF.
Credit Recovery Displays ON when Credit Recovery is enabled on the port
or (..)/OFF
( )/OFF when disabled
disabled. This value is set by the portCfgCreditRecovery
command. The credit recovery feature is enabled by default, but only ports
configured as long distance ports can utilize this feature.
Revision 1008
4 - 52

The switchshow command can be used to verify the switch is operating correctly and display
information about the switch status.
switchName - The switchs name
switchType - model.motherboard-rev, where the model number is as follows:
1 B
1=Brocade
d 1000 2
2=Brocade
B
d 2800 3
3=Brocade
B
d 2400 4
4=Brocade
B
d 20
20x0
0 5
5=Brocade
B
d 22
22x0
0 9
9=Brocade
B
d 3800
10=Brocade 12000 12=Brocade 3900 16=Brocade 3200 21=Brocade 24000 26=Brocade 3850 27=Brocade
3250 32=Brocade 4100 34=Brocade 200E 38=AP 7420 42=Brocade 48000 44=Brocade 4900 46=Brocade
7500 58=Brocade 5000 64=Brocade 5300 62=Brocade DCX 66=Brocade 5100 71=Brocade 300
switchState - The state of this switch: Online, Offline, Testing, or Faulty
switchMode - The switch mode, Native or InterOp
switchRole - The switch role: Principal, Subordinate, or disabled
switchDomain - The domain ID of this switch: 0 to 31 or 1 to 239
239.
switchID - The 24-bit address of this switch's embedded port: hex fffc00 to fffcef.
switchWwn - The World Wide Name of this switch
switchBeacon - Indicates if the beacon is turned on or not
Zoning - zoning status
Port Number - Each line shows the port number: 0 to 23, the GBIC type, the port state and a comment field
Port module type - The GBIC/SFP or other type follows the port number.
The four types include (--= none; sw=short wave; lw long wave; cu copper; id - intelligent)
P
Port
speed
d - The
Th speed
d off the
h port (1G
(1G, 2G,
2G 4G,
4G 8G,
8G N1,
N1 N2,
N2 N4,
N4 N8,
N8 AN)
Long distance level - L0 (default), LE, LD, LS
Port state - The possible port states include:
No_Card - no card present in this switch slot
No_Module - no SFP module in this port
No_Light - the module is not receiving light
No_Sync - the module is receiving light but is out of sync
In_Sync - the module is receiving light and is in sync (copper displays Sync, fiber, Online)
Laser_Flt - the module is signaling a laser fault (defective GBIC)
Port_Flt - the port has been marked faulty (defective GBIC, cable, or device)
Di
Diag_Flt
Flt - the
th portt ffailed
il d diagnostics
di
ti (defective
(d f ti G_Port
G P t or FL_Port
FL P t card
d or motherboard)
th b d)
Online - the port is up and running
Lock_Ref - the port is locking to the reference signal
Testing - running diagnostics
Comment field - Some possible comments include: Disabled, Loopback
Revision 1008
4 - 53

Port Speed and Serial Port LEDs

Location
of LED
Purpose
of LED
Below serial port
Indicates switch
power
Above each port

on right
Indicates port
speed
Color
of LED
Status of Hardware
Recommended Action
No light
Boot not complete or failed; switch

may be off
Verify boot completed; contact

switch vendor
Steady
g
green
Switch on and boot completed
None
Slow
green
One or more ports failed POST
Check error log
No light
POST is running, or port is Tx/Rx at

1 Gbit/sec
None
Steady
green
Port is Tx/Rx at 2 Gbit/sec
None
Steady
amber
Port is Tx/Rx at 4 Gbit/sec
None
Port Speed LED for 3900/12000: upper LED
Revision 1008
4 - 54

Revision 1008
4 - 55

An example of some of the configuration parameters:

B51:admin> switchdisable; configure
Configure...
Fabric parameters (yes, y, no, n): [no] y
Domain: (1..239) [1]
R_A_TOV: (4000..120000) [10000]
E_D_TOV: (1000..5000) [2000]
WAN_TOV: (0..30000) [0]
MAX_HOPS: (7..19) [7]
Data field size: (256..2112) [2112]
Sequence Level Switching: (0..1) [0]
Disable Device Probing: (0..1) [0]
Suppress Class F Traffic: (0..1) [0]
Per-frame Route Priority: (0..1) [0]
Long Distance Fabric: (0..1) [0]
BB credit: (1..27) [16]
Revision 1008
4 - 56

Other fabric.ops parameters:

fabric.ops.mode.noClassF: Class F frames will not be used for inter-switch
communications - Class 2 is used. Toggle using Suppress Class F Traffic under
Fabric Parameters
Parameters.
fabric.ops.mode.sync: Used to prevent time out delays in remote Fabrics.
fabric.ops.mode.useCsCtl: Type configure then yes to Fabric parameters and
toggle Per-frame Route Priority: (0..1) [0] to change fabric.ops.mode.useCsCtl.
Creates additional Virtual Channel ID for per-frame based prioritization, using existing
VCs plus frame header information.
p
The output
p relates to Virtual Channel ((vc)) settings
g
fabric.ops.mode.vcEncode:
for establishing communication priority over ISLs and are configurable only when VC
Encoded Address Mode is set. Like all fabric.ops parameters, they must be the
same on all fabric switches.
Other fabric.ops parameters are explained in the Fabric OS Reference Guide.
Fabric operating mode parameters include:
Disable Device Probing fabric.ops.mode.fcpProbeDisable
Isolated
I l t d Operation
O
ti fabric.ops.mode.isolate
i
i
Long Distance Fabric fabric.ops.mode.longDistance
Suppress Class F Traffic fabric.ops.mode.noClassF
Sequence Level Switching fabric.ops.mode.tachyonCompat
Unicast-only Operation fabric.ops.mode.unicastOnly
Per-frame Route Priority fabric.ops.mode.useCsCtl
Revision 1008
4 - 57

The fastboot command reboots the Control Processor (CP) and bypasses the
Power On Self Tests (POST).
Footnote 1: The reboot command reboots the Control Processor (CP) and
incl des POST unless
includes
nless diagdisablepost
di di bl
t is configured.
config red
The hareboot command will reboot the Control Processor (CP). This command is
useful to reboot the standby CP blade in a director or backbone.
Revision 1008
4 - 58

Revision 1008
4 - 59

Revision 1008
4 - 60

Revision 1008
4 - 61

Features denoted with * are implemented but not officially qualified/supported nor
are they disabled.
Footnote 1: Once the supported switch has Fabric OS v6.0.0 loaded, all subsequent
upgrades
d will
ill supportt H
Hott C
Code
d L
Loads.
d H
Hott C
Code
d L
Loads
d ffrom F
Fabric
b i OS v5.2.1_NI
5 2 1 NI
to v6.0.0 are also supported on Brocade 5000s.
Footnote 2: SCC policies are only supported in conjunction with L2 Fabric Binding
support.
Footnote 3: Fabric OS v5.3 introduced an E_Port authentication called Fabric
Element Authentication; authentication can be configured for both E_Ports and/or
F P t (only
F_Ports
( l supported
t d in
i fabrics
f b i with
ith allll B
B-Series
S i switches).
it h )
Footnote 4: Supported on ISLs between B-Series switches but ISLs between BSeries and M-Series will continue to use R_RDY with no frame level trunking.
Footnote 5: FICON features are supported on qualified B-Series hardware
platforms.
Recall:
Mode 0 is Fabric OS Brocade Native Mode
Mode 2 is Fabric OS McDATA Fabric Mode
Mode 3 is Fabric OS McDATA Open Fabric Mode
MIHPTO stands for Missing Interrupt Handler Primary Timeout.
Revision 1008
4 - 62

Footnote 1: Switch requires Fabric OS v6.1+.

Footnote 2: The Brocade 4100 can load Fabric OS v6.0, but does not
support interopmode 2 or 3.
Revision 1008
4 - 63

Revision 1008
4 - 64

Revision 1008
4 - 65

This page intentionally left blank.
Revision 1008
4 - 66

Revision 1008
Zoning
5-1

Revision 1008
Zoning
5-2

Zoning
A zone is a specified group of fabric-connected devices, also called zone

members. Any device, or zone member, connected to the fabric can be included in
one or more zones. Devices can communicate only with devices that are in the
same zone
zone.
After zoning has been enabled, if a device is not explicitly defined in a zone that
device is considered not to exist. In the example above both Server 4 and Loop 2
are not defined. When Server 4 queries the fabric to discover what devices it can
see, Zoning rejects the request because it is not defined in any zone. Likewise,
when the Servers in the Red, Blue and Green zones query the fabric, none of them
will see the disk in Loop
p 2 because it is not defined in any
y zone. The device will be
isolated and will be inaccessible by other devices in the fabric. Devices that attach
to the fabric need to be added to a new or existing zone before their ability to
communicate is enabled.
After the zone members are grouped into zones, zones are grouped into a zone
configuration and the zone configuration can then be enabled. When enabled, the
zone configuration is distributed to all switches in the fabric and an RSCN is
delivered by each switch to its local nodes that are effected by changes in the
enabled zone configuration.
The Fabric OS Administrators Guide describes zoning concepts in more detail.
Revision 1008
5-3

Zoning
Create a detailed switch diagram of the fabric showing ISL connectivity. This will
help account for every switch in the fabric and the E_Ports that are in use. Expand
each switch diagram to show every port (F_Port, FL_Port). Switch ports that are not
in use should remain disabled with a portcfgpersistentdisable command.
command
Define a naming convention to help identify and reference devices in the fabric.
Naming conventions can also be used when creating zones and zone
configurations.
The zoning syntax when creating a zoning set ultimately defines what zoning
scheme will be enforced as the frame is delivered to the destination port. More
i f
information
ti on thi
this will
ill ffollow.
ll
Analyze the zones to ensure that all nodes are members of the correct zone(s).
When the aliases have been added to zones and the zones are added to the zone
configuration, enable the zone configuration and test from the host that each target
can be accessed. For fabrics with multiple zones enabled, it is generally best to
configure one zone at a time and then test it with the Zone Analyzer available in
Web Tools
Tools. If you create all the zones without testing each zone as it is created
created, it is
difficult to debug. After the first zone is set up in the fabric, the user may plug in
devices and then test the connections to confirm that everything is functioning
properly.
Revision 1008
5-4

Zoning
Member:
Alias is given a name, e.g. Server_1, Disk_Array_2.
In EFCM, these are called nicknames.
Physical Fabric port number or area number.
Node World Wide Name - Obtained using nsshow or switchshow.
Port World Wide Name Obtained using nsshow or portloginshow.
64 characters maximum: A-Z, a-z, 0-9 and the _ are allowed.
Zone:
Is given a name, e.g. Red_Zone.
Contains two or more members and uses a ; as a separator.
The same member can be in multiple zones.
Zone definition is persistent; it remains until deleted or changed by an
administrator.
Configuration:
Is given a name, e.g. Production_Cfg.
Is one or more zones.
Configuration may be disabled or one configuration may be in effect from any
switch in the fabric.
An administrator selects which configuration is currently enabled
enabled.
A configuration is saved when enabled and then distributed to the remaining
switches in the fabric where it is enabled and saved.
Revision 1008
5-5

Zoning
The following commands are used to create/modify the defined zone configuration:
*create
Creates a new alias, zone, or configuration
*delete
Deletes the entire alias, zone, or configuration
*add
Adds a member to an existing alias, zone, or configuration
*remove
Removes one or more members from an existing alias, zone, or

configuration
*show
Displays alias, zone, and/or configuration information
Web Tools and Fabric Manager, EFCM, provide a GUI simplifies the administration
of zoning.
oning
Revision 1008
5-6

Zoning
Zone objects identified by port number or area number are specified as a pair of
decimal numbers d,area, where d is the Domain ID of the switch and area is
the area number on that switch. If the switch is replaced that is referenced by
<domain port> or <domain
<domain,
<domain, area>
area>, the new switch should be configured with the
predecessors Domain ID. If a Domain ID is changed to a new value, all zones that
referenced the predecessors domain number will need to be updated with the
successors value.
Worldwide Names are specified as a 16 digit hexadecimal number separated by
colons, for example 10:00:00:90:69:00:00:8a. When node name is used to specify
a zone object,
j
all ports on that device are in the zone. When port name is used to
specify a zone object, only that single port is in the zone.
Zone aliases simplify repetitive entry of zone objects such as port numbers or
PWWN. For example, the name Eng could be used as an alias for
10:00:00:80:33:3f:aa:11. An alias is a name assigned to a device or group of
devices. By creating an alias, you can assign a familiar name to a device, or you
can group multiple devices into a single name. This can simplify cumbersome
entries
t i and
d it allows
ll
an iintuitive
t iti naming
i structure
t t
such
h as using
i NT
NT_Storage
St
tto
define all NT storage ports in the fabric.
When a zoned host received the list of network targets (referenced by
<domain,port> or PWWN or NWWN) from the Name Server, the host will send a
PLOGI request to the destination addresses. If the PLOGI frame is allowed to pass
at the egress port and the target at the destination address replies with an accept to
the PLOGI request
request, the Brocade switch and Zoning has completed its responsibility
of networking the source and destination. Limiting the amount of LUNs and target
IDs that the host can access when the SCSI inquiry command is sent, is the
responsibility by the storage provisioning software located at the storage device.
Revision 1008
5-7

Zoning
This example should not be viewed as a best practice but rather an example that
shows how a domain, port and WWN would be coded. The CLI is used to illustrate
the zoning structure. Once this is understood, the Web Tools GUI would be a better
tool to use
use.
Zoning has a very systematic yet simple approach for implementation.
Zoning requires prior planning. What are your goals? How will you achieve them?
Create members using aliases
Create zones using alias members
Create a configuration using zones
Enable the zone configuration throughout the fabric
Revision 1008
5-8

Zoning
The command cfgshow displays the defined configuration and since zoning has
not been enabled, there is no effective configuration. Zoning is fabric-wide, thus any
switch can be used to display the current zoning configurations.
The defined
Th
d fi d configuration
fi
ti is
i the
th Zoning
Z i Database
D t b
and
d contains
t i allll zone objects
bj t th
thatt
have been created. It is possible to have several zone configurations but only one
can be enabled.
Revision 1008
5-9

Zoning
Note: cfgenable also saves the defined configuration and the name of the
effective configuration to flash memory.
Revision 1008
5 - 10

Zoning
Since zoning is now enabled, the configuration in effect is displayed.
Revision 1008
5 - 11

Zoning
A zone configuration is a group of zones that are enforced whenever that zone
configuration is enabled. A zone can be included in more than one zone
configuration.
To define a zone configuration, specify the list of zones to be included and assign a
zone configuration name. Zoning may be disabled at any time. When a zone
configuration is in effect, all zones that are members of that configuration are in
effect.
Defined configuration: The complete set of all zone objects that have been
defined in the fabric.
Effective configuration: A single zone configuration that is currently in effect.
The effective configuration is built when an administrator enables a specified
zone configuration. This configuration is compiled by checking for undefined
zone names, or zone alias names, or other issues.
Saved configuration: A copy of the defined configuration plus the name of the
effective
ff
configuration
f
which is saved in flash
f
memory by the cfgsave
command. There may be differences between the saved configuration and the
defined configuration if the system administrator has modified any of the zone
definitions and has not saved them.
Revision 1008
5 - 12

Zoning
Use the cfgenable command to enable a zone configuration. The specified zone
configuration is built by checking for undefined zone names, zone alias names, or
other inconsistencies by expanding zone aliases, removing duplicate entries, and
then installing the current configuration
configuration.
If the build fails, the previous state is preserved (zoning remains disabled, or the
previous configuration remains in effect). If the build succeeds, the new
configuration replaces the previous configuration.
Revision 1008
5 - 13

Zoning
Use the cfgdisable command to disable the current zone configuration. The
fabric returns to non-zoning mode, in which all devices see each other.
This command clears and commits the current zoning transaction buffer to both
volatile
l til and
d fl
flash
h memory. If a ttransaction
ti iis open on a diff
differentt switch
it h iin th
the ffabric
bi
when this command is run, the transaction on the other switch is automatically
aborted. A message is displayed on the other switches to indicate the aborting of
the transaction.
Revision 1008
5 - 14

Zoning
Use the cfgclear command to clear all zone information in the defined
configuration. All defined zone objects are deleted. If an attempt is made to clear the
defined configuration while a zone configuration is enabled, you are warned to first
disable the enabled zone configuration.
configuration
After using the cfgclear command, use the cfgsave command to commit the
defined and effective configuration to flash memory for all the switches in the fabric.
Revision 1008
5 - 15

Zoning
Use the cfgsave command to save the current zone configuration. The defined
configuration and the name of the enabled configuration are written to flash memory
in all switches in the fabric. This allows changes to be made to the defined
configuration without immediately enabling of them
them.
The saved configuration is automatically reloaded by the switch on power on and, if
a configuration was in effect at the time it was saved, the same configuration is
reinstalled with an automatic cfgenable command.
Because the saved configuration is reloaded at power on, only valid configurations
are saved. The cfgsave command verifies that the enabled configuration is valid
by performing the same tests as cfgenable. If the tests fail, an error is displayed
and the configuration is not saved. Tests might fail if a configuration has been
modified since the last cfgenable.
This command ends and commits the current transaction. If a transaction is open on
a different switch in the fabric when this command is run, the transaction on the
other switch is automatically aborted. A message is displayed on the other switches
to indicate the aborting of the transaction.
If the defined configuration is larger than the supported maximum zoning database
size, the following message is issued: Commit zone DB larger than
supported - <zone db size> greater than <max zone db size>
Note: A cfgsave does not make any changes to the effective configuration. A
cfgenable command is still needed to enable any changes made in the defined
configuration.
Revision 1008
5 - 16

Zoning
The switch with the lowest max zone database size, typically the switch with the
lowest version of Fabric OS, will determine the maximum zoning database size in a
fabric .
Use th
U
the cfgsize
f i
command
d tto display
di l th
the size
i d
details
t il off th
the zone d
database.
t b
Th
The
size details include the Zone DB maximum size, the committed size, and the
transaction size. All sizes are in bytes.
Zone DB max size is the upper limit for the defined configuration, determined
by the amount of flash memory available for storing the defined configuration.
Committed size is the size of the defined configuration currently stored in
fl h memory.
flash
Transaction size is the size of the uncommitted defined configuration. This
value will be nonzero if the defined configuration is being modified, otherwise it
is 0.
sw300:admin> cfgsize
Zone DB max size - 1045274 bytes
Available Zone DB size - 1044056 bytes
committed - 206
transaction - 0
Revision 1008
5 - 17

Zoning
If a switch attempts to join a fabric that has a zone database size greater than the
supported maximum size of the switch, a segmentation error will occur (the request to
join the fabric will be rejected) preventing the switch from joining the fabric.
FOS levels
l
l and
d IInterop
t
modes
d also
l effect
ff t the
th maximum
i
zoning
i d
database
t b
size.
i
Example: if you have a four switch fabric and three of the switches are running FOS
v5.2.x and one is running FOS v5.1.x, then the maximum zoning database is 256 KB.
Revision 1008
5 - 18

Zoning
sw300:admin> cfgshow "*"

cfg: USA_cfg Red_zone; White_zone; Blue_zone
sw300:admin> zoneobjectcopy "USA_cfg", "UK_cfg"
sw300:admin> cfgshow "*"
cfg: UK_cfg Red_zone; White_zone; Blue_zone
sw300:admin> cfgshow
Defined configuration:
zone: Blue_zone 1,0; 1,1
zone: Red_zone 1,2; 1,3
zone: White_zone 1,4; 1,5
sw300:admin> zoneobjectexpunge Blue_zone"
sw300:admin> cfgshow
Defined configuration:
cfg: USA_cfg Red_zone; White_zone
zone: Red_zone 1,2; 1,3
zone: White_zone 1,4; 1,5
Revision 1008
5 - 19

Zoning
This command is a duplicate of the nsshow command with the added feature of
displaying the defined configuration aliases that the device belongs to.
The message There is no entry in the Local Name Server is
di l
displayed
d if there
th
is
i no iinformation
f
ti iin thi
this switch,
it h b
butt th
there still
till may b
be d
devices
i
connected to other switches in the fabric. The command nsallshow shows
information from all switches.
Revision 1008
5 - 20

Zoning
sw300:admin> nodefind 21:00:00:20:37:15:15:a2

Local:
Type Pid
SCR
COS
PortName
NodeName
NL
0110e8;
3;21:00:00:20:37:15:15:a2;20:00:00:20:37:15:15:a2; 0
FC4s: FCP [SEAGATE ST336605FC
0003]
Fabric Port Name: 20:10:00:05:1e:04:88:7e

Permanent Port Name:
Name 21:00:00:20:37:15:15:a2
21 00 00 20 37 15 15 a2
Device type: Physical Target
Port Index: 16
Share Area: No
Redirect: No
Aliases: Disk_1
Revision 1008
5 - 21

Zoning
You can also use the WWN:

sw300:admin> nszonemember 20:00:00:00:c9:59:45:9d
2 local zoned members:
Type Pid
COS
PortName
NodeName
SCR
N
010100;
2,3;10:00:00:00:c9:59:45:9d;20:00:00:00:c9:59:45:9d; 3
FC4s: FCP
NodeSymb: [33] "Emulex LP11000 FV2.10A10 DV8.0.16"
P
Permanent
t P
Port
t N
Name: 10
10:00:00:00:c9:59:45:9d
00 00 00 9 59 45 9d
Device type: Physical Initiator
Port Index: 1
Share Area: No
Redirect: No
NL
0110e8;
3;21:00:00:20:37:15:15:a2;20:00:00:20:37:15:15:a2; 0
FC4s: FCP [SEAGATE ST336605FC
0003]
Permanent Port Name: 21:00:00:20:37:15:15:a2
Device type: Physical Target
Port Index: 16
Share Area: No
Redirect: No
No remote zoned members
Revision 1008
5 - 22

Zoning
Some useful options with nszonemember:

-a: Displays each local devices online zoned data, including PID and zone alias.
-u: Displays all unzoned devices in the entire fabric.
Revision 1008
5 - 23

Zoning
sw300:admin> cfgtransshow
There is no outstanding zone transactions
sw300:admin> cfgclear
Do you really want to clear all configurations? (yes, y,
no, n): [no] y
Clearing All zoning configurations...
sw300:admin> cfgtransshow
Current transaction token is 271010736
It is abortable
sw300:admin> cfgtransabort
sw300:admin> cfgactvshow
Effective configuration:
cfg:
Cfg_EngMkt
zone: Zone_Eng
Z
E
1
1,0;
0
zone:
Revision 1008
Zone_Mkt 1,16;
21:00:00:20:37:87:23:e2;
21
00 00 20 37 87 23 2
21:00:00:20:37:87:48:e7
21:00:00:20:37:87:e5:20;
21:00:00:20:37:87:49:29;
21:00:00:20:37:87:20:c5
5 - 24

Zoning
The new default zone feature can enable or disable device access within a fabric.
Default zones are based on the FC-GS standard, but are not supported when the
switch or Director is in interop mode.
The defzone
d f
-allaccess
ll
is the defa
default
lt beca
because
se it matches ho
how zoning
oning worked
orked
prior to Fabric OS v5.1.0.
Revision 1008
5 - 25

Zoning
The new defzone command configures a default zone configuration and displays
the current configuration. The command has no optional parameters, and takes one
of three required arguments:
--allaccess:
ll
Create a defa
default
lt zone
one that enables all de
device-to-device
ice to de ice access
within the fabric. This is the default behavior in Fabric OS v5.1, and matches
the default behavior in a non-zoned fabric.
--noaccess: Create a default zone that disables all device-to-device access
within the fabric.
--show: Display the current default zone.
Names beginning with d__efault__ are reserved for default zoning use (note: two
underscore characters are used in each instance.)
Note: The setting of the defzone command is stored in the zoning transaction
buffer. Normally, a cfgsave is used to commit the zoning transaction to the entire
fabric. A cfgenable or cfgdisable will do the commit since each command does
an implied cfgsave. Because the setting is stored in the zoning transaction buffer,
a cfgtransabort could
ld b
be used
d tto abort
b t th
the defzone command.
d
Revision 1008
5 - 26

Revision 1008
Zoning
5 - 27

Zoning
From a switch running earlier versions of Fabric OS, the zone* commands cannot
manage the default zone, and the cfg* commands cannot manage the default
zone configuration. You may delete the default zone configuration.
For example,
e ample attempting to disable d__efault__Cfg
d
f lt Cf on a Fabric OS v5.0.1
5 0 1 sswitch
itch
results in the following error message:
RCSRCA_SFC_REJECTED
Sfc Was Rejected: Remote Switch Unable To Process.
Revision 1008
5 - 28

Zoning
When zoning is not enabled and the default zone is set to no access, the
cfgshow output
t t for
f the
th v6.1.0
6 1 0 switch
it h will
ill b
be diff
differentt ffrom a switch
it h running
i a
release prior to v5.1. See slide above.
Use the defzone --show command to determine which mode the default zone
is set to (Access or No Access).
sw5100:admin> defzone --show
Default Zone Access Mode
committed - No Access
transaction - No Transaction
sw5100:admin> switchshow
switchName:
sw5100
switchType:
66.0
switchState:
Online
switchMode:
Native
switchRole:
Subordinate
switchDomain:
2
switchId:
fffc02
switchWwn:
10:00:00:05:1e:03:a6:6d
zoning:
ON (No Access)
switchBeacon:
OFF
<truncated output>
Revision 1008
5 - 29

Zoning
This screen allows for the creation and modification of aliases, zones and
configuration.
Revision 1008
5 - 30

Zoning
Devices that are Session enforced cause any PLOGIs to the device to be rejected.
Devices that are Hardware enforced cause any frames that do not comply with the
effective zone configuration to be rejected. This blocking is performed at the
t
transmit
it side
id off the
th portt where
h
th
the d
destination
ti ti d
device
i iis llocated.
t d Thi
This is
i th
the hi
highest
h t
level of protection for a device.
Footnote 1: A bad citizen is best explained by defining good citizens. Good citizens
are defined as fabric devices that support RSCNs, query a name server when they
receive RSCNs, and only communicate with devices that the name server gives
them when they query. Bad citizens do not do one or more of these things.
The d
Th
decision
i i ffor what
h t enforcement
f
tad
device
i receives
i
iis b
based
d on h
how th
the members
b
in a given zone are defined. The table on the next slide describes this process.
Revision 1008
5 - 31

Zoning
Hardware Enforced Zoning:

Hardware Enforced zoning is used by zones with all members defined by their
<domain ID,, port>
p
or all members defined byy their WWN. This the strongest
g
form
of enforcement and will block all frames that compromise the zone from a device
that is not a member of a zone, such as a bad citizen. Destination ASIC checks
SID on every frame against CAM table entries. Overlapping zones (zone members
that appear in two or more zones) are permitted and hardware enforcement will
continue as long as the overlapping zones have either all WWNs or <domain ID,
port> entries.
Using all WWNs in a zone allows for the node to attach to any port in the fabric
and have hardware enforcement. Using all <domain, port>/<domain, area>
members restricts the movement of devices in the fabric until a zone update is
made.
Session Enforced Zoning:
A session enforced zone is a zoning protection that guarantees that only members
of the zone can complete PLOGI/ADISC/PDISC which prevents any unauthorized
access by devices that are not a member of the zone. Enforcement to a zone with
WWN members and <domain, port> will change from hardware to session
enforcement. The ASIC will perform authentication using the name server to
compare the SID/DID in the primitive commands with the current zone
configuration. If the current zone configuration does not permit the devices to
communicate, the switch issues a reject to the SID, effectively blocking
communications.
Revision 1008
5 - 32

Zoning
Blue Zone: This zone is Hardware enforced because all devices have been
specified by WWN.
Green Zone: This zone is Hardware enforced because all devices have been
specified
ifi d b
by P
Port.
t
Red Zone: This zone is Session enforced because a mix of port and WWN have
been specified in the zone.
Orange Zone: This zone is also Session enforced because of a mix of port and
WWN in the same zone.
Note: The Red and Purple Zones also illustrate that the type
y of device (initiator
(
vs.
target) has no bearing on the type of enforcement.
Revision 1008
5 - 33

Zoning
The diagram above shows the results of Hardware and Session enforced
overlapping zones.
The Blue zone is defined with all WWNs (WWN1 and WWN2) and meets the rules
f Hardware
for
H d
enforcement.
f
t The
Th Purple
P l zone is
i d
defined
fi d with
ith a mix
i off portt and
d
WWNs and meets the rules for Session enforcement.
The target device WWN1 is defined in both zones. When a device is defined in
overlapping zones, where one is Hardware enforced and the other is Session
enforced, the device will become Session enforced in all zones. What is important to
note is the host (WWN2) is still Hardware enforced, even though the target device
(WWN1) is now Session enforced
enforced. Under these conditions
conditions, zoning enforcement is
determined at the device level, not the zone level.
Revision 1008
5 - 34

Revision 1008
Zoning
5 - 35

Zoning
These implementation considerations focus on creating zones to achieve Hardware

enforcement and identify when zoning changes are needed.
Note: Fibre Channel Fast Write zones must use WWN zoning.
Revision 1008
5 - 36

Zoning
Footnote 1: Zoning by single Host Bus Adapter (HBA) most closely recreates the
original SCSI bus. Each zone created has only one HBA (initiator) in the zone and
all the target nodes are members of that zone. Longer than normal boot time can be
caused when single initiator to multiple target or multiple initiator to multiple target
zones are used.
Footnote 2: Defining zone members with either all port numbers or all WWNs
provides Hardware enforcement.
Setting the default zone to no access when the fabric is first built allows devices to
connect to the fabric, do their FLOGI and Name Server update but not access any
other device connected to the fabric.
fabric This permits the physical connection to be
done in one phase and the enabling of a zone configuration to allow access to be
done in another phase.
Monitor the zone database sizing as new switches are added to the fabric. Newer
switches will tend to have a larger maximum size but the fabric may not be able to
take advantage of it due to an older FOS running on an existing switch with a lower
maximum.
With zoning enabled, check the servers to verify they have access to the desired
target devices. Also, use the nszonemember command and SAN Health as tools to
discover devices that are online but not defined in a zone, etc.
Revision 1008
5 - 37

Revision 1008
Zoning
5 - 38

Zoning
A new switch is one that has not previously been connected to a fabric. Before
connecting the new switch, check to see if any zoning data exists with the cfgshow
command. If it exists, use the cfgdisable, cfgclear, and cfgsave commands
to sanitize itit.
When a new switch is connected to a zoned fabric, all zone configuration data is
immediately copied from the zoned fabric into the new switch. If a zone
configuration is enabled in the fabric, then the same configuration becomes enabled
in the new switch. After this operation, the cfgshow command displays the same
output on all switches in the fabric, including the new switch.
Revision 1008
5 - 39

Zoning
The cfgshow command displays the status of the defined and effective
configurations on a new switch.
Revision 1008
5 - 40

Zoning
The defined and effective configurations from the existing fabric are propagated to
the new switch.
Revision 1008
5 - 41

Zoning
If the zoning changes are not done correctly, it is possible to have the merging of
the fabrics fail due to a segmentation error.
The table above shows the three possible mismatches that would cause this
condition.
diti
Th
The actual
t l error would
ld b
be an FABR
FABR-1001
1001 error.
Revision 1008
5 - 42

Zoning
Use Fabric Manager to invoke the Fabric Merge Check. This function allows the
comparing of two fabrics and their settings that could cause a fabric segmentation
error. It is best to check before you connect.
Revision 1008
5 - 43

Zoning
The Fabric Merge Check is under the Tools pull-down menu.
Revision 1008
5 - 44

Zoning
In preparation for the merge check, two fabrics will be selected. In the example
above, fabric-sw51 and fabric-RSL1_BRCD47 have been selected for a check.
Once the fabrics have been selected you can select the Check button to extract
the elements from each fabric for comparison
comparison.
Revision 1008
5 - 45

Zoning
At the end of the process, a Merge Check Results pop-up window will be
displayed. To validate all of the compared results, you can select the up and down
buttons to the right to display any identified mismatches.
Revision 1008
5 - 46

Zoning
SAN Health is a very good tool for cleaning up a zoning database.
Revision 1008
5 - 47

Revision 1008
Zoning
5 - 48

Revision 1008
Zoning
5 - 49

Revision 1008
Zoning
5 - 50

Revision 1008
Fibre Channel Routing
6-1

Revision 1008
6-2

The SAN A is comprised of many interconnected switches. Each switch has a

unique domain ID within the fabric. Hosts (represented by the computer icon) are
able to connect to storage (represented by the disk icon) across switches within
SAN A
A.
FC-FC Routing is performed by a Fibre Channel router, to pass frames from a
source device in one fabric toward a destination device in another fabric without
merging the fabrics. FC-FC Routing will be described in another course.
Revision 1008
6-3

Revision 1008
6-4

Revision 1008
6-5

Ports in the example depicted on this slide are numbered as depicted here:
These are the functions of a Principal Switch:
Ensure unique domain IDs throughout the fabric
Synchronize time throughout the fabric
Port 1
Port 0
Port 4
Port 5
Port 7
FSPF uses several frames to perform its functions. Since it may run before fabric routing is set up,
FSPF does not use the routing tables to propagate the frames, but floods the frames throughout the
fabric hop-by-hop.
p y p At the beginning,
g
g, frames are flooded on all the Inter-Switch Links ((ISLs);
); as the
protocol progresses, it builds a spanning tree rooted on the Principal Switch. Frames are then sent
only on the ISLs that belong to the spanning tree. These ISLs are called Principal ISLs.
Where there are multiple ISLs between switches, the first ISL to respond to connection requests
becomes the Principal ISL. Only one ISL from each switch will be used as the Principal ISL.
For example, there is an ISL between Domain 3 and Domain 6, but it was not chosen as part of the
principal ISL paths. This ISL is still used for frames routed through the fabric. Issuing a switchshow
command from Domain 3 would indicate the following summarized output:
Port 1
E_Port
(Upstream)
Port 3
E_Port
Port 6
E_Port
(Downstream)
Port 7
E_Port
(Downstream)
Upstream means going out that E_Port is going toward the Principal Switch. Downstream means
going out that E_Port is going away from the Principal Switch.
Revision 1008
6-6

Fabric OS v4.1+ gives the SAN administrator the ability to set which switch will be
the principal with the fabricprincipal command. However, if
fabricprincipal is set on more than one switch, the one with the lowest WWN
will become the Principal.
fabricprincipal [-fhq][mode]
Use this command to set Principal Switch selection mode for the switch.
The implementation of the fabricprincipal command is based solely on
mechanisms specified in the Fibre Channel standards. These mechanisms provide
a preference for a switch requesting to be the Principal Switch in a fabric, but they
do not provide an absolute guarantee that a switch requesting to be the Principal
Switch will actually achieve this status.
When dealing with larger fabrics, the selection of the Principal Switch is less
deterministic. In these cases, to help ensure that the desired switch is selected as
the Principal Switch, a small selection of switches should be connected together
first, followed by the addition of the rest of the fabric.
Operands
This command has the following operands:
-f: force a fabric rebuild. This option is required when enabling Principal Switch
mode. This option is not valid with the disabling Principal Switch
mode.
-h: display command usage summary.
-q: display the current mode state.
mode: Specify 1 to enable, or 0 to disable Principal Switch mode (the mode
activates when the fabric rebuilds). This operand is optional.
Revision 1008
6-7

Inter-Switch Links (ISLs) can seamlessly extend a single switch fabric, and they can
provide resiliency and failover. When interconnecting two switches, a normal cable
is used. When an ISL is used to connect a switch, an E_Port and buffer allocation
are configured for inter-switch communication depending on the long distance
setting.
Locality:
Reducing transmission latency by physically placing hosts and storage that
communicate frequently such that no routing (same switch) or minimal routing
(adjacent switches) is required
Easier
E i tto manage iin smallll SANs
SAN than
th llarge or growing
i SAN
SANs
Easier to manage in Directors
Revision 1008
6-8

In the example above:

Six servers have negotiated a total of 40 Gb/s and can share the one 8 Gb/s ISL.
This becomes a 40:8 oversubscription ratio which is placed on that ISL. This value
can be displayed with the TopologyShow command on SW1 switch.
Several reasons why over-subscription is a valid topology are due to certain
considerations:
1. The servers or targets do not request and respond at the same time.
2. When they do communicate, they generally do not transmit at full
bandwidth.
3 Most important
3.
important, Brocade implements a frame prioritization scheme into the
hardware, known as Virtual Channels, that acts like a traffic manager. This
will allow all frame types to get a chance to send the frame through the ISL.
We will discuss VCs in the next few slides.
Depending on the inter-switch communications load, the ISL over-subscription ratio
can vary. ISL over-subscription is the ratio of source device ports to E_Ports.
Calculated as a fraction, and displayed as a percentage of demand placed on the
ISL.
ISL
Thirty-two hosts negotiating 4 Gb/s per port and one 8 Gb/s E_Port is 15:1
The over-subscription ratio can vary, depending on the communications on the
switch.
When the performance expectations are demanding, then Brocade
recommends:
3:1 over-subscription ratio
7:1 and even 15:1 are common
Revision 1008
6-9

Footnote 1:
Applications used to help provide Quality of Service on ISLs include:
QoS Zoning
Ingress Rate Limiting
These application will be discussed in other modules.
Revision 1008
6 - 10

An ISL is divided into 8 virtual channels (0 - 7) which are in 3 groups, defined by

priority:
P1 P3
P1, is the highest within VC 0 and VC 1
P2 has VC 2,3,4,5
P3 has VC 6 and 7
The four data VC channels are used to multiplex data based upon the destination
port number on a switch. Using the lowest two bits, port numbers that have a binary
value of 0000 are sent across VC2
VC2. Port numbers that have a binary value of 0001
are sent across VC3. Port numbers that have a binary value of 0010 are sent
across VC4. Port numbers that have a binary value of 0011 are sent across VC5.
Revision 1008
6 - 11

An ISL is divided into 16 virtual channels (0 - 15) which are in 5 groups, defined by
priority:
P1 P5
P1 is the highest within VC 0 and VC 1
P2 has VC 10,11,12,13,14
P3 has VC 2,3,4,5
P4 has VC 8,9
P5 has VC 6 and 7
The seven data VC channels (VC8-14) are used to multiplex data frames based
upon QoS Zones.
High is identified as a QOSH_* zone name
Low is identified as a QOSL_* zone name
When QoS zones are not configured devices will use VC2-5 for data transfers.
Revision 1008
6 - 12

The FC-SW-2 standard for Storage Area Networks (SANs) uses an algorithm called Fabric Shortest
Path First (FSPF). FSPF is a link state path selection protocol and directs traffic along the shortest
path between the source and destination, based upon the link cost, and makes it possible to detect
link failures, determine shortest route for traffic, update the routing table, provide fixed routing paths
within
ithi a ffabric,
b i and
d maintains
i t i correctt ordering
d i off fframes. FSPF keeps
k
track
t k off the
th state
t t off the
th links
li k on
all switches in the Fabric and associates a cost with each link. The protocol computes paths from a
switch to all the other switches in the fabric by adding the cost of all links traversed by the path, and
chooses the path that minimizes the costs. This collection of the link states (including costs) of all the
switches in the fabric constitutes the topology database (or link state database). Once established,
FSPF programs the hardware routing tables for all active ports on the switch. FSPF is not involved in
frame switching.
There are two types
yp of p
primary
y routing
gp
protocols in intranet networks,, Distance Vector and Link State:
Distance Vector is based on hop count. This is the number of switches you traverse through
to get from the source domain (switch) to the destination domain (switch).
Link State is based on a metric value based on a cost. The cost could be based on bandwidth.
FSPF makes minimal use of the ISLs bandwidth, leaving virtually all of it available for traffic. In a
stable fabric, a Brocade switch will transmit 64 bytes every 20 seconds in each direction. FSPF
frames have the highest priority in the fabric. This guarantees that a control frame is not delayed by
g decisions occur very
yq
quickly
y during
g convergence.
g
user data and that FSPF routing
FSPF guarantees a routing loop free topology at all times. Why is this important? It is essential for a
fabric to include many physical loops, because without loops there would be no multiple path
between switches, and therefore no redundancy. Without multiple paths, if a links goes down part of
the fabric becomes isolated. FSPF ensures that the topology is loop free and that the frame will never
be forwarded over the same ISL more than once.
Brocade recommends no more than 7 hops between two switches. This limit is not required or
enforced by FSPF. Its purpose is to ensure that a frame will never be delivered to a destination after
E_D_TOV has expired.
Brocade switches support unicast, multicast and broadcast traffic. For unicast traffic, the Brocade
supports both Class 2 and Class 3, whereas broadcast and multicast, as defined in Fibre Channel
standards, are supported in Class 3 only.
The Brocade ASIC does all frame forwarding (unicast, multicast, and broadcast) in hardware.
Revision 1008
6 - 13

Fabric Shortest Path First calculates paths based on the destination domain ID. The Fabric Protocol
must complete domain ID assignments before routing can begin.
ISLs provide the physical pathway when the Source ID (SID) address has a frame destined to a port
on a remote switch Destination ID (DID). When an ISL is attached/removed to a switch, Fabric
Shortest Path First (FSPF) will update the route tables to reflect the addition/deletion of the new
routes.
As each host transmits a frame to the switch, the switch will read the SID and DID in the frame
header. If the domain ID of the destination address is the same as the switch (intra-switch
communications), the frame buffer is copied to the destination port and a credit (R_RDY) is sent to
the host. Brocade only needs to read word zero and word one of the Fibre Channel frame to perform
what is known as cut-through routing. A frame may begin to emerge from the output port before it has
been entirely received by the input port. The entire frame does not need to be buffered in the switch.
If the destination domain ID is different than the source domain ID (inter-switch communications)
communications),
then the switch will consult the FSPF route table to identify which local E_Port will provide the Fabric
Shortest Path First to the remote domain, as discussed on the previous slide.
In the example above, looking at FSPF from Domain 1, all ports set to 8 Gbit/sec:
D1 to D3 (Paths)
Port2 = 500*
Port5 = 500*
Port6 = 1000
D1 to D2 (Paths)
Port2 = 1000
Port5 = 1000
Port6 = 500*
Routes = *
Dynamic Load Sharing can be managed with the LinkCost command. Static routes can be
assigned using the UrouteConfig command or Traffic Isolation application.
Footnote 1: Any ISL set to 1G/s will force a metric cost of 1000.
Revision 1008
6 - 14

Dynamic Load Sharing is a standard in Fibre Channel to share multiple available

routes to a destination domain. If multiple routes exist in the routing table, FSPF will
DLS according to the ratio of bandwidth available on the routes.
The Exchange-based
Exchange based Routing Policy depends on the Fabric OS Dynamic Load Sharing
feature (DLS) for dynamic routing path selection. When using the Exchange-based
Routing Policy, DLS is by default enabled and cannot be disabled. In other words, you
cannot enable or disable DLS when the Exchange-based Routing Policy is in effect.
When the Port-based Policy is in force, you can enable DLS to optimize routing. DLS
recomputes load sharing when a switch boots up, an E_Port/EX-port goes offline and
online, an Nx_Port goes offline.
In the example above
above, looking at FSPF from Domain 1:
Paths and routes are created for both Domain 2 and Domain 3 using ports 2, 5, and 6.
D1 to D3 (Paths)
Port2 = 500*
Port5 = 500*
Port6 = 1000
Routes have a 1:1 DLS ratio
D1 to D2 (Paths)
Port2 = 1000
Port5 = 1000
Port6 = 500*
500
Routes = *
Revision 1008
6 - 15

In a stable fabric, frames are always delivered in order, even when the traffic
between switches is shared among multiple paths. However, when topology
changes occur in the fabric and DLS is on (DLSSet), traffic is rerouted around the
change and some frames could be delivered out of order
change,
order. Most destination devices
tolerate out- of-order delivery, but some do not. By default, out-of-order frame-based
delivery is allowed, (IODreset) to minimize the number of frames dropped.
IODreset = 650ms Convergence time
Enabling in-order delivery (IODset) guarantees that frames are either delivered in
order or dropped. You should only force in-order frame delivery across topology
changes if the fabric contains destination devices that cannot tolerate occasional
out-of-order frame delivery.
IODset = e_d_TOV parameter (2000ms)
This value can be changed by issuing the switchdisable;configure
command. This allows you to increase the value of the e_d_TOV parameter.
Be aware if you change this value, you must do the same to the rest of the
switches as this is considered a fabric.ops
switches,
fabric ops parameter.
parameter
By default IOD is = IODreset or off
You can disable the switches ability to perform this re-routing mechanism by using
DLSreset.
Revision 1008
6 - 16

Revision 1008
6 - 17

Revision 1008
6 - 18

The Link State Database calculates the cost of each link and determines the lowest
cost path within each switch. The input port from the source is assigned to all output
ports toward the destination (Dynamic Route Selection).
Exchanges are allocated via round
round-robin
robin assignment
Chosen routes are used regardless of whether or not other devices in the
fabric go offline or fabric changes occur
Changes in fabric, when Dynamic Load Sharing is enabled (DLSset), causes
FSPF to recalculate the Dynamic distribution of exchanges to the remaining
output ports to continue to distribute devices across equal cost routes.
DLS and In-Order Delivery (IOD) options
DLS cannot be disabled when using Exchange-based Routing. When DLS is
enabled (DLSSet), routes are assigned as devices come online, but if something in
the fabric changes (local E_Port goes up/down, local Fx_Port goes up/down) and
DLS is set, exchanges are re-hashed across the remaining routes.
Revision 1008
6 - 19

Exchange-based Routing Policy

With this policy, the path chosen for an ingress frame is based on:
FC address of the source fabric device (SID) for this frame
FC address of the destination fabric device (DID) for this frame
FC Originator Exchange ID (OXID) for this frame.
This policy allows for optimal utilization of the available paths as I/O traffic between
different (SID, DID, OXID) pairs can use different paths. All frames received on a
ingress port (in port) with the same (SID, DID, OXID) parameters will take the same
path unless there is a fabric event
event.
Just like port-based frame allocation when there are multiple paths to a
destination, the ingress traffic will be distributed across the different paths in
proportion to the bandwidth available on each of the paths. This improves utilization
of the available paths, thus reducing possible congestion on the paths. Every time
there is a change in the network (which changes the available paths), the ingress
traffic can be redistributed across the available p
paths. This is a very
y easy
y and nondisruptive process when exchange-based policies are engaged.
Revision 1008
6 - 20

With Port-based Routing, DLS is an option. With Exchanged-based Routing, DLS

must be enabled.
Revision 1008
6 - 21

Source ID and destination domain are used to allocate routes

Devices are round-robin allocated to available equal cost routes
DLS will round-robin according to aggregate bandwidth assigned to the route
It is possible to have congestion, if too many high I/O requiring devices are
allocated to a single route.
It is possible with round-robin assignment of the in-ports to overload a particular
route. If the aggregate throughput does not exceed the bandwidth of any one ISL,
then this is not an issue.
Revision 1008
6 - 22

Footnote 1: apt stands for advanced performance tuning

The default routing policy (Exchange-based Routing) can be changed to Port-based
Routing Policy. When the switch is enabled, aptpolicy can be used to display the
new currentt policy:
li
Revision 1008
6 - 23

routehelp:
Revision 1008
bcastShow
Print broadcast tree information
disreset
Turn off Dynamic Load Sharing
dlsSet
Turn on Dynamic Load Sharing
dlsShow
Print state of Dynamic Load Sharing
fspfShow
Print FSPF global information
interfaceShow
Print FSPF interface information
iodReset
Turn off In-Order Delivery
iodSet
Turn on In-Order Delivery
iodShow
Print state of In-Order Delivery
linkCost
Set or print the FSPF cost of a link
LSDbShow
Print Link State Database entry
nbrStateShow
Print neighbor's summary information
nbrStatsClear
Reset FSPF neighbor's counters
topologyShow
Print paths to domain(s)
uRouteConfig
Configure static unicast route
uRouteRemove
Remove static unicast route
uRouteShow
Print port's unicast routing info
6 - 24

The ELS Echo request elicits an ELS Echo response from a port identifier in the
fabric and is useful for validating link connectivity.
The two port identifiers are then used to determine if the identifiers are zoned
t
together.
th
The ELS Echo requests comprise a 24-byte Fibre Channel frame header, a 4-byte
ELS Echo request header, an 8-byte timestamp from gettimeofday, and an
arbitrary number of bytes as specified by l length to fill out the request frame. The
source identifier in the ELS Echo request is the domain controller and the
destination identifier is either source or destination.
N t
Note:
The ELS Echo might not be supported on all devices. If this is due to the device not
supporting this ELS Echo request, the message returned may not point directly to
the source or destination device. In such cases, the response could be either an
ELS reject or a request timeout. Do not assume that the device is not FC connected
until you have looked at the results from pathinfo and nsallshow.
B d
By
default,
f lt fcping
i
sends
d fi
five ELS E
Echo
h requests
t tto each
h port.
t
Revision 1008
6 - 25

b41_1:admin> fcping 10:00:00:00:c9:2a:a3:9e 21:00:00:20:37:e1:42:40

Source:
10:00:00:00:c9:2a:a3:9e
Destination:
21:00:00:20:37:e1:42:40
Zone Check:
Zoned
Pinging 10:00:00:00:c9:2a:a3:9e [0xa0100] with 12 bytes of data:

received reply from 10:00:00:00:c9:2a:a3:9e: 12 bytes time:694 usec
5 frames sent, 5 frames received, 0 frames rejected, 0 frames timeout
Round-trip min/avg/max = 531/643/694 usec
Pinging 21:00:00:20:37:e1:42:40 [0x1400e2] with 12 bytes of data:
Request timed out
Request timed out
Request timed out
Request timed out
Request timed out
5 frames sent, 0 frames received, 0 frames rejected, 5 frames timeout
Round-trip min/avg/max = 0/0/0 usec
This device may be offline or may not support ELS Echo frames; nsallshow will
display the device if it is online. The device could also reject the request.
Revision 1008
6 - 26

Provides traceroute for SAN functionality.

pathinfo determines edge-to-edge routing between source port and destination
port.
pathinfo provides information in addition to routing:
Destination port state
Link statistics for every hop from source to destination
Link utilization for each hop from source to destination
pathinfo is intended to gather information on a specific data stream, not the entire
fabric.
fabric
pathinfo is most useful when all the switches in the fabric are configured for Portbased Routing Policy.
The pathinfo command displays routing and statistics information that can aid the
optimization of edge-to-edge routing. The pathinfo command is supported in
Fabric OS v2.6.2, v3.1.2, and v4.2+. All switches in the desired data path must
support the pathinfo
thi f command.
command The percentage of time with zero Tx credits
value (Txcrdz) are displayed as a basic statistic to indicate downstream
congestion.
If Exchanged-based Policy is in effect, subsequent exchanges might not take

the same path as displayed in the pathinfo output.
For more information on pathinfo, see the Fabric OS Command Reference
Manual
Manual.
Revision 1008
6 - 27

Command syntax:
pathinfo [domain [src_portnumber [dst_portnumber]] [-r]]
Revision 1008
6 - 28

topologyshow will list all domains that are part of the fabric, and to each of those domains, all
possible paths from the local switch.
For each path - cost, the number of hops from the local switch to the destination switch, and a
y of all p
ports are routed through
g that p
path.
summary
A path is described by the output port that a frame addressed to a certain domain will be forwarded to
by the switchs routing hardware, in order to reach the domain.
With the Domain ID specified, this command displays the topology information for the specified
destination domain.
Total Bandwidth specifies the total available bandwidth.
Bandwidth
B d idth D
Demand
d displays the possible maximum utilization
utilization.
In the above example you have two equal cost paths with 8G total available bandwidth for each path:
The available bandwidth of each input device is 8 Gbit/sec.
The Bandwidth Demand percentage is Bandwidth In/Bandwidth Out.
In this example:
Path1=56 Gbit/sec over 8 Gbit/sec is 56/8 = 7 which equals 700%
The D flag indicates that Dynamic Load Sharing (DLS) is enabled, this is the default and only
available setting for Exchanged-based Routing.
g
Routing,
g, FSPF will p
program
g
all In-port
p to all available Out Port routes.
Under Exchange-based
In this scenario, the Bandwidth Demand on each ISL is equal to each. Over time each path will be
utilized equally by spreading exchanges across each path.
Theoretically you can divide the 700% by 2, which gives you a Bandwidth Demand of 350% or 3.5:1
for each link.
Revision 1008
6 - 29

In the above example, you have two equal cost paths with 8 G total available
bandwidth for each path.
The available bandwidth of each input
p device is 8 Gbit/sec.
The Bandwidth Demand percentage is Bandwidth In/Bandwidth Out.
In this example:
The D flag indicates that Dynamic Load Sharing (DLS) is enabled, this is the default
and only available setting for Exchanged-based Routing.
Revision 1008
6 - 30

Revision 1008
6 - 31

The routing data base determines how frames are routed from input port to output
port when going to the next destination. Fabric Shortest Path First (FSPF) puts
available equal cost routes in the routing data base. One output port in the trunk
group is put into the routing data base
base. When a communication between two end
devices in a fabric is assigned a route through a trunk, the ASIC of the assigned
trunk group port will be the same ASIC as all ports in the trunk group. This ASIC will
multiplex frames across ISLs in the trunk group and maintain in-order delivery. The
ASIC will send a frame down each link to determine the links latency, these
individual link latency calculations will be used to maintain in-order delivery.
Revision 1008
6 - 32

To use trunking, you must first install the Brocade trunking license. Trunking is enabled automatically
when the trunking license is activated and ports are reinitialized. Trunks are easily managed using
either Fabric OS CLI commands or Web Tools.
Switches are shipped with trunking enabled.
Trunk port-groups are ASIC specific and will be discussed in more detail in subsequent slides.
2 GBit/s port groups have historically been called quads. Bloom port groups include: ports 0-3; 4-7;
8-11; and so on. Bloom ASICs include the Bloom I ASIC which is the foundation for the following
Brocade models: Brocade 3000, 3200, 3800, 3900, and 12000. The Bloom II ASIC is the
foundation for the following Brocade switches: Brocade 3250, 3850, and 24000.
4 and 8 GBit/s port groups include: ports 0-7; 8-15; and so on
5000,
The Condor (4 GBit/s) ASIC is the foundation for the following Brocade models: Brocade 5000
4900, 48000
The Condor2 (8 GBit/s) ASIC is the foundation for the following Brocade models: Brocade DCX
Backbone
GoldenEye port groups include: ports 0-3; 4-7; 8-11; and so on
The GoldenEye ASIC is the foundation for the following Brocade switch: Brocade 200E.
The GoldenEye2 ASIC is the foundation for the following Brocade switches: Brocade 300,
5100 5300
5100,
Additional ASIC specific and advanced trunking criteria include:
Interoperability is not supported with M-series and third-party switch vendors.
The port ISL mode must be disabled (use the portcfgislmode command). This parameter is
related to using third-party gateways to extend fabric distances. This will be further discussed
later in this course.
Long Distance Trunks must be set to the same distance.
When using QoS, all ports in the Trunk Group must be QoS enabled or they will not join the Trunk.
See the Brocade Fabric OS Administrators Guide for additional information.
Revision 1008
6 - 33

The Brocade 4100, 5000, 4900, and 48000 use Brocade Condor ASICs.
Footnote 1: Automatically creates 2-8 ISL trunks when switches are connected to
form 1 logical link. Condor logical links can provide aggregate speeds up to 32
Gbit/sec.
Gbit/
Revision 1008
6 - 34

The Brocade 200E use Brocade GoldenEye ASICs.
Revision 1008
6 - 35

The Brocade 300, 5100, 5300 and DCX use 8 Gbit/sec ASICs (Condor2 and
GoldenEye2).
Footnote 1: Automatically creates 2-8 ISL trunks when switches are connected if
requirements
i
t are met.
t
Fibre Channel links of between 2 and 8 ISLs are connected to form 1 logical link.
Condor/Condor2 logical links can provide aggregate speeds up to 32 Gbit/sec.
Revision 1008
6 - 36

When the allocation to ISL becomes saturated, congestion may occur on an

independent ISL creating a hot spot.
Revision 1008
6 - 37

With 4+ Gbit/sec ASIC trunking in effect, 8 ISLs are aggregated into a single logical
ISL capable of up to 64 Gbit/set aggregate bandwidth. Frames entering the logical
pipe will be allocated to a predetermined port which just happens to be the lowest
back ASIC port in the port group
group. If that port is busy
busy, they will then be allocated
another predetermined port. When the bandwidth of the pipe is fully utilized the
frames are evenly distributed. When the aggregate bandwidth of the logical pipe is
not fully utilized and the 1st predetermined port is free, the frames do not need to be
evenly distributed. Calculated time differentials are needed to maintain in-order
delivery of frames.
If traffic p
patterns involve large
g sequences
q
of frames and high
g bandwidth utilization,,
then the frame allocation across the ISLs in the trunk will begin to evenly distribute
frames across the ISLs.
The predictable way frames are allocated to 4+ Gbit/sec ASIC trunk groups enables
non-disruptive loss of trunk master behavior which is also referred to as PseudoMaster Trunking. This is sometimes called Masterless Trunking.
Bloom trunks stripe
p frames across p
participating
p
g links regardless
g
of total traffic load,,
no ISL in trunk group has priority over another. With Bloom trunking in effect, the 4
ISLs are treated as a single logical ISL capable of 8 Gbit/set aggregate bandwidth.
The sequences of frames are distributed evenly on the frame level across all four
physical paths. Since the source and destination quads use the same buffer pool,
frames received on the ports will be reassembled in the same sequence preserving
in-order delivery.
Revision 1008
6 - 38

Revision 1008
6 - 39

Dynamic Path Selection (DPS) is exchange-base routing where exchanges or

communications between end devices in a fabric are assigned to egress ports in
ratios proportional to the potential bandwidth of the ISL or trunk group.
When th
Wh
there are multiple
lti l paths
th tto a d
destination,
ti ti
th
the iinputt ttraffic
ffi will
ill b
be di
distributed
t ib t d
across the different paths in proportion to the bandwidth available on each of the
paths. This improves utilization of the available paths, thus reducing possible
congestion on the paths. Every time there is a change in the network (which
changes the available paths), the input traffic can be redistributed across the
available paths. This is a very easy and non-disruptive process when the Exchangebased Routing
g Policy
y is engaged.
g g
Exchanges in the example depicted on this slide are allocated based on the primary
criteria: link cost and secondary criteria: potential bandwidth. The potential
bandwidth allocation depicted in this example yields flow allocations of 3:1.
Revision 1008
6 - 40

Revision 1008
6 - 41

When the Trunk Master is disabled, another pre-determined port takes over the role
without fabric disruption.
Revision 1008
6 - 42

Light in a vacuum travels much faster, but in optical cable the rate is about 5
ns/meter. 5ns/meter times 30 meters is equal to 150 ns. The difference in cable
lengths between the ISLs in a trunk determines the deskew value. This is needed
for timing purposes so that delivery of frames across the trunk can be ensured
ensured. The
shortest ISL is selected as the base and is assigned a deskew value of 150 nsec.
The deskew values are expressed (shown in all command displays) by dividing the
time value by 10. Example: A deskew value of 150 nanoseconds is shown as 15
(150/10).
The first ISL in the Bloom trunk to initialize is selected as the trunk master. On 4+
Gbit/sec ASICs the ISL attached to the lowest backport
p will become the g
groups
p
trunk master. The length of the cable is not a consideration when selecting the
master. The deskew values for the other ISLs in the trunk will be calculated from the
base ISL and will have a higher value. Each switch connected by the ISL will have a
deskew value since each has a separate transmit line to the other. Due to the signal
quality/optical media, cables that are identified as the same length may have a
different deskew value. For example, one cable may have a deskew value of 16 and
a cable of the same length may calculate to be 17
17. This is not a problem since
deskew is a true measurement of its transmission capabilities.
Revision 1008
6 - 43

trunkshow
Displays each trunk group
Displays which local port is connected to which remote port
Displays
Di l
WWN off the
h other
h switch
i h
Displays deskew values and identifies the trunk master port
switchshow
Displays the master port (trunk master) and each non-master port
Displays the WWN of connected switch to trunked ports
islshow
Displays bandwidth information associated with each trunk group
switchcfgtrunk
Used to configure trunking to be ON or OFF on all ports on switch
portcfgtrunkport
Used to configure trunking per port as either ON or OFF, default is ON
portcfgshow
Displays port configuration information including trunk ON/OFF status
trunkdebug <start port>, <end port>
Tests specified ports in trunk group and reports trunking status. Possible
trunkdebug outputs include: Switch does not support trunking; Trunking license
required; Trunking not supported in switch interop mode; port<port_id> is not
E_Port;
_
port<port_id>
p
p _
trunking
g is disabled; p
port<port_id>
p _
speed
p
is not 2
Gbit/sec; port<port_id> and port <port_id> are not on the same quad;
port<port_id> and port <port_id> are connected to different switches;
port<port_id> is not a trunk port due to E_port being disabled, or trunking may
be disabled at remote port; port<port_id> and port<port_id> cannot trunk.
Check the link length to verify that the difference is less than 400 m.
Revision 1008
6 - 44

Trunk master selection is not related to deskew.

Trunk master selection is based on the ASICs internal port values, it is predictable.
It is dependent on distance and quality of the link and its associated connection
points.
i t
The switches in this example have multiple trunk groups between them. Notice that
the trunk master is not always the lowest port number in the group nor is it related to
the deskew value.
Revision 1008
6 - 45

The switchshow commands shows trunk groups associated with ports 12-15 with
the master port of the individual trunk groups noted. The remaining ports in the trunk
groups are the non-master ports. It should also be noted that only one trunk group
represents the principal ISL path
path.
Here is the Brocade 300 switchshow truncated command output:
B300:admin> switchshow
switchName:
B300 *truncated output*
Area Port Media Speed State
==============================
*truncated output*
8
8
id
N8
Online
E-Port 10:00:00:05:1e:34:55:fa "B4100"
(Trunk master)
9
9
id
N8
Online
E-Port (Trunk port, master is Port 8 )
10 10
id
N8
Online
11 11
id
N8
Online
12 12
id
N8
Online
13 13
id
N8
Online
E-Port
E Port 10:00:00:05:1e:34:55:fa "B4100"
B4100
(Trunk master)
14 14
id
4G
Online
15 15
id
4G
Online
E-Port 10:00:00:05:1e:34:55:fa "B4100"
(upstream)(Trunk master)
Revision 1008
6 - 46

islshow displays the speeds for each ISL in the trunk groups, the aggregate
bandwidth for trunk groups, and the WWN of the other switch (in this example, we
are connected to one other switch with three trunk groups). Each trunk group is
defined by the trunk master of each trunk
trunk. Port 8 of trunk group 1 points to port 8 of
a trunk group on the attached switch; port 8 is the trunk master of this trunk group
on both of these switches.
Here is the Brocade 300 islshow truncated command output:
B200E:admin> islshow
1:
8-> 8 10:00:00:05:1e:34:55:fa
2: 13-> 13 10:00:00:05:1e:34:55:fa
3: 15-> 15 10:00:00:05:1e:34:55:fa
Revision 1008
B300 sp:
B300 sp:
B300 sp:
8G bw:
8G bw:
4G bw:
32G TRUNK
16G TRUNK
8G TRUNK
6 - 47

portcfgtrunkport is used to turn on or off trunking on one port, specified by

port number. The last parameter indicates on or off (1 for on and 0 for off).
Usage: portCfgTrunkPort [SlotNumber/]PortNumber Mode
Mode: 1 - Configure port to be Trunking capable
0 - Configure port to be Trunking incapable
switchcfgtrunk is used to turn on or off trunking capability for the whole switch
Usage: switchCfgTrunk Mode
Mode: 0 - Configure ports to be Trunking incapable
1 - Configure ports to be Trunking capable
Revision 1008
6 - 48

Usage: trunkdebug port1, port2

port1 - Specify the first port number where you want to debug a trunk link error. This operand is
required.
port2 - Specify the last port number where you want to debug a trunk link error. This operand is
required.
Example 1: To debug a trunk connection for ports 1 through 4:
sw2:admin> trunkdebug 1, 4
port 4 is not E port
Example 2: To debug a trunk for ports 8 11 when only 8 and 11 are trunked:
sw2:admin> trunkdebug 8, 11
ports 8 and 11 are trunked together
Director command syntax:
Director:admin> trunkdebug
trunkdebug: area_number1 area_number2
Note: spinfab was not designed to do performance testing over trunks. Trunk performance testing
will have to be done without spinfab.
spinfab was designed to run a functional test of individual switch-to-switch ISL cabling
Optionally use porttest with a loopback plug at the other end of the link
What is porttest?
Test online ports (F/FL/E or loopback ports); looks for port type and runs appropriate test can optionally select port types
Runs in the background so does not affect traffic
Use porttestshow to retrieve information from porttest
Use stopporttest to stop porttest
porttestshow will return porttest data when successful or failed reason and error
d if nott successful.
f l
code
Revision 1008
6 - 49

Open Trunking actively monitors the utilization on all ISLs. It is designed to minimize
dropped frames by reducing over congestion on ISLs. It will automatically
recalculate the routing table if the utilization of bandwidth between domains on
multiple ISLs is not in relative balance
balance. The entire route table is not recalculated
recalculated, but
nodes assigned to a particular ISL can be reassigned to a new one.
Revision 1008
6 - 50

Nodes are assigned exit ports from one domain to another domain. There may be
multiple exit ports for a node when one switch is connected to several other
switches. Open Trunking balances the transmission of frames to a connected switch.
That is
is, Open Trunking on an Open Trunking-enabled switch balances the transmit
and not the receive on the link.
Revision 1008
6 - 51

Revision 1008
6 - 52

Revision 1008
6 - 53

Revision 1008
6 - 54

Revision 1008
Fibre Channel Long Distance
7-1

Revision 1008
7-2

The most common reason for extending storage area networks over geographical
distances is to safeguard critical business data and provide near-continuous access
to applications and services in the event of a localized disaster. Designing a
distance connectivity solution involves a number of considerations
considerations. The enterprise
must classify stored data and determine how important it is for business operation,
how often it must be backed up, and how quickly it needs to be recovered in the
event of failure. Two important objectives in the designing process are the Recovery
Point Objective (RPO) and the Recovery Time Objective (RTO). The RPO is the
time period between backup points and describes the acceptable age of the data
that must be restored after a failure has occurred. For example, if a remote backup
occurs every day at midnight and a site failure occurs at 11 pm, changes to data
made within the last 23 hours will not be recoverable. The RTO describes the time
needed to recover from the disaster. The RTO determines the acceptable length of
time a break in continuity can occur with minimal or no impact to business services.
Options for replication generally fall into one of several categories.
A business continuity solution with strict RTO and RPO may require highspeed synchronous or near
near-synchronous
synchronous replication between sites as well as
application clustering for immediate service recovery.
A medium level Disaster Recovery (DR) solution may require high-speed
replication that could be synchronous or asynchronous with an RTO from
several minutes to a few hours.
Backup of non-critical application data that does not require immediate access
after a failure can be accomplished via tape vaulting
vaulting. Recovery from tape has
the longest RTO.
Revision 1008
7-3

There are two basic types of optical fiber: Multimode Fiber (MMF) and Single-Mode
Fiber (SMF). Multimode fiber has a larger core diameter of 50 m or 62.5 m (the
latter was common for Fiber Distributed Data Interface (FDDI)) and carries
numerous modes of light through the waveguide
waveguide. It is less expensive than singlemode fiber, but its characteristics make it unsuitable for distances greater than
several hundred meters. Because of this, multimode fiber is generally used for short
distance spans and is common for interconnecting SAN equipment within a data
center. Single-mode fiber has a smaller core diameter of 9 m and carries only a
single mode of light through the waveguide. It is better at retaining the fidelity of
each light pulse over long distances and thus results in lower attenuation. Single
mode fiber is always used for long-distance extension over optical networks and
often used even within the data center for FICON installations.
Optical Small Form-Factor Pluggable (SFP) transceivers are available in short- and
long-wavelength versions. Short-wavelength transceivers transmit at 850 nm and
are used with 50 or 62.5 m multimode fiber cabling. For fiber spans greater than
several hundred meters without regeneration, use long-wavelength transceivers
with 9 m single-mode
single mode fiber
fiber. Long
Long-wavelength
wavelength SFP transceivers typically operate in
the 1310 or 1550 nm range.
Refer to the Appendix in this module for specific Brocade Transceiver models.
Revision 1008
7-4

Revision 1008
7-5

Native Fibre Channel over dark fiber is highly reliable with high performance and
does not require a protocol conversion. It can provide distances up to 100 km at 4
Gbits/sec and 500 km at 1 Gbit/sec.
Native FC connections can also be made to a course or dense wave division
multiplexer. The data is carried on a dedicated wave-length. Transponder-based
solutions use a transponder to convert the different data wavelengths (850 nm,
1310 nm, 1550 nm) to a common wave-length. SFP-based solutions have the
wavelength set by whichever SFP interface is installed.
Fibre Channel can be carried over Synchronous Optical Networks using carriers
such as OC3 (155 Mbits/sec) or OC12 (622 Mbits/sec).
Revision 1008
7-6

Refer to the Appendix in this module for specific Brocade Transceiver models.
Revision 1008
7-7

Dense Wavelength Division Multiplexing (DWDM) is optimized for high-speed, highcapacity networks and long distances. DWDM is suitable for large enterprises and
service providers who lease wavelengths to customers. Most equipment vendors
can support 32
32, 64
64, or more channels over a fiber pair with each running at speeds
up to 10 Gbit/sec. Fiber distances between nodes can generally extend up to 100
km or farther. DWDM equipment can be configured to provide a path protection
scheme in case of link failure or in ring topologies that also provide protection.
Switching from the active path to the protected path typically occurs in less than 50
ms.
Coarse Wavelength
g Division Multiplexing
p
g ((CWDM)) p
provides the same optical
p
transport and features of DWDM, but at a lower capacity, which allows for lower
cost. CWDM is generally designed for shorter distances (typically 50 to 80 km) and
thus does not require specialized amplifiers and high-precision lasers (lower cost).
Most CWDM devices will support up to 8 or 16 channels. CWDM generally operates
at a lower bit rate than higher-end DWDM systemstypically up to 4 Gbit/sec.
There are two basic types of Wavelength Division Multiplexing (WDM) solutions:
Transponder-Based Solutions. Allows connectivity to switches with standard
850 or 1310 nm optical SFP transceivers. A transponder is used to convert
these signals using Optical-to-Electrical-to-Optical (OE-O) conversion WDM
frequencies for transport across a single fiber. By converting each input to a
different frequency, multiple signals can be carried over the same fiber.
SFP-Based Solutions. These eliminate the need for transponders by requiring
switch equipment to utilize special WDM transceivers (also known as colored
optics), reducing the overall cost. CWDM SFPs are like any standard
transceiver used in Fibre Channel switches, except that they transmit on a
particular CWDM frequency.
Revision 1008
7-8

Revision 1008
7-9

Revision 1008
7 - 10

Effective bandwidth is a measure of how much of the available bandwidth can actually be used,
taking into consideration dropped packets and retransmission due to congestion and protocol
inefficiency.
Delay or latency (synonym) is the amount of time that it takes a packet of data to get from one
point to another.
another Delay is often measured by sending a packet that is returned to the sender; the
packets round-trip time (RTT) is considered the delay in the network. In an IP network, delay
depends not only on distance, but also upon router hop count, router processing, and packet size.
A reasonable rule of thumb to estimate delay is driving distance * 1.5.
Packet Loss: The Internet Standards treat packet loss and congestion as synonyms. Congestion
is the prime cause of packet loss. Congestion occurs when Routers discard incoming packets that
cant be stored or transmitted because the average sum of the inputs to a router exceeds the
capacity of its output. Slightly reworded: anytime the average feed from the Ethernet exceeds the
capability
p
y of a router,, packets
p
will be lost. When the output
p connection is a costly
y nation-to-nation
or satellite link, it becomes very expensive to make the pipe big enough so packets wont be lost.
If the pipe is too big, then you become concerned with effective bandwidth. Other contributors that
cause packet loss include bit errors, deliberate discards, and router in/out times. See
http://www.gigabytex.com/whttcp.htm for additional information. TCP assumes that all
packet loss is caused by congestion and responds by reducing the transmission rate.
Different applications use protocols with different block sizes to transfer data. Block access protocols
access "blocks" of data in portions that are a multiple of the OS system block. Consider using the
following guidelines to determine block sizes: Transaction data (4-8k block size); Office automation
(16 32k block size); Data warehousing (64 - 256k block size); CAD/Design (64
(16-32k
(64-128k
128k block size);
Multimedia (512k - 4M block size). Small block sizes of contiguous data mean more I/O especially if
the data is spread across the disk; large block sizes that don't use all the space read the whole block
just to get a small piece of data.
Applications can be configured to allow multiple outstanding I/Os to occur before requiring an
acknowledgement. The # of outstanding I/Os is typically 1 to 16.
FOS 5.3.x and later support FC FastWrite to improve SCSI write operations over long-distance ISLs.
This is accomplished by the creation of a Proxy Target (PT) local to the initiator host, and a Proxy
Initiator (PI) local to the target storage device
device. When a SCSI write is received
received, the PT responds with
XFER_RDY, enabling the host to begin transmission of data over the network without having to wait
for response from the target device, thus eliminating latency caused by the long-distance link. Once
the PI intercepts the XFER_RDY from the target device, data is sent to the target.
FastWrite is supported on the Brocade 7500 SAN Router and FR4-18i Routing Blade.
Revision 1008
7 - 11

Data traffic Virtual Channels (VCs) are collapsed to optimize performance over long
distances using the portcfglongdistance command, as shown in the diagram
below.
IInformation
f
ti about
b t switch
it h characteristics
h
t i ti and
d capacity
it iin tterms off b
buffers
ff
per portt
group, port speed, and distances supported is contained in FOS Fabric OS
Administrator's Guide and the appropriate Hardware Reference manual specific to
the switch you are configuring.
VC2
VC3
VC2
VC 2 only
VC3
VC4
VC4
VC5
VC5
Revision 1008
7 - 12

LE does not require a license key.

LS mode is for Fabric OS v5.1+ ONLY.
The maximum Extended Fabric distance depends on the version of Brocade switch ASIC installed in the switch.
Extended Fabric distance levels (L0, LE, LD, LS) persist across switch reboots and power cycles:
Can not be set or removed by configure or configdefault
Can be cleared by portcfgdefault
Saved in a switch configuration file (configupload ) as portcfg parameter
Level 0 static mode (L0) - L0 is the normal mode for a port. An extended ISL port competes with other ports for
buffers. No buffer credits are reserved for extended ISLs.
Level E static mode (LE) - LE reserves a static number of buffer credits that supports distances up to 10 km.
The number reserved depends on the port speed. The baseline for the calculation is one credit per km at 2
Gbps. This yields the following values for 10 km:
5 credits per port at 1 Gbps
510 credits for one port only at 120 km for 10Gbps (specific to the FC10-6 blade)
Dynamic Mode (LD) - LD calculates buffer credits based on the distance measured during port initialization. An
upper limit is placed on the calculation by providing a desired distance value. When the measured distance is
more than desired distance, the desired distance is used in the calculation. This is a mechanism for controlling
the
h number
b off reserved
db
buffer
ff credits
di ensuring
i b
buffer
ff availability
il bili ffor other
h ports iin the
h same group.
Static long distance mode (LS) - LS calculates a static number of buffer credits based on a desired distance
value. For both LD and LS distance levels, you can use the following formula to get an approximation of the
calculated number of buffer credits:
buffer credits = [(distance in Km) * (data rate) * 1000] / (payload size)
For LD, distance in Km is the smaller of the distance measured during port initialization versus the desired
distance value.
For LS, distance in Km is always the desired distance value. data rate is:
1.0625
1 0625 for
f 1 Gbps
Gb
2.125 for 2 Gbps
4.25 for 4 Gbps
8.5 for 8 Gbps
Revision 1008
7 - 13

Syntax portcfglongdistance [Slot/]<port_number> , distance

level, [vc_translative_init ] , <desired_distance>
The vc_translative_init value of 1 in the example on this slide is needed as
a syntax
t position
iti h
holder.
ld If a desired_distance is
i nott specified
ifi d th
then
vc_translative_init is enabled without explicitly calling it out.
Best Practice When VC Translation Link Initialization is enabled (set to ON), the
port uses an extended link initialization sequence, which is an enhanced link reset
protocol designed to avoid excessive resetting of ports used on a long distance
connection. As a best practice, Brocade recommends this feature be enabled. It is
enabled by default on Fabric OS v4
v4.4+
4+ switches
switches. Use the portcfgshow
portcfgsho command
to verify that vc_translaltion_link_init is enabled at both ends of the link.
Revision 1008
7 - 14

Revision 1008
7 - 15

Revision 1008
7 - 16

It is also important to note that when frame-based trunking is used, the deskew
value is calculated when the trunk is formed. (For more about the deskew value,
see the Fabric OS Administrators Guide.) If a protection switch results in a change
of latency,
latency then these deskew values are no longer valid and unpredictable behavior
could result. Therefore a key design principle is to ensure that if any rerouting is
possible, then entire trunks must be rerouted as homogenous groups.
Revision 1008
7 - 17

Revision 1008
7 - 18

Before discussing FC-level buffer allocation, it is important to note that the

availability of sufficient FC-level buffering is not itself sufficient to guarantee
bandwidth utilization. Specifically other limitations, particularly at the SCSI
level of the storage initiator and/or target, are often the limiting factor. The I/O
size, IO per second limit, and concurrent/outstanding IO capability at the
SCSI level of the initiators/targets can be and often are gating factors. While
exact calculations are possible, a simple rule of thumb is used in the
calculation of the BB credit requirement of a given link. Based on the speed
of light in an optical cable, a full-size FC frame spans approximately 4 km at
1 Gbit/sec, 2 km at 2 Gbit/sec, 1 km at 4 Gbit/sec, or 400m at 10 Gbit/sec. To
calculate the BB credit requirement
requirement, simply take the round
round-trip
trip cable distance
and divide by the frame length. For example, at 1 Gbit/sec at 10km, 5 BB
credits are required to allow wire speed data transfer. As the FC speed or
distance increases the number of BB credits increases in a linear fashion. In
addition, as a rule of thumb, a link with a round-trip latency of 1 ms spans a
distance of approximately 100 km. With a round-trip distance of 200 km, 200
buffer credits would be required at 4 Gbit/sec. Having insufficient BB credits
will
ill nott cause lilink
k ffailure,
il
b
butt will
ill reduce
d
th
the maximum
i
th
throughput.
h t S
So th
the
aforementioned 1 ms link running at 4 Gbit/sec with only 100 BB credits can
achieve a maximum throughput of only approximately 2Gbit/sec. Finally,
these calculations use approximate metrics. The most important thing to
remember is that if a given implementation had in a high percentage of FC
frames significantly less than maximum size, then more BB credits may be
required.
q
Refer to the Appendix in this module for distances on specific products.
Revision 1008
7 - 19

Footnote 1: This is done by sending an LR (Link Reset) and the target switch sends
back an LRR (Link Reset Response). Because this happens on an E_Port, the link
does not reset, it just resets the BB counters.
F t t 2:
Footnote
2 The
Th default
d f lt portt mode
d L0 is
i nott supported.
t d
Supported on 300, 5100, 5300 and DCX Backbone (GE2 and Condor2 ASICs only)
Footnote 3: To disable this feature run:
portcfgcreditrecovery --disable [slot/]port
Revision 1008
7 - 20

Credit Recovery is defined in FC-FS

BB_SC_N (Buffer-to-Buffer State Changer Number) in ELP is defined in FC-SW4.
BB_SC_N: This field indicates the Buffer-to-Buffer State Change number. The
BB SC N field
BB_SC_N
fi ld is
i valid
lid only
l if the
th R_RDY_Flow
R RDY Fl
Control
C t l mode
d is
i specified
ifi d in
i the
th ISL
Flow Control Mode field. A value between 0 and 15 indicates that the sender of the
ELP frame is requesting a 2BB_SC_N number of frames to be sent between two
consecutive BB_SCs Primitive Signals, and a 2BB_SC_N number of R_RDY
Primitive Signals be sent between two consecutive BB_SCr Primitive Signals. When
the two ports exchanging link parameters specify different non-zero values of
BB_SC_N,, the larger
g value shall be used. If either p
port specifies
p
a BB_SC_N value
of zero, then the BB_Credit recovery process shall not be performed and no
BB_SCx Primitive Signals shall be sent. If a port specifies a non-zero BB_SC_N
value it shall support the BB_SCs and BB_SCr Primitive Signals. See the FC-FS-2
standard on the www.t11.org website for a description of the BB_Credit recovery
process.
Revision 1008
7 - 21

Revision 1008
7 - 22

Revision 1008
7 - 23

Revision 1008
7 - 24

Revision 1008
7 - 25

For each user port, the following information is displayed:

Possible Port Type values are E (E_Port), F (F_Port), G (G_Port), L (L_Port), or U (U_Port).
Possible Lx Mode (level) values are (Mode L0), LE (mode LE), LM (mode L0.5), L1 (mode
L1), L2 (mode L2), LD (mode LD) and LS (mode LS).
Max/Resv Buffers displays the maximum number of buffers that will be allocated to the
port, based on the maximum possible speed for the port and the specified LX mode. In the
example on this slide, port 8 is configured as mode L0.5. To support the maximum possible
speed of 4 Gbit/sec, the Condor reserves 56 buffers for port 8.
Buffer Usage displays the actual number of buffers the system was able to allocate to the
port. In the example above, port 8 is configured as mode L0.5 but is operating at 2 Gbit/sec. To
pp
the actual speed,
p
, the Condor allocates 31 buffers (at
( 2 Gbit/sec)) for port
p
8 but
support
reserves 56 (for 4 Gbit/sec potential) which will not be available in the Remaining
Buffers. If port speed is locked to 2 Gbit/sec the Max/Resv Buffers will equal the
Buffer Usage.
Needed Buffers displays the estimated number of buffers that are needed for the port to
operate at full bandwidth (depending on the port configuration). In the example above, 31
buffers are needed for port 8, to support a full-bandwidth, 2 Gbit/sec, mode L0.5 port.
Link Distance displays (in kilometers) either the physical distance of an LD/LS link, or the
maximum distance for a non-LD/LS port. In the example above, the link distance value for port
8 is 25 km, which matches the 2 Gbit/sec, mode L0.5 configuration of the port.
On a Condor/GoldenEye-based switch, the Remaining Buffers value is determined by subtracting the
larger of the per-port Max/Resv Buffers or Buffer Usage values from the total buffers
available. Since the example above is taken from a Brocade 4100 with a Condor ASIC, there are a
total of 1000 buffers available to the ports (there are no internal buffers). These values are:
There are 21 ports with 8 buffers (ports 0-2, 6,7, and 16-31): 8 x 21 = 168
There are 9 ports with 26 buffers (ports 3-5, and 10-15): 26 x 9 = 234
There are 2 ports with 56 buffers (ports 8 and 9): 56 x 2 = 112
Remaining Buffers is 1000 168 234 112 = 486 which is the value shown on the last line in
the slide above.
Revision 1008
7 - 26

Revision 1008
7 - 27

Revision 1008
7 - 28

Revision 1008
7 - 29

A
Appendix
di A FC-PI-4
FC PI 4 LINK NOMENCLATURE
Revision 1008
7 - 30

A
Appendix
di B Brocade
B
d T
Transceiver
i
P
Partt N
Numbers
b
Revision 1008
7 - 31

A
Appendix
di C Cable
C bl ttypes and
d di
distances
t
Revision 1008
7 - 32

A
Appendix
di D Link
Li k L
Loss b
based
d on 8G
Revision 1008
7 - 33

A
Appendix
di E 4G S
Switches
it h E
Extended
t d dF
Fabrics
bi
Revision 1008
7 - 34

A
Appendix
di E 48k E
Extended
t d dF
Fabrics
bi
Revision 1008
7 - 35

A
Appendix
di E 5300 E
Extended
t d dF
Fabrics
bi
Revision 1008
7 - 36

A
Appendix
di E 300 E
Extended
t d dF
Fabrics
bi
Revision 1008
7 - 37

A
Appendix
di E 5100 E
Extended
t d dF
Fabrics
bi
Revision 1008
7 - 38

A
Appendix
di E DCX E
Extended
t d dF
Fabrics
bi
Revision 1008
7 - 39

A
Appendix
di F Trunking
T ki and
dE
Extended
t d dF
Fabrics
bi
Revision 1008
7 - 40

A
Appendix
di G FC Frame
F
Length
L
th over Fiber
Fib Optics
O ti
The given we have is it takes light 5 nsec to traverse 1 meter over a fiber optic
medium, thus in 1 second light traverses 200,000,000 meters (1/.000000005).
If the speed is 1 Gbit/sec, thats 100 Mbytes/sec, which is 100,000,000 bytes/sec.
200,000,000 meters / 100,000,000 bytes = 2 meters per byte.
Using 2K as the given frame size, a frame is 4 km in length (2K x 2 meters per byte)
when the speed is 1 Gbit/sec.
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
If the speed is 2 Gbits/sec: 200,000,000 meters / 200,000,000 bytes = 1 meter per
byte
2K frame x 1 meter per byte = 2 km per frame
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
If the speed is 4 Gbits/sec: 200,000,000 meters / 400,000,000 bytes = .5 meter per
byte
2K frame x .5 meter per byte
y = 1 km per frame
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
If the speed is 8 Gbits: 200,000,000 meters / 800,000,000 bytes = .25 meter per
byte
2K frame x .25 meter per byte = .5 km per frame
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Revision 1008
7 - 41

Revision 1008
7 - 42

Revision 1008
Administration and Maintenance
8-1

Appendix Additional information on firmware downloads
Revision 1008
8-2

With today's demand for compliance and continuous data access, storage networks
are growing in size and complexity. Without the right tools, management costs can
skyrocket, and make business objectives and government mandates extremely
difficult to achieve.
achieve Brocade SAN management solutions help organizations
optimize their storage network resources, maximizing availability while improving
efficiencies and reducing costs. Brocade's comprehensive management software
enables centralized management of SAN resources - regardless of vendor, protocol,
or geography. Whether a SAN is small or very large, straightforward or extremely
complex, Brocade management software can reduce both complexity and overall
storage costs.
EFCM stands for Enterprise Fabric Connectivity Manager.
Revision 1008
8-3

Revision 1008
8-4

Fabric Manager lets you manage your Storage Area Network (SAN) from a single user
interface. With this software, you can configure any aspect of your SAN. Fabric Manager
encompasses multiple fabrics, all switch types, and all firmware versions.
Use Fabric Manager to configure multiple switches simultaneously from one location
location. View
the status of multiple devices in one window. Perform SAN-level maintenance without
accessing switch after switch. Fabric Manager lets you administer all of the elements of
your SAN from one point of entry.
Fabric Manager features include: Firmware Downloads, Management/Monitoring Groups,
Topology ISL Monitoring, Sequenced Reboots, At-a-glance Views, Subnet Scanning,
License Management, Fabric Checking, Configuration Handling, Multi Fabric
Administration Fabric Merge Tool
Administration,
Tool, FDMI
FDMI, FICON/FICON CUP
CUP, Change Management
Management, Fibre
Channel Routing management, supportshow capturing, Performance monitoring, Alerts
view, Secure Fabric OS (SFOS) Management, Reports - link to SAN Health, RADIUS,
configuration and replication, Server Management Console (SMC), and FCIP Tunneling.
See Brocades Fabric Manager Administrators Guide for additional information. You can
additionally search for Brocade Fabric Manager courseware from Brocades Education
Services web site.
Y should
You
h ld h
have th
the ffollowing
ll i iinformation
f
ti available
il bl prior
i tto starting
t ti th
the iinstallation:
t ll ti
Fabric Manager serial number and license key (if you are installing the full version)
Type of authentication to use:
Windows authentication: You need to know the Windows Domain or Workgroup
name.
RADIUS server authentication: For each RADIUS server, you need to know the IP
or DNS name, authentication
th ti ti portt and
d type
t
(PAP or CHAP),
CHAP) and
d shared
h d secret.
t
Switch-based authentication: You need to know the switch IP addresses.
Solaris and Linux only: NIS authentication. You need to know the host name or IP
address of your NIS server and NIS domain name.
Revision 1008
8-5

EFCM V9.7 comes in the following levels of service:

EFCM Basic is the embedded web server within the M-Series products and does not require a
license. The Mi10K does not have EFCM basic, use CLI or EFCM to manage.
EFCM Standard Edition is intended for small business SANs, up to a maximum of 140 ports.
EFCM Enterprise Edition is intended for all environments, including the largest Enterprise-class
data centers. EFCM Enterprise is the only option for those customer who wish to license the
optional EFCM V9.7 Advanced Module or Software Bundles. EFCM Enterprise (required for MEOS Directors) supports up to 2500 ports and 25 simultaneous clients. It also gives you the
option to export configurations to flat-files and to databases (mySQL and DB2). It also allows
auto discovery via subnets.
Each M-Series Switch in the fabric requires an Element Manager license installed in order to
be managed by EFCM. Like Fabric OS Switches/Directors, M-EOS Switch/Director license
keys are linked to serial numbers. Directors with M-EOS automatically add license features for
all previously purchased software. M-EOS Switches with M-EOS 6.0 or later need a feature
key to enable Element Manger or any other Switch feature. When you purchase additional
software license keys you receive a new feature key that includes existing features.
The Advanced SAN Routing option gives you the equivalent functionality to manage classic
McDATA routers by linking to SANvergence Enterprise, and is available with either the EFCM
Enterprise Edition or EFCM Standard Edition
Edition.
The Advanced Module enables four EFCM software modules: Performance, Event
Management, Group Configuration, and Security Center.
Bundling options include the Advanced modules and PFE (Product Feature Enhancement)
keys for firmware-based features like Open Trunking, and FICON Management Server.
Binding and Authentication are also available a la carte for individual switches or combined into
Software Bundles with the Advanced Module. These software bundles are sold on a perproduct basis and are designed to provide all the advanced capabilities necessary for a
particular type of environment. They are sold separately and include device-specific firmware
enhancements, such as FICON, large-scale mixed fabrics, enhanced encryption and
performance tuning.
An FTP server can be configured and installed during the EFCM installation.
Revision 1008
8-6

Web Tools - A browser-based application, Web Tools simplifies management by

configuring, monitoring, and managing individual switches and small SANs.
Available for SANs running Brocade Fabric OS.
EFCM B
Basic
i - A simple,
i l straightforward
t i htf
d managementt ttooll ffor configuring,
fi i
monitoring, and managing individual switches and small SANs up to 100 ports.
Available for SANs running Brocade M-EOS.
Revision 1008
8-7

Users can perform the following tasks from the EFCM Basic interface:
Display the properties and operational status of the product, FRUs, and Fibre Channel ports,
display product operating parameters, and display fabric parameters.
Configure the director or switch, including:
Fibre Channel port parameters, port types, and data transmission speeds
Product identification, date and time, operating domain parameters, fabric parameters, and
network addresses
Parameters for product management through Simple Network Management Protocol
(SNMP), the Command Line Interface (CLI), the Open System Management Server
(OSMS) feature, or the Fibre Connection (FICON) management server (FMS) feature
Security options which include authorized users, settings, port binding, switch binding, and
fabric binding
binding. Additional security includes setting access to IP ACL
ACL, Authorization
Authorization, and
RADIUS functions.
Zones and zone sets
Authentication for users
Monitor ports, port optics, and port statistics, and display the event log and node list.
Perform product operations and maintenance tasks, including:
Enable unit beaconing, turn off the system error light, set the product online or offline, and
perform a configuration reset
Enable port beaconing, perform port diagnostics, and reset ports
Retrieve dump files and retrieve product information files
Install optional feature keys
Configure product Internet Protocol (IP) addresses, names, and SNMP settings
Install new versions of product firmware
Manage user access to features
Control product ports on an individual basis
Troubleshoot problems using event log and error status indicators. Administrators and
operators can access real-time information about the product and fabric.
The interface requires an industry-standard web browser.
EFCM Basic Edition does not require any licensing.
Revision 1008
8-8

For some switch models, Web Tools provides a simplified interface, Web Tool EZ,
that allows less experienced users to perform basic management tasks. Web Tools
EZ is a simplified version of Web Tools. It simplifies switch management by
providing an easy-to-use subset of basic switch-management tasks.
tasks
Web Tools works for a single switch fabric only. It displays only the launch switch
and associated tasks, without fabric information.
You can use Web Tools to do the following:
monitor the switch, including port and FRU status
manage
g basic zoning
g
perform basic switch configurations.
Web Tools is part of the Fabric OS of a switch, and Web Tools functionality is
engaged with a Web Tools license. When you launch Web Tools on a switch, you
can manage other switches in the fabric that have lower or higher firmware
versions. It is important to note that when accessing these switches you are
opening
p
g the remote switchs version of Web Tools, and the functionality
y available for
those switches might vary.
From the Port Management application, you can:
Swap ports
Enable/disable ports
Configure port type/speed
Revision 1008
8-9

The GUI can be launched by itself or from EFCM (right-click on the HBA and select
ESCM.
To access the CLI commands run escm_cli. CLI options are:
1.
Identify firmware, BIOS, and/or FCode
2.
Download firmware (update the FLASH)
4.
Download/erase BIOS and/or FCode (update the FLASH)
8.
Scan for devices10. Change IOC settings (interrupt coalescing,

EEDP)
13.
Change FC Port settings
16
16.
Di l
Display
l
logged-in
d i d
devices
i
20.
Diagnostics
22.
Reset bus
23.
Reset target
30.
Beacon on
31.
Beacon off
42
42.
Display operating system names for devices
60.
Show non-default settings61. Restore default settings
68.
Show port state summary
69.
Show board manufacturing information
98.
Reset FC link
99.
Reset port
Enable paged mode in menus
Enable logging
Main menu, select an option: [1-99 or p/w or 0 to quit]
Revision 1008
8 - 10

Boot BIOS Can be accessed by interrupting boot of the host. See Brocade 400
Host Bus Adapters Users Guide for more information.
Remote host HBAs can be discovered and managed.
Diagnostics:
Di
ti
R
Requires
i
HBA to
t have
h
access to
t a target
t
t device.
d i
The
Th diagnostics
di
ti will
ill
then test the path from the HBA to the target to make sure there is a reliable
connection.
Device Persistence: Locks the association of an FC target with a specific SCSI ID.
By marking a device persistent, the Brocade adapter will remember the target
mappings between power cycles. This will ensure that the OS will not accidentally
move a target to a new drive letter or mount point
point.
Revision 1008
8 - 11

Simple Network Management Protocol (SNMP) is a standard method for monitoring

and managing network devices. Using SNMP components, you can program tools
to view, browse, and manipulate Brocade switch variables and set up enterpriselevel management processes.
processes
Every Brocade switch runs an SNMP agent and Management Information Base
(MIB). The agent accesses MIB information about a device and makes it available
to a network manager station. You can manipulate information of your choice by
trapping MIB elements using the Fabric OS CLI, Web Tools, or Fabric Manager.
The SNMP Access Control List (ACL) provides a way for the administrator to restrict
SNMP get/set operations to certain hosts/IP addresses
addresses. This is used for enhanced
management security in the storage area network.
Network management is all about keeping the network up and running, monitoring
and controlling network devices remotely using conventional network technology.
Local management and remote management are two ways of managing a device.
Enterprise management software, like HP OpenView, Tivoli, and CA Unicenter,
monitor thousands of devices in an enterprise.
p
Many
y of these p
packages
g have addon components that are tailored to manage Brocade SANs. There are also
commercial and freely available packages that can be run separately by the SAN
administrator so that they can get alerts, trend performance and capture details of
errors of switches separately from large network management station.
Revision 1008
8 - 12

Within the SNMP model, a manageable network consists of one or more manager
systems (or network management stations), and a collection of agent systems (or
network elements). SNMPv3 calls both manager and agent systems entities.
A Manager or Network Management System (NMS) is a separate entity that
manages the agent entity from a remote place. This is typically a computer that is
used to run one or more network management systems. The NMS is responsible for
managing the SNMP agent/entity; it is also responsible for polling and receiving
traps from agents. The manager communicates with an agent using the Simple
Network Management Protocol (SNMP). The Brocade agent supports both SNMP
version 1 (SNMPv1) and Community-based SNMP version 2 (SNMPv2C). SNMP
allows a manager
g to retrieve management
g
information, such as its identification,
from an agent. A manager can change management information on the agent. This
operation is termed SNMP-SET. An agent can send information to the manager
without being explicitly polled. This operation is termed a trap in SNMPv1 or a
notification in SNMPv2C/v3. Traps or notifications alert the manager to events that
occur on the agent system, such as a reboot. For the rest of the document, the term
trap is used.
An agent
g
entity
y is a mediator between the manager
g entity
y and an end device in a
SAN. The agent entity resides in a SAN device and makes the management
information available to the manager entity. It acts as a mediator between
management entity and an end device in the SAN. It resides in a SAN devicein the
Brocade world it resides in the switch. It makes information available to the SNMP
Management Entity. Brocades agent entity does the following: implements SNMP
protocol, stores and retrieves management information defined in supported SNMP
MIBs, and collects and maintains information about the switches local environment.
MIB is an acronym for Management Information Base. A MIB is a database of
network management information. The information is a combination of switch
settings, hardware configuration, status and statistical data (information about the
agent). Brocade MIBs can be compiled in the management station.
Revision 1008
8 - 13

Every Brocade switch carries an Agent and Management Information Base (MIB),
as illustrated above.
The agent accesses information about a device and makes it available to an SNMP
NMS When
NMS.
Wh active,
ti
the
th NMS can gett information
i f
ti or sett information
i f
ti when
h it queries
i
an agent. SNMP commands, such as: get, set, getnext, setnext, and getresponse
are sent from the NMS, and the agent replies once the value is obtained or
modified. Agents use variables to report such data as the number of bytes and
packets in and out of the device, or the number of broadcast messages sent and
received. These variables are also known as managed objects. All managed objects
are contained in the MIB.
The NMS can also receive traps, unsolicited messages, from the switch (agent) if
an unusual event occurs. NMS SNMP application must be configured to respond to
traps.
To receive SNMP traps ensure proper MIBs are enabled and the trap recipient(s) on
the switch have been set.
Revision 1008
8 - 14

SNMPv1 Authentication between the NMS and the switch is performed using unsecure clear text community
strings only.
SNMPv3 Authentication between the NMS and the switch is performed using an MD5 or SHA cryptographic
hash of the Username stored on both the switch and the NMS station. In addition privacy is also supported in
SNMPv3 by encrypting packets using DES. SNMPv3 supports thee security levels:
No authentication and no privacy Username is not hashed and data is not encrypted
encrypted.
Authentication and no privacy Username is hashed but data is not encrypted.
Authentication and privacy Username is hashed and data is encrypted.
The security subsystem can contain more than one security model. In the case of Brocade switches, for
example, it contains: SNMPv1 with community strings; SNMP v3 user-names, authentication, and privacy; and
Access Control Lists (ACLs). Brocade enabled SNMP ACLs in all Fabric OS versions.
Authentication occurs using either MD5 or SHA algorithm checks. MD5, message-digest algorithm, is an
extension of the MD4 algorithm. The MD5 algorithm takes an input a message of arbitrary length and produces
an output 128-bit "fingerprint" or "message-digest". It is designed for circumstances where a large file must be
"
"compressed"
d" in
i a secure manner b
before
f
being
b i encrypted
t d with
ith a private
i t ((secret)
t) kkey under
d a public-key
bli k
cryptosystem such as RSA. RSA is a public-key encryption technology created by Ron Rivest, Adi Shamir, and
Leonard Adleman. RSA was built from the fact that, given that it is pretty easy to multiply two very large prime
numbers together to get their product, it is extremely difficult to go the reverse direction: to find the two prime
factors of a composite number. This one-way nature of RSA allows an encryption key to be generated and
shared with the world, and still not allow a message to be decrypted. SHA is an acronym which stands for
Secure Hash Standard. SHA is modeled after MD4. Like MD5, SHA takes a message and produces a
message-digest, its output is 160-bit. The message-digest can then be put into the Digital Signature Algorithm
(DSA) to generate or verify the signature for the message. This signing of the message-digest rather than the
message is designed to improve the efficiency. The same algorithm must be used at both ends of the
communication.
DES is a Digital Encryption Standard that uses a secret key to encrypt/decrypt messages.
Example configure output SNMP authentication section:
RSL1_ST07_B41:admin> configure
Not all options will be available on an enabled switch.
To disable the switch, use the "switchDisable" command.
Configure...
System services (yes, y, no, n): [no]
ssl attributes (yes, y, no, n): [no]
http attributes (yes, y, no, n): [no]
snmp attributes (yes, y, no, n): [no] y
Select SNMP Security Level:
(0 = No security, 1 = Authentication only, 2 = Authentication and
Privacy): (0..2) [0]
Revision 1008
8 - 15

To configure an external management application to receive traps:

Verify that the community string matches the value used in the monitoring
tools.
Ensure the proper MIBs are enabled.
Revision 1008
8 - 16

Revision 1008
8 - 17

The firmware download process for a single-CP switch is less complex than for the
Brocade directors.
The flash is divided into two equal size partitions. The version of the firmware
running
i on th
the switch
it h iis iin th
the P
Primary
i
P
Partition
titi and
d another
th version
i off firmware
fi
is
i
kept in the Secondary Partition. The version in the Secondary can be the same as
the Primary or different.
The following slides will illustrate the individual steps that occur when a
firmwaredownload command is issued.
Revision 1008
8 - 18

The firmwaredownload command is issued to download Fabric OS version

v6.1.0c.
The default firmwaredownload options are automatically executed. The default
settings are to auto-reboot
a to reboot and a
auto-commit.
to commit If the -s option is used
sed and a
auto-boot
to boot
mode is not enabled, a reboot command must be issued to activate the downloaded
firmware. If the -s option is used and auto-commit mode is not enabled, a
firmwarecommit command must be issued to replicate the downloaded firmware
to the Secondary Partition.
Current status:
Firmware FOS A is
Fi
i in
i the
th Primary
Pi
Partition
P titi and
d has
h been
b
committed
itt d to
t the
th
Secondary Partition.
Revision 1008
8 - 19

The firmware is downloaded into the Secondary Partition.
Revision 1008
8 - 20

Upon completion of the download, the Secondary Partition becomes the Primary
and the Primary Partition becomes the Secondary.
Revision 1008
8 - 21

After rebooting, the new version is now running on the switch.
Revision 1008
8 - 22

The firmware in the Primary Partition is committed to the Secondary Partition.
Revision 1008
8 - 23

The results of firmwaredownload are:

r7-st04-b51-1:admin> firmwareshow
Appl
Primary/Secondary Versions
-----------------------------------------FOS
v6.1.0c
v6.1.0c
Revision 1008
8 - 24

Revision 1008
8 - 25

The Active CP:

manages POST for all FRUs
runs Fabric OS; configures all port cards for data transfers, hard-zoning, and routing
interacts
i t
t with
ith allll attached
tt h d d
devices
i
presents the IP addresses for the switch and the CP card
issues all SNMP-based messages
illuminates the Active CP LED
synchronizes the Standby CP to the Active CP through the High Availability Monitor.
The Standby CP runs only those parts of Fabric OS associated with CP failover, so only a
limited set of management tasks can be performed on the Standby CP.
The High Availability Monitor (HAM) tracks several failover mechanisms, including the CP
Error Line and private IP-based heartbeat between the CP Cards; the Watchdog Timer
running on each CP Card; and the Health Monitor running on the Standby CP card is ready
to become active at any time.
The failover to the Standby CP is a hot failover that is, all I/Os continue uninterrupted
through the switch ports. The flashing of LEDs is driven by Fabric OS. Thus, during the CP
card failover, the software that would normally flash the port LEDs cannot do so, and the
port LEDs will remain solidly lit.
To ensure stable functionality during a failover, you must install the same version of Fabric
OS on both CP cards, and have the same version of Fabric OS running on both CP cards.
The procedures that you will perform later in this class conform to this requirement.
Use the hashow command to determine the current High Availability (HA) status:
B48000:admin> hashow
Local CP (Slot 5, CP0): Active
Remote CP (Slot 6, CP1): Standby, Healthy
HA enabled, Heartbeat Up, HA State synchronized
Revision 1008
8 - 26

The hashow command displays many details about the current high-availability
status of the Brocade Director including:
Local CP state (slot number, CP ID): Active or Standby
Remote CP state (slot number, CP ID): Active or Standby
High Availability: HA Enabled or HA Disabled
Heartbeat: Up or Down
Health of standby CP: Healthy (standby CP is running, and the health
monitor has not detected any errors), Failed (standby CP is running, but the
health monitor has discovered a problem with the blade), and Unknown (the
standby CP does not exist, the heartbeat is down, or the health monitor detects
a configuration file error).
HA synchronization status: HA State Synchronized (the system is
currently fully synchronized; a CP Card failover would be non-disruptive) and
HA State Not In Sync (the system is unable to synchronize the two CPs,
due to the standbyy CP being
g faulty
y or another system
y
error; a CP Card failover
would be disruptive).
In the example, CP0 is the Active CP, and CP1 is the Standby CP, which is healthy.
HA is enabled on this Director, the Heartbeat is up, and the HA Monitor is
synchronized; thus, a non-disruptive failover would occur.
Revision 1008
8 - 27

These commands are rarely used by most system administrators.
Revision 1008
8 - 28

In the example, P2 is currently the primary partition and holds the version of
firmware currently running on this CP card (FW vC). P1 is the secondary partition
and holds an older revision of Fabric OS firmware (FW vB).
Revision 1008
8 - 29

RSL_48K8:admin> firmwaredownload
This command will upgrade both CPs in the switch. If you
want to upgrade a single CP only, please use -s option.
You can run firmwareDownloadStatus to get the status of this
command.
This command will cause the active CP to reset and will
require that existing telnet, secure telnet or SSH sessions
be restarted.
Do you want to continue [Y]: y
Server Name or IP Address: 10.255.253.43
User Name: root
File Name: /training/firmware/v6.0.0
Password:
Firmwaredownload has started on Standby CP. It may take up
to 30 minutes.
Firmwaredownload has completed successfully on Standby CP.
Standby CP reboots.
Standby CP booted up.
Standby CP booted up with new firmware.
Revision 1008
8 - 30

The firmware download process for a Brocade Director is more complex than for
other Brocade switches. It has dual Control Processor cards (CP0 and CP1) with
one performing an active role and the other a standby role.
Each
E
h CP card
d has
h a Compact
C
t Flash
Fl h card
d that
th t h
holds
ld the
th F
Fabric
b i OS fi
firmware. Th
The
flash is divided into two equal size partitions. The version of the firmware controlling
the switch is in the Primary Partition and another version of firmware is kept in the
Secondary Partition. The version in the Secondary can be the same as the Primary
or different.
It is these factors that make the firmware process more complex. The following
slides will illustrate the individual steps that occur when a firmwaredownload
command is issued.
Revision 1008
8 - 31

The firmwaredownload command is issued from the Active CP to download

Fabric OS version v6.0.0.
The default firmwaredownload options are automatically executed. The default
settings are to auto-reboot
a to reboot and a
auto-commit.
to commit If the -s option is used
sed and a
auto-boot
to boot
mode is not enabled, a reboot command must be issued to activate the downloaded
firmware. If the -s option is used and auto-commit mode is not enabled, a
firmwarecommit command must be issued to replicate the downloaded firmware
to the Secondary Partition.
Current status:
The
Th A
Active
ti CP iis CP0
CP0; th
the St
Standby
db CP iis CP1
CP1.
Firmware FOS A is in the Primary Partition and is committed to the Secondary
Partition on both CP cards.
The firmwareshow command issued from the current standby CP will have
the following output (notice that the Secondary Partition is being written):
RSL_48K8:admin> firmwareshow
Local CP (Slot 6, CP1): Standby
Primary partition:
v5.3
Secondary Partition: Version is not available
Remote CP (Slot 5, CP0): Active
Primary partition:
v5.3
Secondary Partition: v5.3
Revision 1008
8 - 32

The Active CP instructs the Standby CP to perform a firmware download of version

v6.0.0.
The firmware is downloaded into the Secondary Partition. Upon completion, the
Secondary Partition becomes the Primary and the Primary Partition becomes the
Standby.
If the Standby CP does not respond within 10 minutes, the Active CP will time out
and abort the command.
As mentioned above, we perform all Brocade Directors firmware downloads to the
Standby CP card.
Before the firmware download process begins
begins, Fabric OS checks to see that
both CP cards are running the same version of Fabric OS. If they are running
different versions, you will have to update each CP card separately.
Before proceeding from this step, new firmware should be downloaded to the
standby CP. The Standby CP should send a status=OK to the Active CP, Active
CP will respond with reboot instructions and then the Standby will begin the
reboot.
The results of the firmwaredownloadstatus command, issued from the standby
CP appear as follows:
RSL_48K8:admin> firmwaredownloadstatus
[0]: Thu Aug 28 20:01:53 2005
cp0: Firmwaredownload has started on Standby CP. It may take up to 30
minutes.
Revision 1008
8 - 33

Upon successful completion of the firmware download, the Standby CP returns

status to the Active CP.
The results of the firmwaredownloadstatus command issued from current
St db CP are:
Standby
[0]: Thu Aug 28 20:01:53 2005
cp0: Firmwaredownload has started on Standby CP. It may take
up to 30 minutes.
[1]: Thu Aug 28 20:07:15 2005
cp0: Firmwaredownload has completed successfully on Standby
CP.
Revision 1008
8 - 34

If the return status is OK, the Active CP instructs the Standby CP to reboot. After
reboot, version 6.0.0 is now running on CP1 (and is still the Standby).
If it doesnt receive a heartbeat from the Standby CP within 4 minutes, the Active
CP will
ill time
ti
outt and
d abort
b t the
th command.
d
The results of the firmwaredownloadstatus command issued from current
Standby CP are:
[0]: Thu Aug 28 20:01:53 2005
minutes.
minutes
[1]: Thu Aug 28 20:07:15 2005
cp0: Firmwaredownload has completed successfully on Standby CP.
[2]: Thu Aug 28 20:07:18 2005
cp0: Standby CP reboots.
Revision 1008
8 - 35

After reboot, the Standby CP synchronizes with the Active CP by a heartbeat signal.
The output of the firmwaredownloadstatus command is:
[0]: Thu Aug 28 20:01:54 2005
cp0: Firmwaredownload has started on Standby CP. It may take
up to 30 minutes.
[1]: Thu Aug 28 20:07:15 2005
cp0: Firmwaredownload has completed successfully on Standby
CP.
[2]: Thu Aug 28 20:07:18 2005
[3]: Thu Aug 28 20:10:13 2005
cp0: Standby CP booted up
Revision 1008
8 - 36

The Active CP is ready to initiate a failover to the Standby CP.
Revision 1008
8 - 37

Failover will cause the Standby CP to become the Active and the Active CP to
become the Standby. Notice the hashow, still synchronized output:
RSL_48K8:admin> hashow
Local
L
l CP (Sl
(Slot
t 5
5, CP0)
CP0): St
Standby
db
Notice firmwareshow, issued from standby CP shows the new Active CP, Primary
Partition and Secondary Partition firmware status:
Local
l C
CP (Sl
(Slot 5
5, C
CP0):
0) S
Standby
db
Primary partition:
v5.3
Secondary Partition:
Version is not available

Primary partition:
v6.0.0
v5.3
H
Here
iis the
h same command
d iissued
d ffrom A
Active
i CP
CP:
Primary partition:
v6.0.0
v5.3
Remote CP (Slot 5, CP0): Standby
Revision 1008
Primary partition:
v5.3
Version is not available
8 - 38

Note: From this slide forward, the Active CP refers to CP1 and the Standby CP
refers to CP0.
The Active CP waits for the Standby CP to synchronize with a heartbeat signal. This
iindicates
di t ffailover
il
h
has completed.
l t d N
Notice
ti th
the fifirmware d
download
l d status
t t att thi
this time:
ti
[0]: Thu Aug 28 20:01:53 2005
minutes.
[1]: Thu Aug 28 20:07:15 2005
[2]: Thu Aug 28 20:07:18 2005
[3]: Thu Aug 28 20:10:14 2005
cp0: Standby CP booted up.
[4]: Thu Aug 28 20:13:04 2005
cp1: Active CP forced failover succeeded. Now this CP becomes Active.
Revision 1008
8 - 39

The Active CP instructs the Standby CP to perform a firmware download of version

v6.0.0.
The firmware is downloaded into the Secondary Partition. Upon completion, the
S
Secondary
d
P
Partition
titi becomes
b
the
th Primary
Pi
and
d the
th Primary
Pi
Partition
P titi becomes
b
the
th
Secondary.
[0]: Thu Aug 28 20:01:54 2005
minutes.
[1]: Thu Aug 28 20:07:15 2005
[2]: Thu Aug 28 20:07:18 2005
[3]: Thu Aug 28 20:10:13 2005
[4]: Thu Aug 28 20:13:04 2005
[5]: Thu Aug 28 20:13:07 2005
minutes.
Revision 1008
8 - 40

Upon successful completion of the firmware download, the Standby CP returns

status to the Active CP. Current outputs as this process occurs:
HA enabled, Heartbeat Up, HA State not in sync
Revision 1008
8 - 41

After reboot, version v6.0.0 is now running on CP0.

[0]: Thu Aug 28 20:01:54 2005
cp0:
0 Firmwaredownload
i
d
l d h
has started
d on Standby
db CP. It may take
k up to 30
minutes.
[1]: Thu Aug 28 20:07:15 2005
[2]: Thu Aug 28 20:07:18 2005
[3]: Thu Aug 28 20:10:13 2005
[4]: Thu Aug 28 20:13:04 2005
[5]: Thu Aug 28 20:13:07 2005
minutes.
[6]: Thu Aug 28 20:18:41 2005
[7]: Thu Aug 28 20:18:43 2005
[8]: Thu Aug 28 20:22:02 2005
cp1: Standby CP booted up with new firmware.
Revision 1008
8 - 42

The firmware in the Primary Partition on the Standby CP is committed to the

Secondary Partition.
[0] Th
[0]:
Thu A
Aug 28 20
20:01:54
01 54 2005
minutes.
[1]: Thu Aug 28 20:07:15 2005
[2]: Thu Aug 28 20:07:18 2005
[3]: Thu Aug 28 20:10:13 2005
[4]: Thu Aug 28 20:13:04 2005
[5]: Thu Aug 28 20:13:07 2005
p
Firmwaredownload has started on Standby
y CP. It may
y take up
p to 30
cp1:
minutes.
[6]: Thu Aug 28 20:18:41 2005
[7]: Thu Aug 28 20:18:43 2005
[8]: Thu Aug 28 20:22:02 2005
cp1: Standby CP booted up with new firmware.
[9]: Thu Aug 28 20:22:05 2005
cp1: Firmwarecommit has started on both Active and Standby CPs.
Revision 1008
8 - 43

The results of firmwaredownload are:

RSL_48K8:admin> firmwareshow -v
Slot Name
Appl
Primary/Secondary Versions
Status
------------------------------------------------------6
CP0
FOS
v6.0.0a
Active
v6.0.0a
Co-FOS
v6.0.0a
v6.0.0a
CP1
FOS
v6.0.0a
Standby
v6.0.0a
Co-FOS
v6.0.0a
v6.0.0a
Revision 1008
8 - 44

Revision 1008
8 - 45

Prm Part = Primary Partition

Sec Part = Secondary Partition
FW A = Old firmware
FW B = New firmware
The total time to upgrade the firmware is up to 30 minutes (about the same as the
48000).
Revision 1008
8 - 46

Revision 1008
8 - 47

Revision 1008
8 - 48

Revision 1008
8 - 49

Revision 1008
8 - 50

Revision 1008
8 - 51

Revision 1008
8 - 52

Revision 1008
8 - 53

Revision 1008
8 - 54

When the s argument is added to the firmwaredownload command, two

additional options are available:
-b: Autoreboot mode; if included, switch reboots automatically after the firmware
d
download
l d completes.
l t
-n: Disable autocommit mode, so firmware is downloaded only to the secondary
partition.
The example above was recorded on a dual-CP Brocade 48000, running Fabric
OS v6.0.0a. The only argument included at the command line is s, so you are
prompted to provide the FTP server IP address, user name, firmware filename,
and
d password,
d as wellll as th
the ttwo single-CP
i l CP options
ti
available
il bl in
i Fabric
F b i OS v5.x+.
5
Revision 1008
8 - 55

The example above is on a dual-CP Brocade 48000. Both CP cards have Fabric OS
v6.0.0 installed on both memory partitions.
The auto-leveling process automatically ensures compatibility between the version
off Fabric
F b i OS running
i on th
the A
Active
ti CP and
d allll off th
the AP bl
blades
d iinstalled
t ll d iin th
the
Director chassis. When the Brocade 48000 boots up or an advanced blade (FR418i, FA4-18 and FC4-16IP) is inserted or powered on, the Active CP determines if
the installed advanced blades are running the same version of Fabric OS as the
active CP. If the Fabric OS versions differ, the Active CP will install the BFOS image
on all installed advanced blades. Auto-leveling occurs automatically, without
administrator intervention.
Auto-leveling occurs:
After CP failover (CP error), including during a dual-CP firmware download
(e.g. firmwaredownload -s)
After hot-plug of an FR4-18i, or FC4-16IP blade or when slotpoweron is
invoked on a slot with these blades
Th
The active
ti CP always
l
updates
d t th
the FC4
FC4-16IP
16IP blade
bl d and
d MAY update
d t th
the FR4
FR418i blade
Auto-leveling is always enabled, and cannot be disabled
Revision 1008
8 - 56

Revision 1008
8 - 57

Revision 1008
8 - 58

This command uploads a current switch configuration to a host file on a FTP server. The
upload process uses either FTP (File Transfer Protocol) or the RSHD protocol (TCP service
514). Both of these services are widely available on Unix hosts, but less so on Windows
y reasons it is recommended to remove the switch configuration
g
file from
hosts. For security
the FTP server to a secure location.
The command may be invoked without any parameters, in which case the user is prompted
for input, including choice or FTP or RSHD (VxWorks switches only). RSHD is not an
option on Linux based switches. In the case of VxWorks switches, if invoked from one
command line session (non-interactive) with three parameters then RSHD is used, the
presence of the fourth parameter (FTP password) selects FTP. Here is an example from an
interactive session on a VxWorks switch:
rsl1_st07_b38_1:admin> configupload
Server Name or IP Address [host1] : 10.255.252.50
User Name [none] : anonymous
File Name [config.txt] : config.txt
Protocol (RSHD or FTP) [FTP] : ftp
Password :
upload complete
The upload may fail for many reasons, including:

the host name is not known to the switch
the host IP address can't be contacted
the user doesn't have permission on the host
the user runs a script that prints something at login
the RSHD (VxWorks only) or FTP server isn't
isn t running on the destination host
Note: Opening the ASCII txt files in Microsoft Word could change formatting and cause the
file to become corrupt. WordPad is a better option.
Revision 1008
8 - 59

FOS switches: configupload / configdownload

License information
Port configuration
Security policies
SNMP configuration
Switch configuration parameters
Zoning information
Revision 1008
8 - 60

This command downloads the switch configuration from a host file. The
configuration file may have been generated previously using configupload or
may have been created by a user to download only specific configuration changes.
rsl1_st07_b41_1:admin>
l1 t07 b41 1 d i > switchdisable
rsl1_st07_b41_1:admin> configdownload
Protocol (scp or ftp) [ftp]: ftp
Server Name or IP Address [host]: 10.255.252.50
User Name [user]: configs
File Name [config.txt]: config.txt
Password:
*** CAUTION ***
This command is used to download a backed-up configuration
for a specific switch. If using a file from a different
switch, this file's configuration settings will override
any
y current switch settings.
g
Downloading
g a configuration
g
file, which was uploaded from a different type of switch,
may cause this switch to fail. A switch reboot might be
required for some parameter changes to take effect.
Do you want to continue [y/n]: y
download completed
completed.
Note: You must re-enable the switch to see the effects of the configuration changes.
Revision 1008
8 - 61

Revision 1008
8 - 62

Revision 1008
8 - 63

Revision 1008
8 - 64

Revision 1008
8 - 65

Revision 1008
8 - 66

No
When
Scenario of
Error
Error Handling
User action
During
step 1
During
downloading to
the main CPU of
the standby CP, if
an error occurs
and the main CPU
reboots.
1)When the main CPU boots up,

firmwaredownload is aborted.
2)firmwarecommit will be initiated on the
main CPU and the original firmware is
restored on that CPU.
3)Both CPUs on both CPs will have the
original firmware.
Restart
firmwaredownload
after the repair is
done.
During
step 2
During
downloading to
the co CPU of the
standby CP, if an
error occurs and
the co CPU
reboots.
1)When the co CPU boots up,

firmwaredownload is aborted.
2)firmwarecommit will be initiated on
both CPUs and the original firmware is
stored on both CPUs.
original firmware.
Restart
firmwaredownload
after the repair is
done.
During
step 1
or 2
During
downloading to
any of the CPUs on
the standby CP, if
the downloading
takes too long and
exceeds the 30
minute timeout.

both CPU on the standby CP and the
original firmware is restored on both
CPUs.
original firmware.
Restart
firmwaredownload
after the repair is
done.
During
step
1,2
Active CP fails over

during the
downloading to
any of the CPUs on
the standby CP.

both CPU on the new active CP and the
original firmware is restored on both
CPUs.
original firmware.
Restart
firmwaredownload
after the repair is
done.
During
step 4
If the standby CP
failed to reboot or
unable to
synchronize with
the active CP.
1)Active CP will wait for 10 minutes and

abort firmwaredownload.
2)If the standby CP boots up,
firmwarecommit will start on both CPUs
on the standby CP.
3)Both CPUs on the standby CP will have
the new firmware, and both CPUs on the
active CP will have the old firmware.
Determine why the

CPs fail to gain HA
sync and remedy it
before restarting
firmwaredownload
During
step 6
When downloading
to the main CPU of
the standby CP, if
an error occurs
and the main CPU
reboots.
b t
1)When the main CPU on the standby CP

boots up, firmwaredownload is aborted.
both CPUs on both CPs.
th old
the
ld fi
firmware and
db
both
th CPU
CPUs on the
th
active CP will have the new firmware.
Restart
firmwaredownload
after the repair is
done.
Revision 1008
8 - 67

No
When
Scenario of
Error
Error Handling
During
step 7
When
downloading to
the co CPU of the
standby CP, if an
error occurs and
the co CPU
reboots.
1)When the co CPU on the standby CP boots

up, firmwaredownload is aborted.
both CPUs on both CPs.
the old firmware and both CPUs on the
Restart
firmwaredownload
after the repair is
done.
During
step 6
or 7
When
downloading to
any of the CPUs
on the standby
CP, downloading
takes too long and
exceeds the 30
minute timeout.

both CPU on both CPs.
the old firmware and both CPUs on the
Restart
firmwaredownload
after the repair is
done.
During
step 6
or 7
The active CP fails

over during the
downloading to
any of the CPUs
on the standby
CP.
1)The standby CP will become the new

active CP.
both CPUs on the active CP and the original
firmware is restored on both CPUs.
3)When
)
the standby
y CP boots,,
firmwarecommit will be initiated on both
CPUs on the standby CP and they will have
the new firmware.
the new firmware and both CPUs on the
active CP will have the old firmware.
Restart
firmwaredownload
after the repair is
done.
10
During
step 8
If the standby CP
failed to reboot or
unable to
synchronize with
the active CP.
1)Active CP will wait for 10 minutes and

abort firmwaredownload.
both CPUs on the active CP.
3)If the standby CP boots up,
firmwarecommit will start on both CPUs on
the standby CP.
4)Both CPUs on the both CPs will have the
new firmware
Determine why the

CPs fail to gain HA
sync and remedy it
before restarting
firmwaredownload
11
During
step 10
If commit fails
The affect CPUs will have different versions

of firmware on its partitions. An error
message is logged.
It requires fixing the

switch manually. It
rarely happens.
Revision 1008
User action
8 - 68

Revision 1008
8 - 69

DCX10:admin> firmwaredownload
Server Name or IP Address: 10.255.252.50
User Name: upd207
File Name: /firmware/v6.0.0
Network Protocol(1-auto-select, 2-FTP, 3-SCP) [1]: 2
Password: xyz
Checking system settings for firmwaredownload...
Protocol selected: FTP
Trying address
address-->AF_INET
>AF INET IP: 10
10.255.252.50,
255 252 50 flags : 2
System settings check passed.
This command will upgrade the firmware on both CP blades. If you want to
upgrade firmware on a single CP only, please use -s option.
You may run firmwaredownloadstatus to get the status of this
command.
This command will cause a warm/non-disruptive boot on the active CP,
but will require that existing telnet, secure telnet or SSH sessions
be restarted.
Do you want to continue [Y]: Y

Firmware is being downloaded to standby CP. This step may take up to 30
minutes.
Firmware is being downloaded to Standby CP. Please wait...
Completed download of 2/104 packages (1%). Please wait...
C
Completed
l t d d
download
l d of
f 8/104 packages
k
(7%)
(7%). Pl
Please wait...
it
<Trunked Output>.
Completion
.
.
percentage
.
is displayed
.
<Trunked Output>
Firmware has been downloaded successfully to Standby CP.
Standby CP is going to reboot with new firmware.
Standby CP booted successfully with new firmware.
Revision 1008
8 - 70

DCX10:admin> firmwaredownloadstatus
[1]: Wed Oct 24 22:13:41 2007 (Step 1: Only access to the FTP Server).
Slot 6 (CP0, active): Firmware is being downloaded to standby CP. This step
may take up to 30 minutes.
[2]: Wed Oct 24 22:18:56 2007
Slot 6 (CP0, active): Firmware has been downloaded successfully to Standby CP.
[3]: Wed Oct 24 22:19:01 2007 (Step 3 and 4)
Slot 6 (CP0, active): Standby CP is going to reboot with new firmware.
[4]: Wed Oct 24 22:20:19 2007
Slot 6 (CP0, active): Standby CP booted successfully with new firmware.
[5]: Wed Oct 24 22:21:23 2007 (Step 5)
Slot 7 (CP1, active): Forced failover succeeded. New Active CP is running new
firmware
[6]: Wed Oct 24 22:31:30 2007 (Step 6: This is done via the internal network)
Slot 7 (CP1
(CP1, active): Firmware is being downloaded to standby CP
CP. This step
may take up to 30 minutes.
[7]:
Slot
[8]:
Slot
[9]
[9]:
Slot
Wed Oct
7 (CP1,
Wed Oct
7 (CP1,
W
Wed
d O
Oct
t
7 (CP1,
24 22:35:07 2007
active): Firmware has been downloaded successfully on Standby CP.
24 22:35:11 2007 (Steps 8 and 9)
active): Standby CP reboots.
24 22
22:36:27
36 27 2007
active): Standby CP booted successfully with new firmware.
[10]: Wed Oct 24 22:36:30 2007 (Step 10)

Slot 7 (CP1, active): Firmware commit operation has started on both active and
standby CPs.
[11]: Wed Oct 24 22:36:30 2007
Slot 7 (CP1, active): The firmware commit operation has started. This may take
up to 10 minutes.
[12]: Wed Oct 24 22:39:04 2007
Slot 7 (CP1, active): The commit operation has completed successfully.
[13]: Wed Oct 24 22:39:05 2007
Slot 7 (CP1, active): Firmware commit operation has completed successfully on
both CPs.
[14]: Wed Oct 24 22:39:05 2007
Slot 7 (CP1, active): Firmwaredownload command has completed successfully. Use
firmwareshow to verify the firmware versions.
Revision 1008
8 - 71

DCX10:admin>errdump
<Trunked
Trunked Output>
Output
2007/10/25-01:08:09, [SULB-1001], 9349,, WARNING, ?,
Firmwaredownload command has started.
2007/10/25-01:08:09, [SULB-1036], 9350,, INFO, ?,
The current Version: Fabric OS v6.0.0_beta1
2007/10/25-01:10:48, [FSSM-1002], 9398,, INFO, ?, HA State is in
sync.
2007/10/25-01:11:38, [SULB-1007], 9399,, INFO, ?, Standby CP
reboots. (Step 4)
2007/10/25-01:11:38, [FSSM-1003], 9400,, WARNING, ?, HA State out of
sync.
2007/10/25-01:11:47, [HAM-1005], 9401,, INFO, ?, HeartBeat Miss
reached threshold.
2007/10/25-01:11:47, [HAMK-1004], 9402,, INFO, ?, Resetting standby
CP (double reset may occur)
2007/10/25-01:11:47, [EM-1033], 9403,, ERROR, ?, CP in Slot 7 set to
faulty because CP ERROR asserted. (Message is normal, this is the CP failover step 5.)
2007/10/25-01:12:02, [EM-1047], 9404,, INFO, ?, CP in slot 7 not
faulty, CP ERROR deasserted.
2007/10/25-01:12:50, [FSSM-1002], 9405,, INFO, ?, HA State is in
sync.
2007/10/25-01:12:56, [SULB-1008], 9406,, INFO, ?, Standby CP booted
successfully with new firmware. (Step 9)
2007/10/25-01:12:59, [SULB-1003], 9407,, INFO, ?, Firmwarecommit has
started. (Start of step 10)
2007/10/25 01 15 25 [SULB
2007/10/25-01:15:25,
[SULB-1004],
1004] 9408
9408,, INFO
INFO, ?
?, Fi
Firmwarecommit
it h
has
completed. (Completion of step 10)
2007/10/25-01:15:25, [SULB-1002], 9409,, INFO, ?,
Firmwaredownload command has completed successfully.
2007/10/25-01:15:25, [SULB-1036], 9410,, INFO, ?,
The new Version: Fabric OS v6.0.0
<Trunked Output>
Revision 1008
8 - 72

Note if the firmware versions do not match you will see the following message at the
bottom of this output:
WARNING: The local CP and remote CP have different versions of
firmware please retry firmwaredownload command
firmware,
command.
If a firmware download was just done suggest waiting several minutes for the
firmware to be copied to all partitions. Suggest waiting for about 30 minutes: If the
firmware still has not been copied to all partitions then I would look at the
firmwaredownloadstatus and errdump output to see what happened.
Revision 1008
8 - 73

Revision 1008
8 - 74

Student Guide - CFP 270 - Brocade 8 GB/s Introduction To Administration and Theory

Hochgeladen von

Dokumentinformationen

Originaltitel

Copyright

Verfügbare Formate

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

Verfügbare Formate

Student Guide - CFP 270 - Brocade 8 GB/s Introduction To Administration and Theory

Hochgeladen von

Copyright:

Verfügbare Formate

CFP270:

Corporate Headquarters - San Jose, CA USA

2008 Brocade Communications Systems, Inc. All Rights Reserved.

FR4-18i Blade (DCX and 48000) ................................................................................................................ 41

Module 4 Installation and Configuration

Module 6 FSPF Routing

Exchange-based Routing and DLS............................................................................................................. 19

Module 8 Administration and Maintenance

CFP270 Brocade 8 Gbit/sec

CFP270 Brocade 8 Gbit/sec

CFP270 Brocade 8 Gbit/sec

CFP270 Brocade 8 Gbit/sec

CFP270 Brocade 8 Gbit/sec

CFP270 Brocade 8 Gbit/sec

CFP270 Brocade 8 Gbit/sec

CFP270 Brocade 8 Gbit/sec

CFP270 Brocade 8 Gbit/sec

CFP270 Brocade 8 Gbit/sec

CFP270 Brocade 8 Gbit/sec

CFP270 Brocade 8 Gbit/sec

CFP270 Brocade 8 Gbit/sec

This page is intentionally blank.

CFP270 Brocade 8 Gbit/sec

Hardware Product Features

CFP270 Brocade 8 Gbit/sec

Hardware Product Features

CFP270 Brocade 8 Gbit/sec

Hardware Product Features

CFP270 Brocade 8 Gbit/sec

Hardware Product Features

CFP270 Brocade 8 Gbit/sec

Hardware Product Features

CFP270 Brocade 8 Gbit/sec

Hardware Product Features

CFP270 Brocade 8 Gbit/sec

Hardware Product Features

Footnote 1: FL_Ports are not supported on the 48-port blade.

CFP270 Brocade 8 Gbit/sec

Hardware Product Features

Footnote 1: FICON support is on a platform / blade basis.

CFP270 Brocade 8 Gbit/sec

Hardware Product Features

CFP270 Brocade 8 Gbit/sec

Hardware Product Features

CFP270 Brocade 8 Gbit/sec

Hardware Product Features

CFP270 Brocade 8 Gbit/sec

Hardware Product Features

CFP270 Brocade 8 Gbit/sec

Hardware Product Features

CFP270 Brocade 8 Gbit/sec

Hardware Product Features

CFP270 Brocade 8 Gbit/sec

Hardware Product Features

CFP270 Brocade 8 Gbit/sec

Hardware Product Features

CFP270 Brocade 8 Gbit/sec

Hardware Product Features

CFP270 Brocade 8 Gbit/sec

Hardware Product Features

CFP270 Brocade 8 Gbit/sec

Hardware Product Features

Footnote 1: The Service IP port is for future use.