Sie sind auf Seite 1von 130

CFA 250

Revision 0312

Preparing for the 16 Gbps BCFA Exam

CFA 250

Revision 0312

Preparing for the 16 Gbps BCFA Exam

CFA 250

Revision 0312

Preparing for the 16 Gbps BCFA Exam

CFA 250

Revision 0312

Preparing for the 16 Gbps BCFA Exam

CFA 250

Revision 0312

Preparing for the 16 Gbps BCFA Exam

CFA 250

Revision 0312

Preparing for the 16 Gbps BCFA Exam

CFA 250

Preparing for the 16 Gbps BCFA Exam

FC-0 and FC-1 levels specify physical and data link functions needed to physically send
data from one port to another.
FC-0 level specifications include information about feeds and speeds.
FC-1 level contains specifications for 2, 4, 8 Gbps 8b/10b encoding, ordered set and
link control communication functions. 10 and 16 Gbps communication uses 64b/66b
encoding.
FC-2 level specifies content and structure of information along with how to control and
manage information delivery. This layer contains basic rules needed for sending data
across the network. This includes: (1) how to divide the data into frames, (2) how much
data should be sent at one time before sending more (flow control), and (3) where the
frame should go. It also includes Classes of Services, which define different
implementations that can be selected depending on the application.
FC-3 level defines advanced features such as striping (to transmit one data unit across
multiple links) and multicast (to transmit a single transmission to multiple destinations)
and hunt group (mapping multiple ports to a single node). While the FC-2 level concerns
itself with the definition of functions with a single port, the FC-3 level deals with
functions that span multiple ports.
FC-4 level provides mapping of Fibre Channel capabilities to pre-existing protocols, such
as IP, SCSI, or ATM.

Revision 0312

CFA 250

Preparing for the 16 Gbps BCFA Exam

For a switch port that goes through port initialization, it arrives at an ending status of
F_Port, FL_Port or E_Port.
Footnote1: Loop ports are not supported on Condor3 ASICs.
Footnote 2: EX and VEX_Ports allow communication between devices in independent
fabrics without having to merge the fabrics. This is done through the use of FC-FC
Routing. To learn more about FC-FC routing and fabric extension solutions please refer
to CFP300 or RE300 courses.

Revision 0312

CFA 250

Preparing for the 16 Gbps BCFA Exam

When a node attaches to the fabric, it must receive a unique 24-bit address. The
network address is a three-byte address based upon the Domain ID, the Area ID and, if a
loop device, its AL_PA. This address is the source address and is used for routing data
thru the fabric from one device to another.
Footnote 1: There are other variations of addressing that are beyond the scope of this
course.
Footnote 2: XX will be 00 for Fabric OS switches.
Fabric-attached devices use an address format of DD AA 00. This is the address of
any Fabric-attached device that has logged into the fabric as point-to-point.
Public Loop attached devices use an address format of DD AA PP. The DD AA bytes
of the address come from the fabric login process and the PP byte is assigned during
FC_AL initialization.
NPIV attached devices use an address format of DD AA PP. The DD AA bytes of the
address come from the fabric login process and the PP byte is assigned during login
process. More information on NPIV at the end of this module.

Revision 0312

CFA 250

Preparing for the 16 Gbps BCFA Exam

Shared Area IDs will use the Node Address to allow 384 ports to be addressed in a
single domain.

Footnote 1: The FC8-48 blade does not use shared areas when installed into a DCX-4S
since the total port count in the domain would not exceed 256.

Revision 0312

10

CFA 250

Preparing for the 16 Gbps BCFA Exam

Footnote1: The flow chart outlines the Fabric Initialization process supported by Fabric OS, but not all
Brocade hardware, such as the Condor3, support loop.

A Universal Port (U_Port) is the initial state of a port. (State 1)


Is something connected (sending a light/electrical signal) to the port? If yes, continue. (Transition 1)
U_Port starts mode detection process by transmitting at least 12 LIP(F7) Primitive Sequences. (Transition
2)
If at least 3 consecutive LIP Primitive Sequences are received, then the port enters OPEN_INIT state
and attempts FC-AL loop initialization. (State 2)
If LIP Primitive Sequences are not received, the U_Port attempts OLD_PORT initialization by taking
the link down then transmitting NOS primitives. If Link Initialization Protocol fails after 1 retry or LIP
received after 1 second, go to FC-AL initialization. (Transition 2)
When operating in the FL_Port mode, a U_Port will try the loop initialization procedure three times.
If these fail, the port will be marked as faulty. To ensure N_Port, reinitialize the port and the switch
port will cut the laser forcing a loss of signal state for at least 20 s. Then the switch port will bring
back the laser and issue NOSs. (Transition 2)
If the attached device is not loop it continues into the G_Port stage. A device can be a switch, target
(usually storage), or an initiator (usually a host). (State 3)
If the attached device is a target or initiator it changes its port state from G_Port to an F_Port. (State 5).
If the attached device is a switch then it changes its port state from a G_Port to an E_Port. (State 4).

F_Ports and E_Ports continue to login to the fabric which is explained later in this module.

Revision 0312

11

CFA 250

Preparing for the 16 Gbps BCFA Exam

Every switch has reserved 24-bit called Well-Known Addresses. The services residing at
these addresses provide a service to either nodes or management applications in the
fabric. Below are other servers not listed in the table:
FFFFF6 Clock Synchronization Server: Clock synchronization over Fibre Channel is
attained through a Clock Synchronization Server that contains a reference clock. The
server synchronizes clients clocks to the reference clock on a periodic basis, using
either primitive signals or ELS frames.
FFFFF7 Security Server: The security-key distribution service offers a mechanism for the
secure distribution of secret encryption keys.
FFFFF8 Alias Server: The Alias Server manages the registration and deregistration of
alias IDs for both hunt groups and multicast groups. The Alias Server is not involved in
the routing of frames for any group.
FFFFFB Time Server: The Time Server sends to the member switches in the fabric the
time on either the principal switch or the primary FCS switch.
FFFFFD Fabric Controller: The fabric controller provides state change notifications to
registered nodes when a change in the fabric topology occurs.

FFFFFF Broadcast Server: When a frame is transmitted to this address, the frame is
broadcast to all operational N and NL_Ports.

Revision 0312

12

CFA 250

Revision 0312

Preparing for the 16 Gbps BCFA Exam

13

CFA 250

Preparing for the 16 Gbps BCFA Exam

Provides a more robust credit recovery for lost buffer credits or frames on long distance
E_Ports on between two non-Condor3 ASIC switches or backbones. If a lost buffer credit
or frame is detected, the switch performs a Link Reset (LR) to recover the lost credit.
This is done by sending an LR to the target switch which sends back an LRR (Link Reset
Response). Because this happens on an E_Port, the link does not reset, only the BB
(frame and credit loss) counters at both ends of the link are reset.

16 Gbps QSFP
and cable

Revision 0312

14

CFA 250

Preparing for the 16 Gbps BCFA Exam

Footnote 1: Hardware detection in the Condor3 ASIC


Footnote 2: If one side is a Condor2 it uses port-level credit recovery.

Revision 0312

15

CFA 250

Preparing for the 16 Gbps BCFA Exam

This is the case where credits are simply not returned/permanently lost. This is different
from a latency bottleneck where credits may eventually return slowly.

Footnote 1: ASICs hold a frame for 500ms if it cannot forward the frame. If the frame is
discarded the ASIC should return the credit. The 600ms time is past the time the switch
has to return the credit by about 100ms.

portCfgCreditRecovery
Enables or disables credit recovery on a port.
portcfgcreditrecovery --enable [slot/]port
portcfgcreditrecovery --disable [slot/]port
Use this command to enable or disable credit recovery on a port. The credit recovery
feature enables credits or frames to be recovered. Only ports configured as long
distance ports can utilize the credit recovery feature. The default credit recovery
configuration is enabled.

Revision 0312

16

CFA 250

Revision 0312

Preparing for the 16 Gbps BCFA Exam

17

CFA 250

Preparing for the 16 Gbps BCFA Exam

For additional information on Brocade transceiver optics visit our web site:
http://www.brocade.com/products/all/transceivers/product-details/transceivermodules/index.page.
SFPs (Small Form Pluggable) come in a variety of speeds with a number of different
standards in use for different applications. The SFP/SFP+ standards define the physical
specifications for the transceiver.
In addition to the physical specifications the SFP+ standard is used for transceivers that
are capable of 10 Gbps or greater transmission rates.
Footnote 1: QSFPs (Quad SFPs) are a new transceiver design that allows for four
separate data paths through the transceiver and across an industry standard cable.
Currently these are only being used in the 16 Gbps Brocade Backbones which are
covered in the following module.

16 Gbps QSFP
and cable

Revision 0312

18

CFA 250

Preparing for the 16 Gbps BCFA Exam

Switch1:admin> sfpshow -all


<truncated output>
============
Slot 3/port 0:
============
QSFP No. 0 Channel No:0
Identifier:
13
QSFP+
Connector:
12
MPO Parallel Optic
Transceiver: 0000000000000000 16_Gbps id
Encoding:
5
64B66B
Baud Rate:
140 <units 100 megabaud>
Length 9u:
0
<units km>
Length 9u:
25
<units 100 meters>
Length 50u:
0
<units 10 meters>
Length 62.5u: 0
<units 10 meters>
Length Cu:
0
<units 1 meter>
Vendor Name: Brocade
Vendor OUI:
00:15:1e
Vendor PN:
57-0000090-01
Vendor Rev:
A
Wavelength:
850 <units nm>
Options:
00000fde
Max Case Temp:
70 <C>
Device Tech:
0x00
Serial NO:
HTA111111002103
Date Code:
110317
DD Type:
0x8
Enh Options
0x0
Status/Ctrl: 0x0
Alarm flags [0,1] = 0x0, 0x0
Warn Flags [0,1] = 0x0, 0x0
Alarm
Warn
low
high
low
high
Temperature: 29
Centigrade
-5
85
0
80
Current:
6.748 mAmps
0.500
10.000
1.000
9.500
Voltage:
3271.9 mVolts
2970.0
3630.0 3134.9
3465.0
RX Power:
-6.9
dBm <202.8uM> 44.6 uW2187.8uW 112.2 uW
1737.8 uW
State transitions: 2
<truncated output>

Revision 0312

19

CFA 250

Preparing for the 16 Gbps BCFA Exam

The table above offers a side-by-side comparison of the different types of Brocade Fibre
Channel optics available. For each optic the supported speeds, cable types, and
maximum distance are listed. To get a full list of compatible optics please visit the
Brocade website and download the Brocade Compatibility Matrix?

Footnote 1: The distance shown in this chart represents the maximum distance, with
high quality cabling, that is available at the maximum line speed of the transceiver.
Distances can actually be increased beyond what is listed by reducing transmission
speed. For a full list of supported distances, speeds, and cable types for any transceiver
please visit our web site:
http://www.brocade.com/products/all/transceivers/product-details/transceivermodules/features.page.

Footnote 2: The 4x16 Gbps QSFP is currently only used on the DCX8510-4 and DCX
8510-8 Backbone switches for ICL links between chassis. These switches will be
covered in greater detail in the next module.

Revision 0312

20

CFA 250

Preparing for the 16 Gbps BCFA Exam

Brocade HBAs, CNAs, and Fabric Adapters are design to support a wide range of
networking environments. They can be broken into three main categories: Fibre Channel,
Ethernet, and converged. The media used will depend on the application that the
adapter is being used for. For example, if you wish to use a Fabric Adapter in an Ethernet
or converged environment you will install Ethernet style SR or LR transceivers.
Converged networking is a new data center technology that combines the Fibre Channel
protocol with Ethernet frames for transport. For more information on converged
technologies please see the following Brocade University courses:
http://www.brocade.com/education/product-training/index.page
WBT: FCoE 101, VCS 101, VDX 110
ILT/VCT: FCoE 200, CEF 200

Fibre Channel

Ethernet

Converged

HBA
CNA
Fabric Adapter
Revision 0312

21

CFA 250

Preparing for the 16 Gbps BCFA Exam

Additional Condor3 Features


768 Gbps of bandwidth per ASIC, 420 million frames switched per second
Layer 2 latency of 800ns without FEC (F_Port to F_Port)
Layer 2 latency of 1.2s with FEC (E_Port to E_Port)
Up to 5,000 Km distance at 2 Gbps
Integrated in-flight encryption and compression
Auto-detection of fill word on links (IDLE vs ARB)
Forward Error Correction (FEC)
Forward Error Correction (FEC) can only be enabled on E_Ports. FEC is a system of
error control for data transmissions, whereby the sender adds error-correcting code
(ECC) to its transmission. This allows the receiver to detect and correct errors without
the need to ask the sender for additional data. The Condor3 FEC implementation can
enable corrections of up to 11 error bits in every 2,112 bit transmission. FEC can be
enabled or disabled using the portcfgfec command.
Footnote 1: FL_Ports are not supported on the Condor3 ASIC.
Footnote 2: 2 Gbps speeds requires the use of an 8 Gbps Brocade branded SFP. 1 Gbps
speeds are not supported.
Footnote 3: The number of user accessible buffer credits vary depending on the type of
platform the ASIC is installed in. Use the portbuffershow command to see available
credits.

Revision 0312

22

CFA 250

Preparing for the 16 Gbps BCFA Exam

The Brocade switches, directors, backbones and blades are discussed later.
Footnote 1: On the 16-port and 32-port blades, two of these port groups (2 x 8 = 16 total
per ASIC) are used for external ports and the other three port groups are used for
internal ports. On the 48-port blade, three of these port groups (3 x 8 = 24 total per
ASIC) are used for external ports and the other two port groups are used for internal
ports. This will be discussed later in the course. The FC8-64 uses four Condor2 ASICs
with two port groups per ASIC dedicated to user ports.
Footnote 2: The number of user accessible buffer credits vary depending on the type of
platform the ASIC is installed in and how it is being used.

Condor2 ASICs are used in the Brocade 5100, DCX, DCX-4S, 8000, Brocade Encryption
Switch, FC8-16, FC8-32, FC8-48, and FC8-64.
Condor3 ASICs are used in the Brocade 6510, CR16-8, CR16-4, FC16-32, FC16-48.
GoldenEye2 ASICs are used in the Brocade 300, 5300, and 7800.

Revision 0312

23

CFA 250

Preparing for the 16 Gbps BCFA Exam

The two rack kit options for the Brocade 6510 use rails that are slimmer than standard rails to
accommodate the slightly wider chassis. Be sure to use one of these kits. Do not use standard
rails to install the Brocade 6510 in a rack, they will not fit with the switch.
Footnote 1: Power supplies are available in front-to-back or back-to-front airflow options.
Footnote 2: The Brocade 6510 comes with 24 licensed ports. Additional ports are available in
12 port increments (36 and 48 ports).
Footnote 3: Supports 2/4/8/10/16 Gbps speeds with the following optics:
16 Gbps optics: 4/8/16 Gbps
8 Gbps optics: 2/4/8 Gbps
10 Gbps FC optics: 10 Gbps
Support for 10 Gbps FC is by using a 10 Gbps FC optic. 10 Gbps Ethernet optics do not
work.
Footnote 4: The Condor3 has a total of 8192 buffer credits, 7712 are available to the user in
the Brocade 6510.
Footnote 5: Brocade 6510 Integrated Routing support requires an Integrated Routing license.
Once the license is installed, you can configure EX_Ports on a per-port basis as needed. For
more information on Integrated Routing refer to CFP 300 Brocade Certified Fabric Professional
(BCFP) 16 Gbps training course.

Supports Virtual Fabrics with up to 4 Logical Switches and data-in-flight encryption and
compression.
For more detailed information about installation techniques and compatible components,
please visit the Brocade website and download this products Hardware Reference Manual
Revision 0312

24

CFA 250

Preparing for the 16 Gbps BCFA Exam

To learn more about Brocade Fabric Adapters visit our Adapters page on the Brocade
website http://www.brocade.com/products/all/adapters/index.page.
Brocade AnyIO Technology
Brocade AnyIO technology is a unique capability that allows a single Brocade 1860
Fabric Adapter to support either native 16 Gbps Fibre Channel or 10 GbE on a port-byport, user selectable basis. Brocade AnyIO technology enables a dual-port adapter to
run 16 Gbps Fibre Channel on one port, and 10 GbE on the other port. In addition, the
Brocade 1860 can run TCP/IP, Fibre Channel over Ethernet (FCoE), and iSCSI
simultaneously on the same 10 GbE port.
A Brocade 1860 adapter port can be configured in any of the following modes:
HBA mode Appears as a 16 Gbps Fibre Channel HBA to the operating system
(OS).
NIC mode Appears as a 10 GbE NIC to the OS. It supports 10 GbE with DCB,
iSCSI, and TCP/IP simultaneously.
CNA mode Appears as two independent devicesa 16 Gbps Fibre Channel HBA
and a 10 GbE NIC to the OS. It supports 10 GbE with DCB, FCoE, iSCSI, and TCP/IP
simultaneously.
Footnote1: HCM can also be used to manage, test and troubleshoot Brocade 4/8G
HBAs and CNAs.

Revision 0312

25

CFA 250

Preparing for the 16 Gbps BCFA Exam

Fabric Adapter Key Features


Brocade AnyIO technology: 16/8/4/2 Gbps Fibre Channel and 10 GbE DCB for
TCP/IP, FCoE, and iSCSI
Single- and dual-port models
Line-rate 16 Gbps Fibre Channel, 1600 MB/sec throughput per port (3200
MB/sec full duplex)
Line-rate 10 GbE performance with stateless networking offloads for the highest
levels of performance and CPU efficiency
Jumbo frame support, up to 9600 bytes
Over 500,000 IOPS per port for storage (Fibre Channel/FCoE/iSCSI)
Brocade Server Application Optimization (SAO): Application-aware Quality of
Service QoS) and N_Port Trunking
Brocade Virtual Machine Optimized Ports (VMOPs): Offload the hypervisor of
network packet classification and sorting tasks
Brocade virtual Fabric Link (vFLink) I/O Virtualization (IOV): Up to eight virtual
adapters with flexible bandwidth allocations
Single-Root I/O Virtualization: Extends Brocade vFLink with up to 255 Virtual
Functions (VFs)
Fully integrated virtual switching with support for Edge Virtual Bridging (EVB)
802.1Qbg, including Virtual Ethernet Bridging (VEB) and Virtual Ethernet Port
Aggregator (VEPA)
Boot-from-Storage Area Network (SAN), including Fabric-based Boot LUN
Discovery for unmatched simplicity
Dynamic Fabric Provisioning (DFP) virtualizes host WWNs to simplify preprovisioning and eliminate time-consuming fabric reconfigurations when replacing
adapters
Pre-boot eXecution Environment (PXE) for booting over a network connection
PCI-Express (PCIe) Gen2 (2.0), x8 lanes, with INTx and MSI-X Support
Low-profile design for industry-standard 1U rack servers (high-profile bracket also
Included)
Multiple media options, including Short Wave-Length (SWL) and Long WaveLength (LWL) optics for Fibre Channel; and Short-Reach (SR), Long-Reach (LR),
and active Twinax copper for 10 GbE
The Brocade 1860 is available in the following models:
Brocade 1860-1F: Single port, one 16 Gbps Fibre Channel SWL SFP+ included
Brocade 1860-2F: Dual port, two 16 Gbps Fibre Channel SWL SFP+ included
Brocade 1860-1P: Single port, one 10 GbE SR SFP+ included
Brocade 1860-2P: Dual port, two 10 GbE SR SFP+ included
Brocade 1860-1C: Single port, no media included
Brocade 1860-2C: Dual port, no media included

Revision 0312

26

CFA 250

Preparing for the 16 Gbps BCFA Exam

The DCX 8510-8 is intended for large enterprise fabric core.


Switchtype 120.X (8 port blade slots)
Footnote 1: You can achieve 512 user ports by using FC8-64 blades; these will only
operate at 8 Gbps. Using FC16-48 blades you can operate at 16 Gbps with 384 ports.
Footnote 2: Blades are mutually exclusive for the FOS v7.0.0 release. You can install one
or the other, but not both at the same time.
For more detailed information about installation techniques and compatible
components, please visit the Brocade website and download this products Hardware
Reference Manual

Revision 0312

27

CFA 250

Preparing for the 16 Gbps BCFA Exam

Field Replaceable Units (FRUs) accessible from the non-port side of the chassis are:
Power supplies
One power supply is required to run the base system with two required for N+1
redundancy
Power requirements may vary based on the number and types of blades
installed, see the Release Notes for the version of Fabric OS you are running
The Brocade Power and Bandwidth Calculator can be used to determine overall
power usage for a given configuration:
http://www.brocade.com/sites/dotcom/data-center-best-practices/
competitive-information/power.page
Blowers
Two functional blowers are required to cool the DCX
If a blower failure occurs the remaining blowers will operate at a higher RPM to
ensure chassis cooling requirements are met
WWN cards
Stores FRU serial number, run time hours, OEM specific information, and
event/error logs, licenses, IP addresses, and world wide names

Revision 0312

28

CFA 250

Preparing for the 16 Gbps BCFA Exam

The chart above shows a brief comparison between the different director chassis. Core
ASICs are mentioned here for completeness and will be discussed in much greater
detail later in this course.
The bandwidth numbers shown are uni-directional. Brocade marketing numbers are
usually shown as bi-directional, which would be twice what is shown in the presentation.
The DCX 8510 family provides increased total chassis bandwidth:
8.2 Tbps (384x 16 Gbps + 2 Tbps ICL)
4.1 Tbps (192x 16 Gbps + 1 Tbps ICL)
Note: These numbers are calculated using local switching. The total chassis bandwidth
if using the backplane would be:
(8 x 512 Gbps slot bandwidth = 4.096 Tbps) + 2 Tbps ICL = 6.096 Tbps for DCX 8510-8
(4 x 512 Gbps slot bandwidth = 2.048 Tbps) + 1 Tbps ICL = 3.048 Tbps for DCX 8510-4

Footnote 1: The maximum number of ports in the DCX 8510-8 is based on using either
FC16-48 blades (384 ports max) or FC8-64 port blades (512 ports max). There is no
FC16-64 blade available at this time.

Revision 0312

29

CFA 250

Revision 0312

Preparing for the 16 Gbps BCFA Exam

30

CFA 250

Preparing for the 16 Gbps BCFA Exam

Footnote 1: The weight of a configured DCX-4S or DCX 8510-4 chassis varies depending
on which blade options are installed. The weight referenced above reflects a fully
populated chassis. Consult the hardware reference manual for the specific chassis you
are configuring for individual blade chassis and blade weights.
Weights for the DCX-4S is based on 256-port configuration with four FC8-64 port blades.
Weight for the DCX 8510-4 is based on 192-port configuration with four FC16-48 port
blades including two CP blades, 2 core switch blades. two blowers, two power supplies,
and two cable management finger assemblies.

Footnote 2: Heat output varies depending on installed options. The heat output
numbers shown above are based on a full configuration (all power supplies, all blowers,
and all CP, CR, and 48-port blades). Consult the hardware reference manual for the
specific chassis you are configuring for more detailed information.
Note: The following links you to the Brocade Power Calculator web page:
http://www.brocade.com/data-center-best-practices/competitiveinformation/power.page

For more detailed information about installation techniques and compatible


components, please visit the Brocade website and download these products Hardware
Reference Manual.
Revision 0312

31

CFA 250

Preparing for the 16 Gbps BCFA Exam

The DCX 8510-4 is intended for midsize enterprise fabric core or large enterprise edge.
Footnote 1: Up to 256 user ports at 8 Gbps.
Footnote 2: Blades are mutually exclusive for the Fabric OS v7.0.0 release. You can
install one or the other, but not both at the same time.
Switchtype 121.X (4 port blade slots)

Revision 0312

32

CFA 250

Preparing for the 16 Gbps BCFA Exam

If using 220 VAC, power supplies generate 2000 watts each. If using 110 VAC, power
supplies generate 1000 watts each.

There are a number of FRUs that are compatible between the DCX and DCX-4S chassis:
Control Processor blade
Power supply
Blower
Port blades
SFPs
The following components are NOT compatible between the two chassis:
Core Router blades
The Core Router blades are designed to support four slots in the DCX-4S versus
eight slots in the DCX
WWN cards
The WWN cards in the DCX-4S have a different form factor than the DCX WWN
cards

Revision 0312

33

CFA 250

Preparing for the 16 Gbps BCFA Exam

The chart above shows a brief comparison between the different director chassis. Core
ASICs are mentioned here for completeness and will be discussed in much greater
detail later in this course.
The bandwidth numbers shown are uni-directional. Brocade marketing numbers are
usually shown as bi-directional, which would be twice what is shown in the presentation.
The DCX 8510 family provides increased total chassis bandwidth:
8.2 Tbps (384x 16 Gbps + 2 Tbps ICL)
4.1 Tbps (192x 16 Gbps + 1 Tbps ICL)
Note: These numbers are calculated using local switching. The total chassis bandwidth
if using the backplane would be:
(8 x 512 Gbps slot bandwidth = 4.096 Tbps) + 2 Tbps ICL = 6.096 Tbps for DCX 8510-8
(4 x 512 Gbps slot bandwidth = 2.048 Tbps) + 1 Tbps ICL = 3.048 Tbps for DCX 8510-4
Footnote 1: The maximum number of ports in the DCX 8510-8 is based on using either FC16-48
blades (192 ports max) or FC8-64 port blades (256 ports max). There is no FC16-64 blade
available at this time.

Revision 0312

34

CFA 250

Preparing for the 16 Gbps BCFA Exam

The CR16-8 is the new core routing blade used in the DCX 8510-8. It handles ICL
connections between chassis as well as routing between slots within the same chassis.
Slots for up to 16 Brocade branded QSFPs offer up to 64 16 Gbps FC ports in total. New
ICLs use an optical cable, eliminating previous distance limitations and allowing up to
six chassis to be connected together.

Revision 0312

35

CFA 250

Preparing for the 16 Gbps BCFA Exam

The CR16-4 is the new core routing blade used in the DCX 8510-8. It handles ICL
connections between chassis as well as routing between slots within the same chassis.
Slots for up to 8 Brocade branded QSFPs offer up to 32 16 Gbps FC ports in total. New
ICLs use an optical cable, eliminating previous distance limitations and allowing up to
six chassis to be connected together.

Revision 0312

36

CFA 250

Preparing for the 16 Gbps BCFA Exam

Control processing and core routing blades are at the heart of the Brocade director
switches. The core processing blades contain the CPUs for the chassis and run the
Brocade Fabric OS software. The core routing blades handle the routing of Fibre
Channel frames between different blades installed in the chassis.
The 4-slot and 8-slot chassis utilize separate routing and processing blades for
increased redundancy.

Revision 0312

37

CFA 250

Preparing for the 16 Gbps BCFA Exam

16 Gbps FC ports support E, F, EX, Diagnostic, and Mirror ports. No FL port support.
Front end ports can operate at 2, 4, 8, 10, and 16 Gbps
10 Gbps ports can be configured as E_Ports only
Support for 10 Gbps ports is limited to first 8 ports on each blade with FOS v7.0.0
Diagnostic port for Condor3-based ports:
Support electrical and optical loopback (16 Gbps Brocade branded optics only),
cable length detection and spinfab-like cable saturation tests (enough to saturate
a 16 Gbps link).
10 Gbps FC using 10GE license
Existing slot-based 10GbE FCIP license (introduced in FOS v6.3 for FX8-24 blades)
is extended to enable Condor3 FC ports running at 10 Gbps rate. These ports need
to work with DWDM equipment with plain transponder cards).
When applied to a 16 Gbps blade, license allows all ports to be enabled as 10
Gbps FC.

Revision 0312

38

CFA 250

Preparing for the 16 Gbps BCFA Exam

Footnote 1: 16 Gbps tri-mode SFPs operate at 4, 8, or 16 Gbps, 8 Gbps tri-mode SFPs


operate at 2,4 or 8 Gbps. 4 Gbps tri-mod SFPs operate at 1, 2 or 4 Gbps. Mini SFPs
(mSFPs) operate at 2, 4, or 8 Gbps and are used with FC8-64.
Footnote 2: The backend ports connecting to the core run at 4 Gbps while the front end
ports run at 8 Gbps. If the initiator and target are on the same ASIC, the frame would
not go through the core and local switching would be used. If the frame has to pass
through the core ASICs than the oversubscription numbers above become important.
The front end ports operate at 8 Gbps and the back end ports, the ones that connect to
the core ASICs, are limited to 4 Gbps .

Footnote 3: Each Condor2 ASIC has 1420 user buffer-to-buffer (BB) credits, and each
front-end port is allocated 8 credits. The 16 and 32 port blades each have 16 front-end
ports per ASIC, (8 credits x 16 ports = 128 credits) which leaves 1292 credits available
per Condor2 ASIC.
Footnote 4: Each Condor2 ASIC has 1420 user buffer-to-buffer (BB) credits, and each
front-end port is allocated 8 credits. The 48-port blade has 24 front-end ports per ASIC,
(8 credits x 24 ports = 192 credits) which leaves 1228 credits available per Condor2
ASIC. Use the portbuffershow command to see available credits.
Footnote 5: Long distance configurations are not supported on the FC8-64. The FC8-64
blade uses mini-SFPs (mSFP) for increased port density. The mSFP has a smaller form
factor and takes up less space than a standard sized SFP; they are not interchangeable.
The FC8-64, also, does not support FICON or Extended Fabrics.

Revision 0312

39

CFA 250

Preparing for the 16 Gbps BCFA Exam

Port blades are named based on their speed and number of user ports available. For
example the FC8-32 blade operates at 8 Gbps and contains 32 Fibre Channel ports, the
FC4-16 blade operates at 4 Gbps and contains 16 Fibre Channel ports.
Although the 8 Gbps blades are supported the 48000 utilizes a 4 Gbps core. Devices
will connect to the ports at 8 Gbps but any Fibre Channel frames that pass through the
CP blades will be limited to the 4 Gbps available.
The available Brocade port blades will be discussed in more detail later in this course.

Revision 0312

40

CFA 250

Revision 0312

Preparing for the 16 Gbps BCFA Exam

GREEN

AMBER

CONDITION

OFF

OFF

(Cable is not present) OR (Local end is not ready) OR


(Far end is not ready)

OFF

ON

N/A

ON

OFF

(Cable is present) AND (Local end is ready) AND


(Far end is ready)

ON

ON (Blinking)

(Cable is present) AND (Local end is ready) AND


(Far end is ready) AND (Attention is required)

41

CFA 250

Revision 0312

Preparing for the 16 Gbps BCFA Exam

42

CFA 250

Preparing for the 16 Gbps BCFA Exam

DCX Backbones have special Inter-Chassis Links ports for connecting two backbone
chassis using special ICL cables. Latest generation ICL cables are now optical. This
enables greater distance of 50 meters for ICL cables.
Improved Inter-Chassis Link (ICL) connectivity up to:
2 Tbps (32x 64 Gbps) for DCX 8510-8
1 Tbps (16x 64 Gbps) for DCX 8510-4

The DCX 8510-8/DCX 8510-4 offers an ICL POD licensing option, with a single ICL POD
license enabling sixteen 64 Gbps QSFPs (8 per core). A second ICL POD license can be
applied to the DCX 8510-8 to enable the remaining sixteen 64 Gbps QSFPs. (A single ICL
POD license enables the full ICL capability on a DCX 8510-4.) The existing ICL licensing
infrastructure for DCX/DCX-4S will be used to support the ICL POD licensing.

DCX/DCX-4S 8 Gbps ICL licensing does not change with FOS v7.0.0 retain support for
existing 8-link and 16-link licenses.

Revision 0312

43

CFA 250

Revision 0312

Preparing for the 16 Gbps BCFA

44

CFA 250

Preparing for the 16 Gbps BCFA

Each QSFP optical cable bundles four 16 Gbps channels; each channel is an independent
FC port. On the CR16-8 each QSFP channel terminates on a different ASIC. Since the CR164 only has two ASICs two channels from each QSFP terminate on a single ASIC in separate
trunk groups.

Footnote 1: Eight link trunks can only be formed between two DCX 8510-8s or two DCX
8510-4s. Trunks between a DCX 8510-8 and an 8510-4 are limited to four links (ICL ports 03, 4-7, etc.). This is due to the different number of ASICs on the CR16-8 and CR16-4 blades
and how the QSFP ports are aligned on those ASICs.

Revision 0312

45

CFA 250

Preparing for the 16 Gbps BCFA

In the above trunkshow output we can see the trunks that form the ICL connections to a
neighboring backbone. Since each ICL QSFP port has four ports in separate trunk groups
there is a difference of four between each of the ports in an ICL trunk group. This is a result
of one port per QSFP being a member of the same trunk group.

Revision 0312

46

CFA 250

Revision 0312

Preparing for the 16 Gbps BCFA

47

CFA 250

Preparing for the 16 Gbps BCFA Exam

ICL licensing for the DCX 8510 is a Ports-on-Demand (POD) implementation with a new
part number

The display strings in licenseshow CLI and GUI output has changed for DCX families
running Fabric OS v 7.0.0
Pre-Fabic OS v7.0.0:
DCX:admin> licenseshow
X3ffNTZM9CNmM4SKFMYTGS4WmCRCgAZZBJDTB:
Inter Chassis Link (16 link) license
Fabric OS v7.0.0:
DCX:admin> licenseshow
X3ffNTZM9CNmM4SKFMYTGS4WmCRCgAZZBJDTB:
Inter Chassis Link (2nd POD) license

Revision 0312

48

CFA 250

Preparing for the 16 Gbps BCFA Exam

The following table lists the switch type assigned to each switch or blade and is
displayed in the switchshow command.
Brocade Switch/Blade

Switch Type

ASIC

FC16-48 (48 Port 16 Gbps blade)

96

Condor3

FC16-32 (32 port 16 Gbps blade)

97

Condor3

FC8-64 (64 port 8 Gbps blade)

77

Condor2

CR16-8 (16 Gbps core for DCX 8510-8)

98

Condor3

CR16-4 (16 Gbps core for DCX 8510-4)

99

Condor3

CP-8

50

Condor2

Brocade 6510 (16 Gbps FC switch)

109.1

Condor3

DCX 8510-4

121.3

Condor3

DCX 8510-8

120.1

Condor3

Below is a slotshow -m output from a DCX 8510-4 chassis with FC16-32, FC16-48
and CR16-4 blades.
Slot BladeType
ID
Model Name Status
-------------------------------------------------1
SW BLADE
96
FC16-48
ENABLED
2
SW BLADE
97
FC16-32
ENABLED
3
CORE BLADE 99
CR16-4
ENABLED
4
CP BLADE
50
CP8
ENABLED
5
CP BLADE
50
CP8
ENABLED
6
CORE BLADE 99
CR16-4
ENABLED
7
SW BLADE
77
FC8-64
ENABLED
8
SW BLADE
97
FC16-32
ENABLED

Below is a slotshow -m output on a DCX 8510-8 chassis with FC16-32, FC16-48


and CR16-8 blades.
Slot Blade Type ID
Model Name Status
-------------------------------------------------1
UNKNOWN
VACANT
2
UNKNOWN
VACANT
3
SW BLADE
96
FC16-48
ENABLED
4
UNKNOWN
VACANT
5
CORE BLADE 98
CR16-8
ENABLED
6
CP BLADE
50
CP8
ENABLED
7
CP BLADE
50
CP8
ENABLED
8
CORE BLADE 98
CR16-8
ENABLED
9
UNKNOWN
VACANT
10
UNKNOWN
VACANT
11
SW BLADE
97
FC16-32
ENABLED
12
UNKNOWN
VACANT

Revision 0312

49

CFA 250

Preparing for the 16 Gbps BCFA Exam

The port blades in DCX 8510-8 are assigned port numbers from 0-383 and 768-895.
The ICL ports are assigned the following numbers:
On Slot 5:
384 through 415
1152 through 1183
On Slot 8:
416 through 447
1184 through 1215

Revision 0312

50

CFA 250

Revision 0312

Preparing for the 16 Gbps BCFA Exam

51

CFA 250

Revision 0312

Preparing for the 16 Gbps BCFA Exam

52

CFA 250

Preparing for the 16 Gbps BCFA Exam

When a new switch has arrived for installation into a fabric, it is suggested to use a
serial cable to configure the switch with an IP address. After the IP address is
configured, the serial connection to the switch may be dropped and an SSH, telnet, or
Web Tools session may be used for further switch configuration because of its
convenience and speed.
To configure the connection in a B-Series environment
Bits per second: 9600
Data bits: 8
Parity: None
Stop bits: 1
Flow control: None

Installation steps
1. Insert the serial cable provided to an RS-232 serial port on the workstation. FOS
switches use a straight-through cable.
2. Verify the switch has power and is past the POST stage.
3. Enter the ipaddrset command to set the IP address, subnet mask, and default
gateway.

Revision 0312

53

CFA 250

Preparing for the 16 Gbps BCFA Exam

IP addresses are assigned to the management interface of a switch, director, or


backbone and used to remotely manage the switch through telnet or SSH. A switch (e.g.
B300, B5100) only has a single management interface and only uses a single IP
address.
Backbones require three IP addresses: one for chassis/switch management and one for
each CP blade. The IP addresses used for the CP blades always connect to the blades
they are assigned, the chassis management IP always connect to the active CP.
Additionally, the DCX backbones have two Ethernet management interfaces on each CP
(eth0 and eth3). These interfaces use port bonding to create a logical interface (bond0)
to which the IP address of the CP is assigned. Ethernet bonding provides link (physical)
layer redundancy using an active/standby model. By default all traffic is transmitted
over the active interface, eth0. If eth0 experiences a link failure (e.g. cable unplugged)
then eth3 becomes active.
For more information on port bonding refer to the Fabric OS Command Reference
Manual and the ifmodeshow command.

Revision 0312

54

CFA 250

Preparing for the 16 Gbps BCFA Exam

You can use the IPFilter policies to block any of the IP based Mangement interfaces
Footnote 1: The SCP(Secure Copy) protocol is a network protocol, based on the BSD RCP
protocol, which supports file transfers between hosts on a network. SCP uses Secure
Shell (SSH) for data transfer and utilizes the same mechanisms for authentication,
thereby ensuring the authenticity and confidentiality of the data in transit. Usually used
for uploading configuration files from the switch. USBs can also be used to accomplish
more secure file transfers.
Footnote 2: Secure Socket Layer (SSL) is part of base Fabric OS. SSL works by using a
key to encrypt data transferred over an SSL connection. By convention, URLs that
require an SSL connection start with https: instead of http: All Brocade supported
Internet browsers support SSL.
Configuration of the SSL protocol involves obtaining, installing, and configuring PKI
certificates:
Public Key Infrastructure (PKI) is a system of public key encryption using digital certificates
from a Certificate Authority (CA) and other registration authority to verify and authenticate
the validity of each party involved in an electronic transaction.
The CA works as part of a Public Key Infrastructure (PKI) and therefore checks with a
registration authority (RA) to verify digital certificate requestor information. Once RA
verifies information CA can issue a certificate. The information that the RA verifies
depends on the CA, but includes items such as owners public key; certificate expiration
date; owners name and other public key owner information.
Revision 0312

55

CFA 250

Preparing for the 16 Gbps BCFA Exam

Setting the domain ID


1. Connect to the switch and log in on an account assigned to the admin role.
2. Enter the switchDisable command to disable the switch.
3. Enter the configure command.
4. Enter y after the Fabric Parameters prompt.
Fabric parameters (yes, y, no, n): [no] y
5. Enter a unique domain ID at the Domain prompt. Use a domain ID value from 1
through 239 for normal operating mode (FCSW-compatible).
Domain: (1..239) [1] 3
6. Respond to the remaining prompts, or press Ctrl-D to accept the other settings and
exit.
7. Enter the switchEnable command to re-enable the switch.

Revision 0312

56

CFA 250

Preparing for the 16 Gbps BCFA Exam

Footnote1: Date parameters defined:


mm is the month, valid values are 01-12
dd is the date, valid values are 01-31
hh is the hour, valid values are 00-23

- mm is minutes, valid values are 00-59


- yy is the year, valid values are 00-99

SW1:admin> date
Tue May 16 15:00:57 UTC 2006
SW1:admin> tsclockserver
LOCL
SW1:admin> tsclockserver 128.118.25.3
Updating Clock Server configuration...done.
SW1:admin> tsclockserver
128.118.25.3
SW1:admin> date "0516073406"
External Time Synchronization in place. Cannot execute this
command.
SW1:admin> tsclockserver LOCL
Updating Clock Server configuration...done.
SW1:admin> tsclockserver
LOCL
SW1:admin> date "0516073406"
Tue May 16 07:34:00 UTC 2006
Revision 0312

57

CFA 250

Preparing for the 16 Gbps BCFA Exam

Default timeout on switches is 10 minutes. When changing the timeout value you can
use the login command to restart the login session and use the new timeout value.

Revision 0312

58

CFA 250

Preparing for the 16 Gbps BCFA Exam

sw1:admin> licenseshow
SRScQbceeyTSTdRI:
Fabric Watch license
cQQ9edcRzedRVAfw:

One feature per


license key

Extended Fabric license


Second Ports on Demand license - additional 8 port upgrade
license
FFQ4FNRPQrNNMrN79BJWfPSLgETXHfmYB7fZM:
Fabric Watch license

Multiple features
per license key

Performance Monitor license


Trunking license
FICON_CUP license
First Ports on Demand license - additional 8 port upgrade
license
Integrated Routing license
Adaptive Networking license
8 Gig FC license
Unknown30 license

Revision 0312

Unknown license results when a license


from a previous OS is no longer valid
Example: Web Tools is no longer a
licensed feature
59

CFA 250

Preparing for the 16 Gbps BCFA Exam

Synopsis:
fabricprincipal
fabricprincipal
fabricprincipal
fabricprincipal
fabricprincipal

--help|-h
[--show|-q]
--enable [-priority|-p priority] [-force|-f]
--disable
[-f] mode

Description:
Use this command to set principal switch selection mode for a switch and to set
priorities for principal switch selection. The implementation of the fabricprincipal
command is based solely on mechanisms specified in the Fibre Channel standards.
These mechanisms provide a preference for a switch requesting to be the principal
switch in a fabric, but they do not provide an absolute guarantee that a switch
requesting to be the principal switch is granted this status.
When dealing with large fabrics, the selection of the principal switch is less
deterministic. In these cases, to help ensure that the desired switch is selected as the
principal switch, a small cluster of switches should be interconnected first, followed by
additional switches to enlarge the fabric.

Revision 0312

60

CFA 250

Preparing for the 16 Gbps BCFA Exam

SW1:admin> motd --set "Access by unauthorized personnel is


prohibited.
SW1:admin> motd --show
Access by unauthorized personnel is prohibited.
SW1:admin> bannerset "You have successfully logged into the
switch."
SW1:FID128:admin> bannershow
You have successfully logged into the switch.

login as: admin


Access by unauthorized personnel is prohibited.
admin@10.255.237.9's password:
You have successfully logged into the switch.
-----------------------------------------------------------SW1:admin> exit
Revision 0312

61

CFA 250

Preparing for the 16 Gbps BCFA Exam

Footnote 1: All printable punctuation characters except colon ":" are allowed.
Footnote 2: The minimum password length may be set from 8 to 40 characters in length.
The password length is the total number of lowercase, uppercase, digits, and
punctuation characters. The total number of these characters may not exceed 40. Keep
this in mind as you specify the minimum number of each type of character required.
Footnote 3: The password history policy is not enforced when an administrator sets a
password for another user, but the password set by the administrator is recorded in the
user's password history.
SW1:admin> passwdcfg --set -lowercase 3 -uppercase 1 -digits
2 -punctuation 2 -minlength 10 -history 3
SW1:admin> passwd
Changing password for admin
Enter old password:
Enter new password:
Password must be between 10 and 40 characters long.
Enter new password:
Insufficient number of upper case letters

Revision 0312

62

CFA 250

Revision 0312

Preparing for the 16 Gbps BCFA Exam

63

CFA 250

Preparing for the 16 Gbps BCFA Exam

Footnote 1: Fabric OS v5.3 and later loads default IP Filter policies for both IPv4 and
IPv6.
Footnote 2: Policy rules will be discussed in more detail later.
Footnote 3: There are also two default policies, one for IPv4 and another for IPv6 and
some implicit policies that enable communication like syslog, from the switch out.

Revision 0312

64

CFA 250

Preparing for the 16 Gbps BCFA Exam

Use the portCfgOctetSpeedCombo command to set the combination speed for the first
port
octet to a setting that supports 10 Gbps operations. Valid settings for 10 Gbps
operations
include:
2autonegotiated or fixed port speeds of 10 Gbps, 8 Gbps,4 Gbps, and 2 Gbps
3autonegotiated or fixed port speeds of 16 Gbps and 10 Gbps

Use the portCfgSpeed command to set the port speed on each port you want to operate
at 10
Gbps.

Revision 0312

65

CFA 250

Preparing for the 16 Gbps BCFA Exam

A new element called Error Ports has been introduced in Fabric OS v7.0.0 to monitor
ports which are segmented or disabled for reasons such as security violations or fabric
watch port fencing.
The unit for the default and user defined thresholds Marginal, Faulty, Error ports and
Missing SFPs has been changed to a percentage of the current number of physical ports
present in the switch at a given time rather than the absolute number of physical ports.
Port-based components in switchstatuspolicy can move either to a Down or
Marginal state based on the thresholds calculated from the percentage of number of
physical FC ports present in the logical switch.
The total number of physical ports used for threshold calculation excludes FCoE ports,
VE ports, and internal ports

Revision 0312

66

CFA 250

Revision 0312

Preparing for the 16 Gbps BCFA Exam

67

CFA 250

Preparing for the 16 Gbps BCFA Exam

Footnote 1: If this switch was purchased from an OEM vendor there may be upgrade
requirements different from those listed here. Contact your vendor for additional
information. Always read the new firmware release notes to review any open or resolved
FOS issues. To review new feature sets, review the FOS Administrators Guide.
Footnote 2: In other words, upgrading a switch from Fabric OS v6.3.0 to v7.0.0 is a twostep processfirst upgrade to v6.4.0, and then upgrade to v7.0.0. If you are running a
pre-Fabric OS v6.2.0 version you must upgrade to v6.2.0, then to v6.3.0,then to v6.4.0,
and finally to v7.0.0.

Footnote 3: Ensure that all serial consoles (both CPs for directors) and any open network
connection sessions, such as Telnet, are logged and included with any trouble reports.

Revision 0312

68

CFA 250

Preparing for the 16 Gbps BCFA Exam

Footnote1: Ensure there is an active FTP service running on a network server where the
firmware package has been unzipped. Alternatively you can use SCP or the USB option
under this command. You can also choose to upgrade the switch with GUI based
applications by using Brocade Network Advisor or Web Tools.

Revision 0312

69

CFA 250

Preparing for the 16 Gbps BCFA Exam

Attempting to mount an unsupported USB device will result in the following failure:
dcx1:admin> usbstorage -e
Fail to enable USB storage device. Error:Device is not found

Revision 0312

70

CFA 250

Revision 0312

Preparing for the 16 Gbps BCFA Exam

71

CFA 250

Preparing for the 16 Gbps BCFA Exam

The FC-SW-2 standard for Storage Area Networks (SANs) uses an algorithm called Fabric
Shortest Path First (FSPF). FSPF is a link state path selection protocol and directs traffic
along the shortest path between the source and destination, based upon the link cost,
and makes it possible to detect link failures, determine shortest route for traffic, update
the routing table, provide fixed routing paths within a fabric, and maintains correct
ordering of frames. FSPF keeps track of the state of the links on all switches in the
Fabric and associates a cost with each link. The protocol computes paths from a switch
to all the other switches in the fabric by adding the cost of all links traversed by the
path, and chooses the path that minimizes the costs. This collection of the link states
(including costs) of all the switches in the fabric constitutes the topology database (or
link state database). Once established, FSPF programs the hardware routing tables for
all active ports on the switch. FSPF is not involved in frame switching.
There are two types of primary routing protocols in intranet networks, Distance Vector
and Link State:
Distance Vector is based on hop count. This is the number of switches you traverse
through to get from the source domain (switch) to the destination domain (switch).
Link State is based on a metric value based on a cost. The cost could be based on
bandwidth.

Revision 0312

72

CFA 250

Preparing for the 16 Gbps BCFA Exam

FSPF makes minimal use of the ISLs bandwidth, leaving virtually all of it available for
traffic. In a stable fabric, a Brocade switch will transmit 64 bytes every 20 seconds in
each direction. FSPF frames have the highest priority in the fabric. This guarantees that
a control frame is not delayed by user data and that FSPF routing decisions occur very
quickly during convergence.
FSPF guarantees a routing loop free topology at all times. Why is this important? It is
essential for a fabric to include many physical loops, because without loops there
would be no multiple path between switches, and therefore no redundancy. Without
multiple paths, if a link goes down part of the fabric becomes isolated. FSPF ensures
that the topology is loop free and that the frame will never be forwarded over the same
ISL more than once.
Brocade recommends no more than 7 hops between two switches. This limit is not
required or enforced by FSPF. Its purpose is to ensure that a frame will never be
delivered to a destination after E_D_TOV has expired.
FSPF calculates paths based on the destination domain ID. The fabric protocol must
complete domain ID assignments before routing can begin. ISLs provide the physical
pathway when the Source ID (SID) address has a frame destined to a port on a remote
switch Destination ID (DID). When an ISL is attached or removed from a switch, the
FSPF updates the route tables to reflect the addition or deletion of the new routes.
As each host transmits a frame to the switch, the switch reads the SID and DID in the
frame header. If the domain ID of the destination address is the same as the switch
(intra-switch communications), the frame buffer is copied to the destination port and a
credit R_RDY is sent to the host. The switch only needs to read word zero and word one
of the Fibre Channel frame to perform what is known as cut-through routing. A frame
may begin to emerge from the output port before it has been entirely received by the
input port. The entire frame does not need to be buffered in the switch.
If the destination domain ID is different than the source domain ID, then the switch
consults the FSPF route table to identify which local E_Port provides the Fabric
Shortest Path First to the remote domain. If IOD is not set, frames are held for 650 ms
before being sent down a new route.

Revision 0312

73

CFA 250

Preparing for the 16 Gbps BCFA Exam

Fabric Shortest Path First (FSPF) calculates paths based on the destination domain ID.
The Fabric Protocol must complete domain ID assignments before routing can begin.
ISLs provide the physical pathway for routing frames from a Source ID (SID) to a
Destination ID (DID) on a different domain (switch). When an ISL goes online or offline
FSPF will update the routing tables to reflect the change.
As each host transmits a frame to the switch, the switch will read the SID and DID in
the frame header. If the domain ID of the destination address is the same as the switch,
intra-switch communication, the frame buffer is copied to the destination port and an
R_RDY is sent to the host.
Since the SID and DID are in the first two words of the frame Brocade switches perform
cut-through routing. The first two words of an incoming frame are read, if the DID is
another port on the local domain the frame input is immediately transferred to the DID
port. The entire frame does not need to be buffered in the switch; a frame may begin to
emerge from the output port before it has been entirely received by the input port.
Between Domain 1 and Domain 3 in the figure above there are three paths: port 2 and
port 5, each with a cost of 500, and port 6 with a cost of 1000. Only the lowest cost
routes are in the routing table. In the figure above Domain 1 ports 2 and 5 would be in
the routing table, port 6 would not.
The routing table can be viewed using the urouteshow command. Static routes can be
assigned using the urouteconfig command.
Revision 0312

74

CFA 250

Preparing for the 16 Gbps BCFA Exam

Footnote 1: Different OEMs may use different default settings. Please check with your
switch vendor for settings.

2 Gbps ASIC routing is handled by the Fabric Shortest Path First (FSPF) protocol and
uses only Port-based routing.

Revision 0312

75

CFA 250

Preparing for the 16 Gbps BCFA Exam

DLS is a standard feature in Fibre Channel to share multiple available routes to a


destination domain. If multiple routes exist in the routing table, FSPF will dynamically
load share according to the ratio of bandwidth available on the routes.
Exchange-based routing depends on DLS for dynamic routing path selection. When
using Exchange-based routing, DLS is enabled by default and cannot be disabled. In
other words, you cannot enable or disable DLS when Exchange-based routing is in
effect. When Port-based routing is in force, you can enable DLS to optimize routing. DLS
recomputates load sharing when a switch boots, an E_Port/EX_port goes offline or
online, or when an Nx_Port comes online or goes offline. DLS is unidirectional, meaning
that it must be set at both ends of the link to be effective in both directions.
In a stable fabric, frames are always delivered in order, even when the traffic between
switches is shared among multiple paths. However, when topology changes occur in the
fabric (for example, if an E_Port goes down), traffic is rerouted around the failure, and
some frames could be delivered out of order. Most destination devices tolerate out-oforder delivery, but some do not.

Revision 0312

76

CFA 250

Preparing for the 16 Gbps BCFA Exam

The choice of the routing path is based only on the incoming port and the destination
domain. To optimize port-based routing, Dynamic Load Sharing (DLS) round-robins the
input ports across the available output ports within a domain.

Revision 0312

77

CFA 250

Preparing for the 16 Gbps BCFA Exam

Exchange-based routing is also known as Dynamic Path Selection (DPS).


DLS cannot be disabled if exhanged-based routing is enabled.
The choice of routing path is based on the Source ID (SID), Destination ID (DID), and
Fibre Channel originator exchange ID (OXID), optimizing path utilization for the best
performance. Thus, every exchange can take a different path through the fabric.
Exchange-based routing requires the use of the Dynamic Load Sharing (DLS) feature;
when this policy is in effect, you cannot disable the DLS feature.
Exchange-based routing is also known as Dynamic Path Selection (DPS). DPS is where
exchanges or communication between end-devices in a fabric are assigned to egress
ports in ratios proportional to the potential bandwidth of the ISL or trunk group. When
there are multiple paths to a destination, the input traffic will be distributed across the
different paths in proportion to the bandwidth available on each of the paths. This
improves utilization of the available paths, thus reducing possible congestion on the
paths. Every time there is a change in the network (which changes the available paths),
the input traffic can be redistributed across the available paths. This is a very easy and
non-disruptive process when the exchange-based routing policy is engaged.

Revision 0312

78

CFA 250

Preparing for the 16 Gbps BCFA Exam

Data traffic Virtual Channels (VCs) are collapsed to optimize performance over long
distances using the portcfglongdistance command, as shown in the diagram
below.
Information about switch characteristics and capacity in terms of buffers per port group,
port speed, and distances supported is contained in the Fabric OS Administrator's Guide
and the appropriate Hardware Reference manual specific to the switch you are
configuring.

VC2

VC3

Revision 0312

VC2

VC 2 only

VC3

VC4

VC4

VC5

VC5

79

CFA 250

Preparing for the 16 Gbps BCFA Exam

The maximum Extended Fabric distance depends on the version of Brocade switch ASIC
installed in the switch.
Extended Fabric distance levels (L0, LE, LD, LS) :
Cannot be set or removed by configure or configdefault
Can be cleared by portcfgdefault
Saved in a switch configuration file (configupload ) as portcfg parameter
Use the portcfglongdistance command to support long distance links and to
allocate sufficient numbers of full size frame buffers on a particular port. Changes made
by this command are persistent across switch reboots and power cycles. This command
supports the following long-distance link modes:
Static Mode (L0) - L0 is the normal (default) mode for a port. It configures the port as a
regular port. A total of 20 frame buffers are reserved for data traffic, regardless of the
ports operating speed; therefore, the maximum supported link distance is up to 10 km
at 1 Gbps, up to 5 km at 2 Gbps, up to 2 km at 4 Gbps, and up to 1 km at 8 Gbps.

Revision 0312

80

CFA 250

Preparing for the 16 Gbps BCFA Exam

The routing database determines how frames are routed from input port to output port
when going to the next destination. Fabric Shortest Path First (FSPF) puts available
equal cost routes in the routing data base. One output port in the trunk group is put into
the routing data base. When a communication between two end devices in a fabric is
assigned a route through a trunk, the ASIC of the assigned trunk group port will be the
same ASIC as all ports in the trunk group. This ASIC will multiplex frames across ISLs in
the trunk group and maintain in-order delivery. The ASIC will send a frame down each
link to determine the links latency. These individual link latency calculations will be used
to maintain in-order delivery.

If some ports in a trunk group have QoS enabled and some ports have QoS disabled, the
two different trunks are formed: one with QoS enabled and one with QoS disabled.

Revision 0312

81

CFA 250

Preparing for the 16 Gbps BCFA Exam

Footnote 1: Automatically creates ISL trunks using from 2 to 8 ISLs when the switches
are connected and all trunking requirements are met.
Footnote 2: Must use the first 8 ports of the switch or blade

Trunk Monitoring
To monitor E_Port (ISL) and F_Port trunks, you can set monitors only on the master port
of the trunk. If the master changes, the monitor automatically moves to the new master
port. If a monitor is installed on a port that later becomes a slave port when a trunk
comes up, the monitor automatically moves to the master port of the trunk.

For masterless trunking, if the master port goes offline, the new master acquires all the
configurations and bottleneck history of the old master and continues with bottleneck
detection on the trunk.

Revision 0312

82

CFA 250

Preparing for the 16 Gbps BCFA Exam

Dynamic Path Selection (DPS) is exchange-based routing where exchanges or


communications between end devices in a fabric are assigned to egress ports in ratios
proportional to the potential bandwidth of the ISL or trunk group.
When there are multiple routes to a destination, the input traffic will be distributed
across the different routes in proportion to the bandwidth available on each of the
routes. This improves utilization of the available routes, thus reducing possible
congestion on the routes. Every time there is a change in the network (which changes
the available routes), the input traffic can be redistributed across the available routes.
This is a very easy and non-disruptive process when the Exchange-based Routing Policy
is engaged.
Exchanges in the example depicted on this slide are allocated based on the primary
criteria: link cost and secondary criteria: potential bandwidth. The potential bandwidth
allocation depicted in this example yields flow allocations of 3:1.

Revision 0312

83

CFA 250

Preparing for the 16 Gbps BCFA Exam

When the Trunk Master is disabled another pre-determined port takes over the role
without fabric disruption.

Revision 0312

84

CFA 250

Preparing for the 16 Gbps BCFA Exam

Footnote 1:
Up to eight ports can be grouped together in one trunk group to create high performance
128 Gbps ISL trunks between switches.
Trunk links can be 2 Gbps, 4 Gbps, 8 Gbps, 10 Gbps, or 16 Gbps depending on the
Brocade platform, or individual port speed settings.
The maximum number of ports per trunk and trunks per switch depends on the Brocade
platform.

There must be a direct connection between participating switches.


The port ISL R_RDY mode must be disabled (using the portCfgIslMode command).
Trunks operate best when the cable length of each trunked link is roughly equal to the
others in the trunk. For optimal performance, no more than 30 meters difference is
recommended. Trunks are compatible with both short wavelength (SWL) and long
wavelength (LWL) fiber optic cables and transceivers.
The compatibility matrix can be found at: http://www.brocade.com/productssolutions/technology-architecture/compatibility/index.page

Revision 0312

85

CFA 250

Preparing for the 16 Gbps BCFA Exam

Light in a vacuum travels much faster, but in optical cable the rate is about 5 ns/meter.
5ns/meter multiplied by 30 meters is 150 ns. The difference in cable lengths between
the ISLs in a trunk determines the deskew value. This is needed for timing purposes so
that delivery of frames across the trunk can be ensured. The shortest ISL is selected as
the base and is assigned a deskew value of 150 nsec. The deskew values are expressed
(shown in all command displays) by dividing the time value by 10. Example: A deskew
value of 150 nanoseconds is shown as 15 (150/10).
The first ISL in the trunk to initialize is selected as the trunk master. The length of the
cable is not a consideration when selecting the master. The deskew values for the other
ISLs in the trunk will be calculated from the base ISL and will have a higher value. Each
switch connected by the ISL will have a deskew value since each has a separate
transmit line to the other. Due to the signal quality/optical media, cables that are
identified as the same length may have a different deskew value. For example, one
cable may have a deskew value of 16 and a cable of the same length may calculate to
be 17. This is not a problem since deskew is a true measurement of its transmission
capabilities.

Revision 0312

86

CFA 250

Revision 0312

Preparing for the 16 Gbps BCFA Exam

87

CFA 250

Revision 0312

Preparing for the 16 Gbps BCFA Exam

88

CFA 250

Preparing for the 16 Gbps BCFA Exam

Footnote 1: A regular or normal zone enables you to partition your fabric into logical
groups of devices that can access each other. Zoning also supports special zones called
LSAN, QoS, and TI zones. Unless otherwise specified, all references to zones in this
module refer to normal zones.

Revision 0312

89

CFA 250

Preparing for the 16 Gbps BCFA Exam

Member:
Alias is given a name, e.g. Server_1, Disk_Array_2.
Physical Fabric port number or area number.
Node World Wide Name - Obtained using nsshow or switchshow.
Port World Wide Name Obtained using nsshow or portloginshow.
64 characters maximum: A-Z, a-z, 0-9 and the _ are allowed.
Zone:
Is given a name, e.g. Red_Zone.
Contains two or more members and uses a ; as a separator.
The same member can be in multiple zones.
Zone definition is persistent; it remains until deleted or changed by an
administrator.
Configuration:
Is given a name, e.g. Production_Cfg.
Is one or more zones.
Configuration may be disabled or one configuration may be in effect from any
switch in the fabric.
An administrator selects which configuration is currently enabled.
A configuration is saved when enabled and then distributed to the remaining
switches in the fabric where it is enabled and saved.

Revision 0312

90

CFA 250

Preparing for the 16 Gbps BCFA Exam

Footnote1: Zone and configuration names are also limited to 64 characters maximum.

Zone aliases simplify repetitive entry of zone objects, such as PWWNs. For example, the
name Eng could be used as an alias for PWWN: 10:00:00:80:33:3f:aa:11. An alias is a
name assigned to a device or group of devices. By creating an alias, you can assign a
familiar name to a device, or you can group multiple devices into a single name. This can
simplify cumbersome entries and it allows an intuitive naming structure such as using
NT_Storage to define all NT storage ports in the fabric.
Alias objects only appear in the defined configuration since they are used to assign a
meaningful name to a device or group of devices
Zone objects identified by Domain, Index are specified as a pair of decimal numbers
where Domain is the Domain ID of the switch and Index is the index number for the
port on that switch.
Zone objects identified by World Wide Name (WWN) are specified as a 16 digit
hexadecimal number separated by colons, for example 10:00:00:90:69:00:00:8a. When
a node name is used to specify a zone object, all ports on that device are in the zone.
When a port name is used to specify a zone object, only that single port is in the zone.

Revision 0312

91

CFA 250

Preparing for the 16 Gbps BCFA Exam

A zone configuration is a group of zones that is enforced whenever that zone


configuration is enabled. A zone can be included in more than one zone configuration.
To define a zone configuration, specify the list of zones to be included and assign a zone
configuration name. Zoning may be disabled at any time. When a zone configuration is
in effect, all zones that are members of that configuration are in effect.
Defined configuration: The complete set of all zone objects that have been defined
in the fabric.
Effective configuration: A single zone configuration that is currently in effect. The
effective configuration is built when an administrator enables a specified zone
configuration. This configuration is compiled by checking for undefined zone
names, or zone alias names, or other issues.
Saved configuration: A copy of the defined configuration plus the name of the
effective configuration which is saved in flash memory by the cfgsave command.
There may be differences between the saved configuration and the defined
configuration if the system administrator has modified any of the zone definitions
and has not saved them.

Revision 0312

92

CFA 250

Preparing for the 16 Gbps BCFA Exam

The default zone feature can enable or disable device access within a fabric. Default
zones are based on the FC-GS standard.
The defzone -allaccess is the default because it matches how zoning worked
prior to Fabric OS v5.1.0.
The defzone command configures a default zone configuration and displays the
current configuration. The command has no optional parameters, and takes one of
three required arguments:
--allaccess: Enables all device-to-device access within the fabric. This is the
default behavior in Fabric OS v5.1, and matches the default behavior in a nonzoned fabric.
--noaccess: Create a default zone that disables all device-to-device access within
the fabric.
--show: Display the current default zone.
Names beginning with d__efault__ are reserved for default zoning use (note: two
underscore characters are used in each instance.)
Note: The setting of the defzone command is stored in the zoning transaction buffer.
Normally, a cfgsave is used to commit the zoning transaction to the entire fabric. A
cfgenable or cfgdisable will do the commit since each command does an implied
cfgsave. Because the setting is stored in the zoning transaction buffer, a
cfgtransabort could be used to abort the defzone command.
Revision 0312

93

CFA 250

Preparing for the 16 Gbps BCFA Exam

A normal zone must be in effect granting access between devices before a TI zone will
be effective. TI zones will only appear in the defined zoning configuration, not in the
effective zoning configuration and can only be created using D,I (domain,index) notation.
TI zones must include E_Ports and F_Ports in order to create a complete, dedicated,
end-to-end route from initiator to target and ports can only be members of a single TI
zone.

Revision 0312

94

CFA 250

Preparing for the 16 Gbps BCFA Exam

Routes are not being changed, but one or more is being dedicated for use by a specific
set of devices. The urouteshow command will not show any changes when TI zones are
created.
Traffic Isolation zones use a special zoning command, zone, and are intended to
control the routing of frames between zone members, not to control access to
devices (uses zone -create not zonecreate command)
A normal zone must be in effect granting access between devices before a TI zone
will be effective
TI zones will only appear in the defined zoning configuration, not in the effective
zoning configuration
TI zones can only be created using D,I (Domain, Index) notation
TI zones must include E_Ports and F_Ports in order to create a complete,
dedicated, end-to-end route from initiator to target
Ports can only be members of a single TI zone
TI zones can have failover enabled or disabled.

Revision 0312

95

CFA 250

Preparing for the 16 Gbps BCFA Exam

Use FA-PWWN to represent a server in boot LUN zone configurations so that any physical
server that is mapped to this FA-PWWN can boot from that LUN, thus simplifying boot
over SAN configuration.
For the server to use this feature, it must be using a Brocade HBA/Adapter with HBA
driver version 3.0.0.0 or later. Some configuration of the HBA must be performed to use
FA-PWWN.

Revision 0312

96

CFA 250

Revision 0312

Preparing for the 16 Gbps BCFA Exam

97

CFA 250

Preparing for the 16 Gbps BCFA Exam

Brocade Network Advisor is the industrys first unified network management platform for
SAN and IP networks. It provides a single, easy-to-use solution across Brocade Fibre
Channel SANs, Layer 2/3 IP networks, Layer 4-7 application delivery networks, Fibre
Channel over Ethernet (FCoE) networks, wireless networks, and Multiprotocol Label
Switching (MPLS) networks. Brocade Network Advisor is a comprehensive tool for
configuring, managing, monitoring, and reporting on Brocade data center, enterprise
campus, and service provider networks with robust Role-Based Access Control (RBAC),
automation, operational simplicity, and end-to-end network visibility. In addition, it
integrates seamlessly with industry-leading orchestration products from Brocade
Partners to provide a best-in-class solution.

Revision 0312

98

CFA 250

Preparing for the 16 Gbps BCFA Exam

Brocade Network Advisor provides the following key features:


Flexible reporting: Includes the Report Manager customizable reporting tool along
with a rich set of predefined asset reports with detailed information about the
discovered devices.
Traffic analyzer: Provides trend analysis, management, and monitoring tools for
sFlow reporting, accounting, and presentation for all IP devices.

Revision 0312

99

CFA 250

Preparing for the 16 Gbps BCFA Exam

For a detailed list of supported features and descriptions, refer to the Brocade Network
Advisor Installation Guide.
Footnote 1: Network Advisor SAN Professional supports only Layer 2 FC Support
Encryption features.
Footnote 2: The FICON/CUP Prohibit Dynamic Connectivity Mask (PDCM) Matrix feature
is accessible through Element Manager on Network Advisor SAN Professional and MidMarket package options. The Cascaded FICON Configuration and Fabric Merge wizards
are only available on Network Advisor SAN Enterprise package option.

Revision 0312

100

CFA 250

Preparing for the 16 Gbps BCFA Exam

Fabric Watch works as a three-tiered hierarchy: Class Area Element


Fabric Class: E_Ports down, fabric reconfigure, Domain ID changes, segmentation, zone
changes, fabric logins, and SFP state changes.
Switch class includes environmental (temperature, fans, and power supplies); SFP (Tx/Rx power,
current, and voltage); Security (too numerous to list, see FW users guide); resource (Flash
memory usage), and FRU (switch specific, for example a Brocade 4100 will have fan and power
supply FRUs).
Ports Class include Port, E and F_Port link, synch, and signal loss; protocol error; invalid words
and CRCs, Rx/Tx performance and state changes.
Performance class includes End-to-End and custom performance options; custom performance
monitoring works with Brocade Advanced performance monitor feature.
The different groups of events listed above fabric, environmental, port, switch-level organize
the various transient and permanent faults that can occur in a fabric. Fabric Watch uses similar
groupings called classes to organize error detection and reporting.
Fabric Watch defines acceptable behavior as being within a lower boundary and an upper
boundary. The upper boundary is most commonly used to define, among other things, limits on
losses of synchronization or signal, SFPs (current/voltage levels too high), or over-temperature
conditions. The lower boundary can detect failed fans (RPM level too low) or power supplies
(current/voltage levels too low); as well as determining when link or port issues have been
resolved (set the lower boundary slightly greater than zero).

Revision 0312

101

CFA 250

Preparing for the 16 Gbps BCFA Exam

Temperature and TXP threshold breaches can indicate an SFP that is going bad.

There are new commands for the 16 Gbps and 10 Gpbs sfps.

sfpshow -- health this shows vendor/speed/health/speed

thmonitor -- enable brcdsfp

Revision 0312

102

CFA 250

Preparing for the 16 Gbps BCFA Exam

RXP is receive power


TXP is transmit power
The thConfig command has been enhanced to provide threshold monitoring
based on SFP type
Switch1:admin> thconfig --show sfp -sfptype QSFP -c
Class
|Area
|Index |Value
|State |Monitoring|SfpType
---------|-------|------|---------|-------|----------|------SFP
|TEMP
|000256|0
|Info
|Continue |QSFP
SFP
|TEMP
|000260|0
|Info
|Continue |QSFP
SFP
|TEMP
|000272|0
|Info
|Continue |QSFP
SFP
|TEMP
|000278|0
|Info
|Continue |QSFP
SFP
|VOLTAGE|000256|0
|Info
|Continue |QSFP
SFP
|VOLTAGE|000260|0
|Info
|Continue |QSFP
SFP
|VOLTAGE|000272|0
|Info
|Continue |QSFP
SFP
|VOLTAGE|000278|0
|Info
|Continue |QSFP

Revision 0312

103

CFA 250

Preparing for the 16 Gbps BCFA Exam

Switch:admin> thMonitor
Usage:
thConfig --show [<class_name>] [ -area <area_type> ][ sfptype <sfptype> ] [-current ] | [ -thresh_level
| -action_level <thresh_action> ]
thConfig --set <class_name> -area <area> [ -timebase
<day:hour:minute:second:none> ] [ -sfptype
<sfptype> ] [ -highth -value <val> -trigger
above | below -action snmp][raslog],[portlog],[email]
| none ] [ -lowth
-value <val> -trigger below
-action [snmp],[raslog],[portlog],[email] | none ]
[ -buffer <val>][ -nosave ]
thConfig --apply [<class_name> -area <area_type> -sfptype
<sfptype> -thresh_level <thresh_level>|
-action_level <action_level>
thConfig --cancel <class_name> -area <area_type> -sfptype
<sfptype> -thresh_level <thresh_level>|
-action_level <action_level>
thConfig --show
Revision 0312

[<class_name>] [ -area_only ]
104

CFA 250

Revision 0312

Preparing for the 16 Gbps BCFA Exam

105

CFA 250

Preparing for the 16 Gbps BCFA Exam

Taking the supportsave after troubleshooting has already started can make
resolution determination harder and may introduce false positives into the supportsave
data. If a supportsave is taken after troubleshooting has taken place; as part of the
escalation identify what troubleshooting steps were taken and when they were taken.

During supportsave in Fabric OS the *.dump files get moved to *.old.dump, the old file
gets overwritten.

Revision 0312

106

CFA 250

Preparing for the 16 Gbps BCFA Exam

If there is a problem with Network Advisor server or client session, a supportsave of the
session can be captured from Monitor > Technical Support > SupportSave.

Revision 0312

107

CFA 250

Preparing for the 16 Gbps BCFA Exam

HBA - Host Bus Adapter


CNA - Converged Network Adapter
FA - Fabric Adapter

Revision 0312

108

CFA 250

Revision 0312

Preparing for the 16 Gbps BCFA Exam

109

CFA 250

Preparing for the 16 Gbps BCFA Exam

Captures and packages the following data:


Firmware and driver traces
Supportshow information
Adapter configuration data
Windows registry data
Linux/VMware/Solaris bfa.conf file
Master and Application logs
Stored in the local file system
Supported from both Host Connectivity Manager (HCM) and Brocade Command Utility
(BCU)

Revision 0312

110

CFA 250

Preparing for the 16 Gbps BCFA Exam

If the adapter is dual ported the script will capture logs from both ports.
In the C:\Program Files\Brocade\Adapter\driver\util directory a subdirectory will be
created with the name of: bfa_ss_out_date_time_stamp. All the files will be in this
directory. They are not zipped. The supportshow output is called: bfa_ss.txt

Revision 0312

111

CFA 250

Revision 0312

Preparing for the 16 Gbps BCFA Exam

112

CFA 250

Preparing for the 16 Gbps BCFA Exam

1. Switch Status - the health status of the switch. Possible values are Healthy, Marginal,
Down, Unknown, Unmonitored, and Unreachable.
Fabric Status. Displays the state of the fabric that is least operational, based on ISL
status. The possible states are: operational, unknown, degraded or failed.

2. The green halo around the HBA in the topology view indicates a healthy device.

Revision 0312

113

CFA 250

Preparing for the 16 Gbps BCFA Exam

Switch Status - the health status of the switch. Possible values are Healthy, Marginal,
Down, Unknown, Unmonitored, and Unreachable.
Fabric Status. Displays the state of the fabric that is least operational, based on ISL
status. The possible states are: operational, unknown, degraded or failed.
SAN Product status ICONs

Revision 0312

Master Event log ICONs

114

CFA 250

Revision 0312

Preparing for the 16 Gbps BCFA Exam

115

CFA 250

Preparing for the 16 Gbps BCFA Exam

BottleneckMon detects a slow drain device (F_Ports)


Finds bottlenecks at the egress side of the port
Not recommended on links that are normally above 85% utilization
No license is required

Bottleneck Detection Supported Configurations


Support on:
4, 8 and 16 Gbps B-Series platforms
Virtual Fabrics enabled or disabled
Access Gateway switches
E, F and EX_Ports
Long distance Fibre Channel ports
Extended fabrics E and EX_Ports
Internal FCoE ports (not external CEE Ports)1
Trunk ports
Latency and congestion bottlenecks are reported only on the master port
When a new master port is selected all of the bottleneck configurations and
stats are transferred from old master to new master port
Alerting can be configured on a per port basis
Alerts are delivered as SNMP traps and RAS logs
Alerts can be configured selectively
While Bottleneck Detection is enabled on all ports, you can enable alerts only on
ports where alerts are required
On-demand inspecting through the CLI
CLI reports value of metric over a period of time (up to 3 hours) using the
bottleneckmon --show command

Revision 0312

116

CFA 250

Preparing for the 16 Gbps BCFA Exam

Use this command to detect latency and congestion bottlenecks on F[L]_Ports and
E_Ports. Bottleneck detection. For more details on this command see the Fabric OS v7.0
Administrators Guide.
bottleneckmon --enable [-cthresh congestion_threshold]
[-lthresh latency_threshold] [-time seconds]
[-qtime seconds] [-alert | -noalert]
[-lsubsectimethresh time_threshold]
[-lsubsecsevthresh severity_threshold]
bottleneckmon --disable
bottleneckmon --config [-cthresh congestion_threshold]
[-lthresh latency_threshold][-time seconds]
[-qtime seconds] [-alert | -noalert]
[-lsubsectimethresh time_threshold]
[-lsubsecsevthresh severity_threshold]
[[slot/]port_list]
bottleneckmon --configclear [slot/]port_list
bottleneckmon --exclude [slot/]port_list
bottleneckmon --include [slot/]port_list
bottleneckmon --show [-interval seconds] [-span seconds]
[-refresh][-congestion | -latency] [[slot/]port | '*']
bottleneckmon status

Revision 0312

117

CFA 250

Preparing for the 16 Gbps BCFA Exam

Footnote 1: Brocade patent pending

Footnote 2: The D_Port is not part of the fabric:


No routes set
No switch control frames flow through
No device data traffic flows through
No impact on the fabric operations or traffic though other ports
Complete isolation from the fabrics at both ends of link

Revision 0312

118

CFA 250

Preparing for the 16 Gbps BCFA Exam

Here is an example of a D_Port test running from port 0 to port 1:


From the switchshow output:
<truncated output>
Index Port Address Media Speed State

Proto

==============================================
0

020000

id

N16

Online

FC

D-Port Loopback->Port

020100

id

N16

In_Sync

FC

D-Port

<truncated output>

From the portdporttest --show 0 output:


<truncated output>
Status:

PASSED

==========================================================================
Test

Start time

Result

EST(secs)

Comments

==========================================================================
Electrical loopback

09:23:09

PASSED

--

--------

Optical loopback

--------

SKIPPED

--

--------

Link traffic test

09:24:14

PASSED

--

--------

==========================================================================
Roundtrip link latency:

990 nano-seconds

Estimated cable distance:

7 meters

Revision 0312

119

CFA 250

Preparing for the 16 Gbps BCFA Exam

Footnote 1:
If added directly, an unstable link may cause disruptions to existing traffic (route
change, DLS etc.)
The D_Port test can be performed on a port without disturbing the traffic.
The new link can be added after the D_Port test passes.
Footnote 2:
A D_Port does not form trunks.
The D_Port test can be performed on a port to be trunked without disturbing the
traffic on the trunk.
A trunk member can be isolated to run D_Port tests and merged with the trunk
after the test passes.
Footnote 3:
The D_Port functionality is supported on long distance ports.
Use D_Port functionality to test long distance links without merging the fabrics:
Single-mode fiber connecting the long-wave SFPs
DWDM links
The cable length can be measured with D_Port test.

Revision 0312

120

CFA 250

Preparing for the 16 Gbps BCFA Exam

b41_1:admin> fcping 10:00:00:00:c9:2a:a3:9e 21:00:00:20:37:e1:42:40


Source:

10:00:00:00:c9:2a:a3:9e

Destination:

21:00:00:20:37:e1:42:40

Zone Check:

Zoned

Pinging 10:00:00:00:c9:2a:a3:9e [0xa0100] with 12 bytes of data:


received reply from 10:00:00:00:c9:2a:a3:9e: 12 bytes time:694 usec
received reply from 10:00:00:00:c9:2a:a3:9e: 12 bytes time:664 usec
received reply from 10:00:00:00:c9:2a:a3:9e: 12 bytes time:665 usec

received reply from 10:00:00:00:c9:2a:a3:9e: 12 bytes time:662 usec


received reply from 10:00:00:00:c9:2a:a3:9e: 12 bytes time:531 usec
5 frames sent, 5 frames received, 0 frames rejected, 0 frames timeout
Round-trip min/avg/max = 531/643/694 usec
Pinging 21:00:00:20:37:e1:42:40 [0x1400e2] with 12 bytes of data:
Request timed out
<Truncated Output>
Request timed out
5 frames sent, 0 frames received, 0 frames rejected, 5 frames timeout
Round-trip min/avg/max = 0/0/0 usec

Note: This device may be offline or may not support ELS Echo frames.

Revision 0312

121

CFA 250

Revision 0312

Preparing for the 16 Gbps BCFA Exam

122

CFA 250

Preparing for the 16 Gbps BCFA Exam

Registering for a certification exam:


Visit http://www.pearsonvue.com/brocade
Call 866-361-5817 toll-free in North America
Visit http://www.pearsonvue.com for other contact numbers worldwide (some
locations may not have toll-free numbers)
Registering for an accreditation exam:
https://www.webassessor.com/wa.do?page=publicHome&branding=BROCADE

Revision 0312

123

CFA 250

Preparing for the 16 Gbps BCFA Exam

Footnote 1: Brocade University releases nutshell guides for each certification exam. The
guides are named after the exam, i.e. BCFP in a Nutshell, and are available from the
Brocade University certification page: http://www.brocade.com/education/certificationaccreditation.

Revision 0312

124

CFA 250

Revision 0312

Preparing for the 16 Gbps BCFA Exam

125

CFA 250

Revision 0312

Preparing for the 16 Gbps BCFA Exam

126

CFA 250

Revision 0312

Preparing for the 16 Gbps BCFA Exam

127

CFA 250

Revision 0312

Preparing for the 16 Gbps BCFA Exam

128

CFA 250

Revision 0312

Preparing for the 16 Gbps BCFA Exam

129

CFA 250

Revision 0312

Preparing for the 16 Gbps BCFA Exam

130