Sie sind auf Seite 1von 46

BCFP in a Nutshell

2010 Study Guide for


Exam 143-070
Exam Preparation Materials

Revision January 2010


Corporate Headquarters - San Jose, CA USA
T: (408) 333-8000
info@brocade.com

European Headquarters - Geneva, Switzerland


T: +41 22 799 56 40
emea-info@brocade.com

Asia Pacific Headquarters - Singapore


T: +65-6538-4700
apac-info@brocade.com

© 2010 Brocade Communications Systems, Inc. All Rights Reserved.

Brocade, the Brocade B-weave logo, Fabric OS, File Lifecycle Manager, MyView, Secure Fabric OS,
SilkWorm, and StorageX are registered trademarks and the Brocade B-wing symbol and Tapestry are
trademarks of Brocade Communications Systems, Inc., in the United States and/or in other countries.
FICON is a registered trademark of IBM Corporation in the U.S. and other countries. All other brands,
products, or service names are or may be trademarks or service marks of, and are used to identify,
products or services of their respective owners.

Notice: This document is for informational purposes only and does not set forth any warranty,
expressed or implied, concerning any equipment, equipment feature, or service offered or to be offered
by Brocade. Brocade reserves the right to make changes to this document at any time, without notice,
and assumes no responsibility for its use. This informational document describes features that may not
be currently available. Contact a Brocade sales office for information on feature and product availability.
Export of technical data contained in this document may require an export license from the United
States government.

Revision: January, 2010


BCFP in a Nutshell 2010 Edition

BCFP in a Nutshell 2010

Objective: The BCFP Nutshell guide is designed to help you prepare for the BCFP Certification, exam number
143-070.

Audience: The BCFP Nutshell self-study guide is intended for those who have successfully completed the
CFP 380 Advanced Fibre Channel Administration and Theory course, and who wish to undertake self-study or
review activities before taking the actual BCFP exam. The BCFP guide is not intended as a substitute for
classroom training or hands-on time with Brocade products.

How to make the most of the BCFP guide: The BCFP guide summarizes the key topics on the BCFP exam for
you in an easy to use format. It is organized closely around the exam objectives. We suggest this guide be
used in conjunction with our free online knowledge assessment test. To benefit from the BCFP guide, we
strongly recommend you have successfully completed the CFP 380 Advanced Fibre Channel Administration
and Theory course.

We hope you find this useful in your journey towards BCFP Certification, and we welcome your feedback by
sending an email to jcannata@brocade.com.

Helen Lautenschlager Joe Cannata


Director of Education Solutions Certification Manager

©2010 Brocade Communications i


BCFP in a Nutshell 2010 Edition

ii ©2010 Brocade Communications


BCFP in a Nutshell 2010 Edition

Table of Contents

1 - Product Features, Implementation and Configuration ...................................................................... 1


Fabric Initialization Process............................................................................................... 1
NPIV .................................................................................................................................... 1
Interoperability ................................................................................................................... 2
ICLs .................................................................................................................................... 2
Security ............................................................................................................................... 3
E_Port Authentication ........................................................................................... 4
Policy Distribution ................................................................................................. 4
Access Gateway ................................................................................................................. 6
Virtual Fabrics ................................................................................................................... 8
ISL Types ................................................................................................................ 8
FIDs and Domain IDs ............................................................................................ 9
10-bit addressing mode ....................................................................................... 9
Converged Enhanced Ethernet (CEE) ............................................................................. 10
FCoE.................................................................................................................................. 11
2 - SAN Management............................................................................................................................... 12
Audit Log ........................................................................................................................... 12
DCFM ................................................................................................................................ 12
Real-time performance data ............................................................................... 12
Fabric Binding ...................................................................................................... 13
High Integrity Fabrics .......................................................................................... 13
HCM .................................................................................................................................. 14
SAN Health ....................................................................................................................... 16
SNMP ................................................................................................................................ 16
Fabric Watch..................................................................................................................... 16
Port Fencing ......................................................................................................... 16
3 - Adaptive Networking........................................................................................................................... 17
Ingress Rate Limiting ....................................................................................................... 17
Traffic Isolation Zones ..................................................................................................... 17
QoS - SID/DID Based Traffic Prioritization ..................................................................... 18
Bottleneck Detection ....................................................................................................... 19
Top Talkers........................................................................................................................ 19
4 - FC - FC Routing ................................................................................................................................... 21
Integrated Routing ........................................................................................................... 21
LSAN Zones ...................................................................................................................... 22
Proxy devices .................................................................................................................... 23
Phantom domains............................................................................................................ 23

©2010 Brocade Communications iii


BCFP in a Nutshell 2010 Edition

5 - FCIP and Extension............................................................................................................................. 25


FCIP ................................................................................................................................... 25
Port Types ............................................................................................................. 25
FCIP Licenses ....................................................................................................... 25
Hardware .............................................................................................................. 26
Circuit Metric ........................................................................................................ 30
Selective Acknowledgement ............................................................................... 32
Long Distance ..................................................................................................................32
Buffer Credits ....................................................................................................... 32
Long Distance Trunking....................................................................................... 33
Protective Switching ............................................................................................34
6 - Troubleshooting .................................................................................................................................. 35
Gathering Information ..................................................................................................... 35
ISL Physical Connection Issues....................................................................................... 35
Slow Drain Device ............................................................................................................ 35
Speed Negotiation ........................................................................................................... 35
Segmented Switches ....................................................................................................... 36
Device Connectivity Problems ......................................................................................... 36
Fabric Changes ................................................................................................................ 37
Taking the Test ......................................................................................................................................... 38

iv ©2010 Brocade Communications


BCFP in a Nutshell 2010 Edition

List of Figures

SCC and DCC Policies ................................................................................................................................ 3


AG Port Basics............................................................................................................................................ 6
Default Port Mapping................................................................................................................................. 6
Virtual Fabric FIDs and Domain IDs.......................................................................................................... 9
Brocade 8000 and FCOE10-24 .............................................................................................................. 11
Phantom domains and devices .............................................................................................................. 23
Brocade 7800 .......................................................................................................................................... 26
FX8-24 ...................................................................................................................................................... 27
Circuit Metric ............................................................................................................................................ 30
FCIP QoS ................................................................................................................................................... 30
Protective Switching ................................................................................................................................ 34
Sample NDA .............................................................................................................................................38

List of Tables

Statistics monitored by component........................................................................................................ 17

©2010 Brocade Communications v


BCFP in a Nutshell 2010 Edition

vi ©2010 Brocade Communications


BCFP in a Nutshell 2010 Edition

1 - Product Features, Implementation and Configuration

Fabric Initialization Process

When a switch joins a fabric several Class F frames are used to exchange various parameters:
• ELP (Exchange Link Parameters)
- Contains sender information
- R_A_TOV / E_D_TOV
- PWWN / Switch Name
- Flow control used
• ESC (Exchange Switch Capabilities)
- Vendor specific info
- Virtual Fabric support
• EFP (Exchange Fabric Parameters)
- Principal switch selection
- Principal switch priority
- Switch name
- Domain ID list
- Vendor specific info
- Virtual Fabric support
- Zoning database

NPIV

N_Port ID Virtualization (NPIV) enables a single Fibre Channel protocol port to appear as multiple, distinct
ports, providing separate port identification within the fabric for each operating system image behind the port
(as if each operating system image had its own unique physical port). NPIV assigns a different virtual port ID
to each Fibre Channel protocol device. NPIV is designed to enable you to allocate virtual addresses without
affecting your existing hardware implementation. The virtual port has the same properties as an N_Port, and
is therefore capable of registering with all services of the fabric.

Each NPIV device has a unique device PID, Port WWN, and Node WWN, and should act the same as all other
physical devices in the fabric; in other words, multiple virtual devices emulated by NPIV appear no different
than regular devices connected to a non-NPIV port.

NPIV is enabled for every port on the Brocade 300, 4100, 4900, 5000, 5100, and 5300 switches, the
Brocade 5410, 5424, 5450, 5480 embedded switches, Brocade HBAs, the Brocade 48000 Director, the
Brocade DCX and DCX-4S enterprise-class platforms, and the FA4-18 blade.

Note: The FC10-6 port blade and the CEE ports on the Brocade 8000 do not support NPIV.

©2010 Brocade Communications 1


BCFP in a Nutshell 2010 Edition

Interoperability

Fabric Interoperability is switch-to-switch connectivity between B-Series and M-Series platforms.

Fabric OS v6.0 and later supports three fabric interop settings:


• Interopmode 0 (Brocade Native Mode)
• Interopmode 1 (no longer available)
• Interopmode 2 (McDATA Fabric Mode)
• Interopmode 3 (McDATA Open Fabric Mode)

The following restrictions apply to interopmode 2 and 3 configurations:


• A maximum of 2048 devices can be logged in to the fabric.
• A maximum of 31 switches or domain IDs can be defined.

ICLs

An Inter-chassis link (ICL) is a licensed feature used to interconnect two or three Brocade DCX Backbones,
which can be a combination of either Brocade DCXs and/ora Brocade DCX-4S Backbones. ICL ports in the
core blades are used to interconnect two Brocade Backbones, potentially increasing the number of usable
ports in the Brocade DCX or DCX-4S chassis. The ICL ports on CR8 and CR4S-8 blades are internally managed
as E_Ports. These ports use proprietary connectors instead of traditional SFPs. When two Brocade Backbones
are interconnected by ICLs, each chassis requires a unique domain and is managed as a separate switch.

On the Brocade DCX there are two ICL connectors at ports ICL0 and ICL1 on each core blade, each
aggregating a set of 16 ports. Thus, each core blade provides 32 ICL ports and there are 64 ICL ports
available for the entire Brocade DCX chassis. All the ICL connector ports must be connected to the same two
Brocade DCX or DCX-4S chassis.

The Brocade DCX-4S has 2 ICL connector ports at ICL0 and ICL1, each aggregating a set of 8 ports. Thus,
each core blade provides 16 ICL ports and there are 32 ICL ports available for the entire Brocade DCX-4S
chassis. All the ICL connector ports must be connected to the same two Brocade DCX or DCX-4S chassis.

2 ©2010 Brocade Communications


BCFP in a Nutshell 2010 Edition

Security
Switch Connection Control (SCC) and Device Connection Control (DCC) Policies

Figure 1: SCC and DCC Policies


• Ensures that only specified switches and devices may join a fabric
• Unspecified switches and devices are denied access to the fabric
• Works at the device login stage – enforced earlier than zoning
• SCC Policies:
Used to restrict which switches can join the fabric
-
Switches are checked against the policy each time an E_Port-to-E_Port connection is made
-
Only one SCC policy can be created per switch
-
To connect a Fibre Channel router to a fabric with an SCC policy, the front domain of the router must
-
be included in the SCC policy
- By default any switch is allowed to join the fabric
• DCC Policies:
- Used to restrict which device WWN can connect to which switch ports
- Each device WWN can be bound to one or more switch ports
- Multiple DCC policies can be configured
- Device and switch ports can be members of more than one DCC policy
- All devices are allowed to connect to all switch ports by default

©2010 Brocade Communications 3


BCFP in a Nutshell 2010 Edition

E_Port Authentication

The authentication (AUTH) policy allows you to configure DH-CHAP authentication on switches with Fabric OS
v5.3.0 and later.

When you configure the switches at both ends of a link to use DH-CHAP for authentication, you must also
define a secret key pair—one for each end of the link. Use the secauthsecret command to perform the
following tasks:
- View the WWN of switches with a secret key pair
- Set the secret key pair for switches
- Remove the secret key pair for one or more switches

Policy Distribution
• Each switch can be set to Accept or Reject individual security policies
• Use the fddcfg command to show or set each policy for Accept or Reject
• All policies can be manually distributed to fabric switches
• The IPFILTER, PWD and AUTH policies can only be manually distributed

The Fabric Wide Consistency Policy can be configured to automatically distribute SCC, DCC, and FCS policies
automatically. Fabric-Wide Consistency has three states:
• Absent (not defined): Fabric-Wide Consistency is not defined (default); manual database distribution is
required using the distribute command
• Tolerant: Switches are not required to have the same ACL policy databases
• Strict: Switches in the fabric always have the same ACL policy databases
- If one switch in a fabric has a strict policy, all switches in the fabric must also have a strict policy

This example shows how to set a strict (S after policy name means strict) SCC and tolerant (no notation after
policy name means tolerant) DCC fabric-wide consistency policy:
switch:admin> fddcfg --fabwideset "SCC:S;DCC“
switch:admin> fddcfg --showall
Local Switch Configuration for all Databases:-
DATABASE - Accept/Reject
---------------------------------
SCC - accept
DCC - accept
PWD - accept
FCS - accept
AUTH - accept
IPFILTER - accept

Fabric Wide Consistency Policy:- "SCC:S;DCC"

4 ©2010 Brocade Communications


BCFP in a Nutshell 2010 Edition

Advanced Device Security (ADS) Policy


• Restricts device access to the fabric at the Access Gateway
- Similar to the DCC Policy
• Only specified hosts are allowed to login to the Fabric
• Allowed host login can be configured per F_Port by specifying their Port WWN

©2010 Brocade Communications 5


BCFP in a Nutshell 2010 Edition

Access Gateway
• Designed to connect numerous servers with minimal impact to an existing fabric
• Focus is connectivity, bandwidth is shared
• Included in the base Fabric OS – no separate license is required
• Attached F_Port devices must be Fibre Channel Protocol (FCP) initiators or targets
- Not supported: loop devices, FICON, virtual iSCSI initiators
• Access Gateway ports can be configured as N_Ports which connect to the edge fabric
• Hosts/HBAs are mapped (through NPIV) to the N_Ports, and connect to the fabric through the N_Ports

Figure 2: AG Port Basics


• Access Gateway uses a port map to direct traffic from host HBAs to the N_Ports that connect to the fabric
- Each F_Port has a primary N_Port that is used to connect to the fabric
- The port map and N_Port configuration can be changed
• Enabling Access Gateway enables the default port group as seen in the Brocade 300 example below:
- N_Ports: 16 - 23
- Two F_Ports are mapped to each N_Port

Figure 3: Default Port Mapping

6 ©2010 Brocade Communications


BCFP in a Nutshell 2010 Edition

AG Port Grouping
• Port Grouping limits N_Port failover to occur within a user-defined group of ports
• Allows Access Gateway to be attached to multiple edge fabrics
• Default port group is enabled by default and requires all N_Ports to be connected to the same fabric
• User-defined port groups must be created to attach to more than one fabric

AG F_Port Trunking
• Trunking aggregates the bandwidth of the ports within the trunk group
• Available in Fabric OS v6.1.0 or later
• Needs to be configured as trunks do not automatically form
• Has the same requirements as ISL Trunking
- Trunking license on both AG and edge fabric switch
- Port group to port group
- Same speed
• Trunks use a shared port area ID/index called a Trunk Area (TA)
• Ports to be used for the F_Port Trunk are configured on the edge switch
• When configuring a TA, an index from one of the trunk port members is used for the TA
• Remaining F_Port Trunk members share the same TA and Port ID
• The porttrunkarea command is used on the edge switch to configure F_Port trunking

Auto Port Configuration

The Auto Port Configuration feature brings plug-and-play functionality to the Access Gateway. With Auto Port
Configuration enabled, there are changes to Access Gateway behavior:
1. No pre-determined port role – Based on the attached device or switch, each switch port automatically
configures as F_Port or N_Port
2. No pre-determined Port Map – F_Ports do not have a Primary N_Port and are evenly mapped (round-robin)
across all available N_Ports
3. No fixed port map – An F_Port is not guaranteed the same N_Port as devices/switches are added/
removed. When F_Ports and N_Ports are added or removed, the port map is automatically updated to
ensure an even distribution (round-robin) of F_Ports across N_Ports

©2010 Brocade Communications 7


BCFP in a Nutshell 2010 Edition

Virtual Fabrics

Virtual Fabrics is an architecture to virtualize hardware boundaries. Traditionally, SAN design and
management is done at the granularity of a physical switch. The Virtual Fabrics feature allows SAN design and
management to be done at the granularity of a port.

Enabling Virtual Fabrics is disruptive. A reboot of the chassis is required once it is enabled. Virtual Fabrics is
supported on the following platforms:
• Brocade 5100
• Brocade 5300
• Brocade DCX
• Brocade DCX-4S

Three types of Brocade Logical Switches:


• Default Switch (DS)
- Initially this contains all the ports in the chassis
- Default FID is 128
- Supports F / FL / E / VE Ports
• Logical Switch (LS)
- Standard Logical Switch
- Support for F / FL / E / VE Ports
• Base Switch (BS)
- Used for routing (sharing ports between Logical Switches)
- Used for ISL connections between logical switches
- Supports E/EX and VEX ports only

ISL Types
• ISL: Inter Switch Link - Standard ISL used to connect two non-virtual switches in the same fabric
- Can also be used to connect a virtual switch to a non-virtual switch in the same fabric
• IFL: Inter Fabric Link - IFL links are configured/enabled between an edge fabrics E_Port and an FC Router
EX_Port
• DISL: Dedicated Inter Switch Link - Used to directly connect two (non base) logical switches in two different
chassis
• XISL: Extended Inter Switch Link - Used to directly connect two base switches in two different chassis
• LISL: Logical Inter Switch Link - Used to connect two logical (non-base) switches in two different chassis via
the XISL on the base switch

8 ©2010 Brocade Communications


BCFP in a Nutshell 2010 Edition

FIDs and Domain IDs


• Every logical switch is created with a unique Fabric Identifier (FID) in the chassis
• All logical switches in a logical fabric must have the same FID, but must also have unique domain IDs

Figure 4: Virtual Fabric FIDs and Domain IDs

10-bit addressing mode

This is the default mode for all the logical switches created in the Brocade DCX and DCX-4S enterprise-class
platforms. This addressing scheme is flexible to support a large number of F_Ports. In the regular 10-bit
addressing mode, the portaddress --auto command supports addresses from 0x00 to 0x8F.

The 10-bit addressing mode utilizes the 8 bit area_ID and the borrowed upper 2 bits from the ALPA portion of
the PID. Areas 0x00 through 0x8F use only 8 bits for the port address and support up to 256 NPIV devices.
This means a logical switch can support up to 144 ports that can each support 256 devices. Areas 0x90
through 0xFF use additional 2 bits from ALPA for the port address. Hence these ports support only 64 NPIV
devices per port.

10-bit addressing mode allows for the following functionality:


• PID is dynamically allocated only when the port is first moved to a logical switch and thereafter it is
persistently maintained
• Removes shared area limitations on 48-port blades
• Any port on a 48-port blade can support up to 256 NPIV devices (In fixed addressing mode only
• 128 NPIV devices are supported in non-VF mode and 64 NPIV devices in VF mode on a 48-port blade)
• Any port on a 48-port blade can support loop devices
• Any port on a 48-port blade can support hard port zoning
• Port index is not guaranteed to be equal to the port area ID

©2010 Brocade Communications 9


BCFP in a Nutshell 2010 Edition

Converged Enhanced Ethernet (CEE)

CEE is an umbrella term for an Ethernet technology that has been enhanced by additional standards to meet
the requirements for transporting fibre channel frames. Some of these requirements are:
• A lossless, full-duplex Ethernet environment which provides in-order delivery and supports a minimum of
2.5 KB mini-jumbo frames
• Coexistence with classic Fibre Channel protocols
• High speed transport of Ethernet traffic

CEE enhancements to Ethernet:


• Data Center Bridging eXchange (DCBX) – IEEE 802.1Qaz
- Leverages functionality provided by IEEE 802.1AB Link Layer Discovery Protocol (LLDP)
- Used for conveying capabilities and configuration features between neighbors
• Priority Flow Control (PFC) – IEEE 802.1Qbb
- Link level flow control mechanism
- Controlled independently for each Class of Service (CoS)
- Goal of this mechanism is to ensure zero loss under congestion in DCB networks
• Enhanced Transmission Selection (ETS) – IEEE 802.1Qaz
- Enables bandwidth management by assigning bandwidth segments to different traffic flows

10 ©2010 Brocade Communications


BCFP in a Nutshell 2010 Edition

FCoE

FCoE provides a method of encapsulating Fibre Channel (FC) traffic into an Ethernet frame and is sent from
one FCoE-aware device, between CEE ports, to a second FCoE-aware device.

On the Brocade 8000 and FCOE10-24, Fabric OS 6.3.0 supports:


• Ethernet switching capability
• FC switching capability (Brocade 8000 only)
• FCoE-to-FC bridging capabilities

The FCOE10-24 blade:


• Supported in the DCX and the DCX-4S
• 24 x10 GbE Converged Enhanced Ethernet (CEE) ports
• No Fibre Channel ports
• Support for Fibre Channel over Ethernet (FCoE)
• Standard L2 Ethernet features
• Converged Enhanced Ethernet (CEE) switching
• No FCoE license required

Figure 5: Brocade 8000 and FCOE10-24

The Brocade 8000 contains two sets of ports:


• 24 CEE ports which are numbered 0–23 and supports a data speed of 10 GbE
• 8 FC ports which are numbered 0–7 and support data speeds from 1/2/4/8 Gbps

©2010 Brocade Communications 11


BCFP in a Nutshell 2010 Edition

2 - SAN Management

Audit Log

When managing SANs you may want to filter or audit certain classes of events to ensure that you can view and
generate an audit log for what is happening on a switch, particularly for security-related event changes. These
events include login failures, zone configuration changes, firmware downloads, and other configuration
changes—in other words—critical changes that have a serious effect on the operation and security of the
switch.

Before you configure audit event logging, familiarize yourself with the following audit event log behaviors and
limitations:
• By default, all event classes are configured for audit; to create an audit event log for specific events, you
must explicitly set a filter with the class operand and then enable it
• Audited events are generated specific to a switch and have no negative impact on performance
• The last 256 events are persistently stored on the switch and are streamed to a system message log
• The audit log depends on the system message log facility and IP network to send messages from the
switch to a remote host. Because the audit event log configuration has no control over these facilities,
audit events can be lost if the system message log and IP network facilities fail

DCFM

The Management application provides easy, centralized management of the SAN, as well as quick access to
all product configuration applications. Using this application, you can configure, manage, and monitor your
networks with ease.

Real-time performance data


Real-time performance enables you to collect data from managed devices in your SAN. Real-time perfor-
mance is only supported on the following managed objects: FC (E_ and F_ports), GE_ports, 10GE_ports, Man-
aged HBA Ports, and FCIP tunnels. You can use real-time performance to configure the following options:
• Select the polling rate from 10 seconds up to 1 minute
• Select up to 32 ports total from a maximum of 10 devices for graphing performance
• Choose to display the same Y-axis range for both the Tx MB/Sec and Rx MB/Sec measure types for easier
comparison of graphs

12 ©2010 Brocade Communications


BCFP in a Nutshell 2010 Edition

Fabric Binding
• The fabric binding feature enables you to configure whether switches can merge with a selected fabric.
This provides security from accidental fabric merges and potential fabric disruption when fabrics become
segmented because they cannot merge
• For M-EOS devices, enabling Fabric Binding activates Fabric Binding and enables insistent domain ID.
Disabling Fabric Binding on M-EOS devices deactivates Fabric Binding
• For Fabric OS devices, enabling Fabric Binding activates the Switch Connection Control (SCC) policy and
sets Fabric Wide Consistency Policy (FWCP) to strict and enables insistent domain ID. Disabling Fabric
Binding on Fabric OS devices deletes the SCC policy and sets FWCP to absent

High Integrity Fabrics

The DCFM High Integrity Fabric (HIF) mode option automatically enables features and operating parameters
that are necessary in multiswitch Enterprise Fabric environments. When HIF is enabled, each switch in the
fabric automatically enforces a number of security-related features including Fabric Binding, Switch Binding,
Insistent Domain IDs, and Domain Register for State Change Notifications (RSCNs).

For Pure Fabric OS fabrics, HIF activates the Switch Connection Control (SCC) policy, sets Insistent Domain ID,
and sets the Fabric Wide Consistency Policy (FWCP) for SCC to strict mode.

For mixed Fabric OS and M-EOS fabrics:


• For Fabric OS switches, HIF activates the SCC policy, sets Insistent Domain ID, and sets the FWCP for SCC
in tolerant mode
• For M-EOS switches, HIF activates Enterprise Fabric Mode, Fabric Binding, Switch Binding, Insistent
Domain ID, and RSCNs

©2010 Brocade Communications 13


BCFP in a Nutshell 2010 Edition

HCM

The Host Connectivity Manager (HCM) is a management software application for configuring, monitoring, and
troubleshooting Brocade HBAs and Converged Network Adapters (CNAs) in a Storage Area Network (SAN)
environment.

Common HBA and CNA management software features include the following:
• Discovery using the agent software running on the servers attached to the SAN, which enables you to
contact the devices in your SAN.
• Configuration management, which enables you to configure local and remote systems. With HCM you can
configure the following items:
- Local host
- Brocade 4 Gbps and 8 Gbps HBAs
- HBA ports (including logical ports, base ports, remote ports, and virtual ports)
- Brocade 10 Gbps single-port and 10 Gbps dual-port converged network adapters (CNAs)
- CEE ports
- FCoE ports (CNA only)
- Ethernet ports (CNA only)
• Diagnostics, which enables you to test the adapters and the devices to which they are connected:
- Link status of each adapter and its attached devices
- Loopback test, which is external to the adapter, to evaluate the ports (transmit and receive trans-
ceivers) and the error rate on the adapter
- Read/write buffer test, which tests the link between the adapter and its devices
- FC protocol tests, including echo, ping, and traceroute
- Monitoring, which provides statistics for the SAN components
- Security, which enables you to specify a CHAP secret and configure authentication parameters
- Event notifications, which provide asynchronous notification of various conditions and problems
through a user-defined event filter

The Host Connectivity Manager (HCM) Port Statistics window enables you to monitor the performance of the
CNA and the traffic between the CNA and the LUNs. You can use the information to isolate and troubleshoot
areas that impact application performance.

14 ©2010 Brocade Communications


BCFP in a Nutshell 2010 Edition

Table 1: Statistics monitored by component

©2010 Brocade Communications 15


BCFP in a Nutshell 2010 Edition

SAN Health
• Automates documenting of a SAN and comes in different formats
• SAN Health is a free utility that aids in creating:
- Comprehensive documentation
- Historical performance graphs
- Detailed topology diagrams

SNMP
• SNMP is a standard method for monitoring and managing network devices (both Ethernet and Fibre
Channel)
• SNMP has the following components:
- SNMP Entities: Network Management Stations and Agents
- Management Information Bases (MIBs) - The agent accesses MIB information about the switch and
makes it available to a network management station
• Every Brocade switch runs an SNMP agent and Management Information Base (MIB)

Fabric Watch
• Optionally licensed per switch, it monitors the performance and status of the switch, including:
- Fabric events – Fabric reconfigs, zone changes, and new logins
- Switch status – Environmental (fans, power supplies, and temperature), SFP (Tx/Rx power, current,
& voltage), Security, resource and FRU
- Port status – Monitors F/FL/E_Port signal quality parameters
- Performance options – monitor end-to-end performance
• Fabric Watch maintains a set of counters for each of the monitored conditions
- Tracks the number of occurrences of each condition
- Each counter is compared with an upper boundary and lower boundary

Virtual Fabric Support for Fabric Watch - Fabric Watch can monitor the switch health on eight logical switches.
You can configure thresholds and alarms for ports that belong to a particular logical switch. Each logical
switch has its own Fabric Watch configuration and triggers alarms based on its local configuration.

Port Fencing

A port that is consistently unstable can harm the responsiveness and stability of the entire fabric and
diminish the ability of the management platform to control and monitor the switches within the fabric. Port
Fencing is a Fabric Watch enhancement that takes the Port class, E_Port class, and F/FL_Port class ports
offline if the user-defined thresholds are exceeded.

When a port that has exceeded its user-defined thresholds is fenced by software, the port is placed into the
disabled state and held offline, thereby removing the port’s ability to transmit or receive frames. After a port is
disabled, user intervention is necessary for frame traffic to resume on the port.

16 ©2010 Brocade Communications


BCFP in a Nutshell 2010 Edition

3 - Adaptive Networking

Ingress Rate Limiting


• A licensed feature available only on 8 Gbps platforms/blades (Adaptive Networking license)
• Allows the ASIC to delay the return of R_RDY or VC_RDY primitives to the external device by throttling back
the ingress port speed, thereby limiting the throughput on the ingress side of the port
• Ingress Rate Limiting is only supported for F/FL_Ports
• It is not supported for E/EX_Ports
• Ingress Rate Limiting is designed to help alleviate choke points in the fabric caused by slow drain devices,
congested ISLs, etc.
• Example use cases:
- To reduce existing congestion in the network or proactively avoid congestion
- To enable more important devices to use the network bandwidth during specific services, such as
network backup

Traffic Isolation Zones


• The Traffic Isolation feature allows you to control the flow of inter-switch traffic by creating a dedicated path
for traffic flowing from a specific set of source ports (N_Ports)
• Traffic Isolation is supported on Condor, Condor2, GoldenEye and GoldenEye2-based switches running
Fabric OS v6.0 or later
• Does not require a license
• Can create a dedicated route
• Do not modify the routing table
• Are implemented across the entire data path from a single location
• Traffic Isolation zones use a special zone command that indicates the set of N_Ports and E_Ports to be
used for a specific traffic flow
• Examples for isolating traffic are:
- Dedicate an ISL for high priority host to target traffic
- Force high volume, low priority traffic onto a given ISL to limit the effect of that traffic on the overall
fabric
- Separate FICON traffic from Open Systems traffic
- A storage system needs to have its data restored from a tape device, and the two systems should
have a dedicated connection

©2010 Brocade Communications 17


BCFP in a Nutshell 2010 Edition

QoS - SID/DID Based Traffic Prioritization


• SID/DID traffic prioritization is a licensed feature (Adaptive Networking)
• SID/DID traffic prioritization enables the setting of priorities between specific hosts and targets
• Allows for the control of frame flow in the fabric when contention occurs
• FSPF route selection is not affected by QoS priorities
• Focus is on latency, not bandwidth
• The QoS features are engaged when there is contention on the link
• Contention occurs if:
- Multiple frames arrive at the link at the same time
- Bandwidth is congested on the link
• If there is no contention on the link QoS will not engage
• The order of operations during congestion is to send frames in the order listed below:
- VC0 Class F frames
- High priority traffic
- Medium priority traffic
- Low priority traffic
• Prioritization is accomplished by the use of QoS zones
- Appear as normal zones
- Regular zoning rules apply
- Can be created using WWN or D,I (domain,index) notation
o D,I notation requires Fabric OS v6.3.0 or later
• To distinguish QoS zones from normal zones, special prefixes are used in the zone names:
- QOSH_ to set high priority
- QOSL_ to set low priority
- Prefixes are not case sensitive
• Default setting is medium priority which is used when no QoS zones are specified or when QoS is not
enforced.

.
• Traffic that requires lower latency is placed in front of the queue

QoS Traffic Prioritization on Brocade HBAs


• Can be enabled on Brocade 8 Gbps capable HBAs only
- Not supported on Brocade 4 Gbps HBAs or Brocade CNAs
• Works with the QoS feature on Brocade switch F_ports
• Requires Server Application Optimization (SAO) and Adaptive Networking license

18 ©2010 Brocade Communications


BCFP in a Nutshell 2010 Edition

Bottleneck Detection
• The bottleneck detection feature identifies devices attached to the fabric that are slowing down traffic
(slow drain device)
• A slow drain device is slow to process received frames and send back credit returns
• Actual throughput into the slow drain port is lower compared to intended throughput
• A slow destination returns credits slower than the sender wants to send frames to it
• Slow drain can exist at any link utilization level from 0% to under 100%; not just high utilization scenarios
• Slow drain spreads into the fabric and can slow down unrelated flows in the fabric
• Commands to manage the bottleneck detection feature:
bottleneckmon --enable
bottleneckmon --show
• Does not require a license
• Is supported on Condor2/GoldenEye2 ASIC Fibre Channel ports only
• F_ and FL_Ports only
• F_Port trunks are not supported
• Bottleneck detection is disabled by default, and must be explicitly enabled for each port that is to be
monitored

Top Talkers
• Top Talkers (TT) is an enhancement to Advanced Performance Monitor (APM) end-to-end monitors
• When enabled, these monitors determine which SID-DID pairs are the major users of switch F_Port
bandwidth
• Can be enabled on specific switch E_Ports or F_Ports in the fabric
- Port Mode: enabled on an F_Port to measure the traffic between the F_Port and all other devices
that it can communicate with
- Fabric Mode: enabled on all E_Ports in the fabric to measure the data rate of all flows in the fabric
(ingress E_Port traffic only)
• Can be configured in either Port Mode or Fabric Mode, but not both simultaneously
• Determines the flows (SID/DID pairs) that are the major users of bandwidth
• Measures bandwidth usage data in real-time and relative to the port on which the monitor is installed
• Requires APM license
• Fabric Mode Top Talker monitors and end-to-end monitors cannot both exist in the fabric
• Data collected is deleted when the switch is rebooted

©2010 Brocade Communications 19


BCFP in a Nutshell 2010 Edition

How do Top Talker monitors differ from end-to-end monitors?

End-to-end monitors provide counter statistics for traffic flowing between a given SID-DID pair. Top Talker
monitors identify all possible SID-DID flow combinations that are possible on a given port and provides a
sorted output of the top talking flows. Also, if the number of flows exceeds the hardware resources, existing
end-to-end monitors fail to get real time data for all of them; however, Top Talker monitors can monitor all
flows for a given E_Port or F_Port (up to 10,000 flows).

20 ©2010 Brocade Communications


BCFP in a Nutshell 2010 Edition

4 - FC - FC Routing
• Fabric OS provides L3 Fibre Channel-to-Fibre Channel routing (FC-FC Routing) between fabrics
• Allows device access between two or more fabrics without merging the fabrics
- LSAN zoning must be enabled in all fabrics that share devices
o Edge-to-edge routing: edge fabrics
o Backbone-to-edge routing: backbone and edge fabrics
• FC-FC routing is supported between the following fabric types:
- Fabric OS-to-Fabric OS
- Fabric OS-to-M-EOS
- M-EOS-to-M-EOS
• M-EOS edge fabric support:
- M-EOS v07.00 and higher
- No VEX_Ports
• Only Fabric OS switches are allowed in the backbone fabric
- Backbone fabric must be set for Brocade Native Mode (Interopmode 0)
• In configurations with two backbones connected to the same edge fabric, routing is not supported between
edge fabrics that are not directly attached to the same backbone. Routing over multiple backbones is a
multi-hop topology and is not allowed.
• Every edge and backbone fabric requires a unique Fabric ID (FID)
• Enabling FC Routing services is done by using the fosconfig command

Integrated Routing
• An Integrated Routing license is required to allow configuration of FC-FC routing capable ports called
EX_Ports
- EX_Port: A type of E_Port used to connect an FC router port to an edge fabric without merging the
two
o EX_Ports on a router connect to E_Ports in an edge fabric
• License enforcement is checked on configuration and when enabling the EX_Port
- If the license is removed while EX_Ports are online, Condor2/GoldenEye2 EX_Ports will continue to
function until the next fabric rebuild, switch disable or port offline event
• Integrated Routing is currently supported on:
- Brocade FC8, FX8 and FS8 blades in a DCX and DCX-4S chassis
- Brocade 5100, 5300, 7800 and Brocade Encryption Switch (BES)

©2010 Brocade Communications 21


BCFP in a Nutshell 2010 Edition

LSAN Zones
• Zones that define which devices are to be shared between fabrics
• Defined in each fabric that will share devices (edge or backbone)
• Is a traditional zone created using normal zoning tools but uses a special naming prefix “lsan”

Once the LSAN zones are active, determine which devices actually exist in the fabric, and which ones are
imported. Executed from the FC router in the backbone:

BB_B51:admin> lsanzoneshow -s
Fabric ID: 10 Zone Name: LSAN_Backbone1_Edge1
10:00:00:05:1e:57:7c:79 EXIST
22:00:00:20:37:dd:d9:29 Imported
Fabric ID: 100 Zone Name: lsan_backbone1_edge1
10:00:00:05:1e:57:7c:79 Imported
22:00:00:20:37:dd:d9:29 EXIST

• Configured - Device is configured to be in an LSAN, but the device is not imported nor does it exist in
this fabric
• EXIST - Device exists in this fabric (the fabric of the zone entry)
• Initializing -Device is in an intermediate state. It is not yet imported into the fabric
• Imported - Device has been imported (proxy created) into this fabric

22 ©2010 Brocade Communications


BCFP in a Nutshell 2010 Edition

Proxy devices

Figure 6: Phantom domains and devices

A FC router achieves interfabric device connectivity by creating proxy devices (hosts and targets) in attached
fabrics that represent real devices in the originating fabrics. For example, a host in Fabric 1 can communicate
with a target in Fabric 2 as follows:
• A proxy target in Fabric 1 represents the real target in Fabric 2
• Likewise, a proxy host in Fabric 2 represents the real host in Fabric 1

The host discovers and sends Fibre Channel frames to the proxy target. The FC router receives these frames,
translates them appropriately, then delivers them to the destination fabric for delivery to the target.

The target responds by sending frames to the proxy host. Hosts and targets are exported from the edge SAN
to which they are attached and, correspondingly, imported into the edge SAN reached through Fibre Channel
routing.

Phantom domains

A phantom domain is a domain emulated by the Fibre Channel router. The FC router can emulate two types of
phantom domains: front phantom domains and translate phantom domains. A front phantom domain is a
domain that is projected from the FC router to the edge fabric. There is one front phantom domain from each
FC router to an edge fabric, regardless of the number of EX_Ports connected from that router to the edge
fabric. Another FC router connected to the same edge fabric projects a different front phantom domain.

The second level of phantom domains is known as a translate phantom domain, also referred to as translate
domain or xlate domain. The translate phantom domain is a router virtual domain that represents an entire
fabric. Device connectivity can be achieved from one fabric to another—over the backbone or edge fabric

©2010 Brocade Communications 23


BCFP in a Nutshell 2010 Edition

through this virtual domain—without merging the two fabrics. The EX_Ports present translate phantom
domains in edge fabrics as being topologically behind the front domains; if the translate phantom domain is in
a backbone fabric, then it is topologically present behind the FC router because there is no front domain in a
backbone fabric.

If a FC router is attached to an edge fabric using an EX_Port, it creates xlate domains in the fabric
corresponding to the imported edge fabrics with active LSANs defined. If you import devices into the
backbone fabric, then an xlate domain is created in the backbone device in addition to the one in the edge
fabric.

Display the physical devices with the fcrphydevshow command:

BB_B51:admin> fcrphydevshow
Device WWN Physical
Exists PID
in Fabric
-----------------------------------------
10 10:00:00:05:1e:57:7c:79 030100
100 22:00:00:20:37:dd:d9:29 6206e4
Total devices displayed: 2

In the command output we see the two physical devices that are currently being shared across the backbone
fabric:
- Edge fabric (FID 10) has one physical device PID 030100
- Backbone fabric (FID 100) has one physical device PID 6206e4

Display proxy devices with the fcrproxydevshow command:


BB_B51:admin> fcrproxydevshow
Proxy WWN Proxy Device Physical State
Created PID Exists PID
in Fabric in Fabric
---------------------------------------------------------------------------
10 22:00:00:20:37:dd:d9:29 01f001 100 6206e4 Imported
100 10:00:00:05:1e:57:7c:79 01f001 10 030100 Imported
Total devices displayed: 2

In the command output we see the two proxy devices that are currently being shared across the backbone
fabric:
• The format of the proxy address is XXfYYY, with XX indicating the translate domain ID, and YYY is a value
beginning at 001

24 ©2010 Brocade Communications


BCFP in a Nutshell 2010 Edition

5 - FCIP and Extension

FCIP

Fibre Channel over IP (FCIP) enables you to use existing IP wide area network (WAN) infrastructure to connect
Fibre Channel SANs. The Fibre Channel fabric and all Fibre Channel targets and initiators are unaware of the
presence of the IP network.

Port Types
• An FCIP tunnel is represented in a Brocade fabric as a Virtual E_Port (VE_Port)
- Just like an E_Port, except underlying transport is IP not FC
• The VE_Port emulates an E_Port on either end of the FCIP tunnel:
- The FCIP platforms at both ends of the links merge to form a single fabric
- VE_Ports do not use FC flow control mechanisms (BB Credits); they utilize TCP flow control mecha-
nisms
- VE_Ports do not support FC ISL trunking, but they do support exchange-based routing (Dynamic
Path Selection)
• Note that with FCIP Trunking, it is recommended to implement a multiple circuit trunk instead of having
multiple VE ports to the same fabric
• The Brocade 7800 supports FC-FC routing over an FCIP tunnel, creating a Virtual EX_Port (VEX_Port)
- Allows long-distance FCIP connections with fabric-to-fabric isolation
- VEX_Ports are no different from EX_Ports, except underlying transport is IP rather than FC
• There are a few connectivity rules with VEX_Ports:
- A VEX_Port connects only to a VE_Port – it may not connect to another VEX_Port
- There can be multiple VEX-to-VE port connections between a Backbone fabric and an Edge fabric
- EX-to-E and VEX-to-VE connections to the same Edge fabric can co-exist in Fabric OS v5.2 and later
fabrics

FCIP Licenses

Fabric OS v6.3.0 available licenses for extension products:


• Brocade 7800 Port Upgrade
- Brocade 7800 4/2 to Brocade 7800 16/6
• 10 GbE FCIP (FX8-24 only) (slot-based)
• Integrated Routing (IR)
• Advanced Extension (slot-based)
- FCIP Trunking
- Adaptive Rate Limiting
• Brocade Accelerator for FICON
• FICON Management Server (CUP)

©2010 Brocade Communications 25


BCFP in a Nutshell 2010 Edition

Hardware

Brocade 7800

Figure 7: Brocade 7800


• The Brocade 7800 Extension Switch has two configurations:
- Brocade 7800 4/2 – 4 x 8 Gbps FC Ports and 2 x 1 GbE Ports
- Brocade 7800 16/6 – 16 x 8 Gbps FC Ports and 6 x 1 GbE Ports (Brocade 7800 Port Upgrade)
• One GoldenEye2 ASIC for FC ports
• One FCIP subsystem
• 2 x 1 GbE RJ45 copper ports
• 6 x 1 GbE optical SFP ports
• 16 x 8 Gbps FC ports
• Trunk groups: 0-7 and 8-15
• Supported port types are:
- Fibre Channel - F, FL, E, EX or Mirror
- GbE - VE and VEX

In a standard configuration, users have the option of using either the top two GbE ports, which are configured
for copper, or the bottom two, left-most, GbE ports (ge0 and ge1) which are configured for optical SFPs. The
remaining four GbE ports use optical SFPs. It is possible to configure ge0 as copper and ge1 as optical and
vice versa.

26 ©2010 Brocade Communications


BCFP in a Nutshell 2010 Edition

FX8-24

Figure 8: FX8-24
• The Brocade FX8-24 Extension Blade has:
- 2 x 10 GbE ports (license required)
- 10 x 1 GbE ports
- 12 x 8 Gbps FC ports
• One Condor2 ASIC
• Max of 2 blades supported in both the DCX and DCX-4S as of Fabric OS v6.3.0
• One Condor2 ASIC for FC ports
• Two FCIP subsystems
• 3 FC trunk groups:
- Ports 0 - 1
- Ports 6 - 7
- Ports 2 - 5 and 8 - 11
• Supported port types as of Fabric OS v6.3.0 are:
- Fibre Channel - F, FL, E, EX and Mirror
- GbE - VE

FCIP Virtual Fabric Support


• FX8-24 support for Fabric OS v6.3.0:
- 8 Gbps FC ports can be assigned to any logical switch
- VE_Ports can be configured in any logical switch
- VE_Ports cannot be used as XISL
- VEX_Ports are not supported
• No Virtual Fabric support for Brocade 7800 in v6.3.0
- Cannot create logical switches on Brocade 7800
- Brocade 7800 can be connected to other switch in Virtual Fabric mode

©2010 Brocade Communications 27


BCFP in a Nutshell 2010 Edition

FCIP Tunnel Configuration

Follow these basic steps to create a VE-to-VE_Port tunnel:


1. Determine the required parameters for the VE_Ports and FCIP tunnel
2. Configure hardware ports if necessary for appropriate media type and mode
3. Persistently disable the virtual FC E_Ports (VE) associated with the tunnel

4. Create an IP interface for each physical Ethernet port to be used

5. Configure an IP route for each port to specify an IP gateway (optional)


6. Verify the IP network between the two IP interfaces that will form the FCIP tunnel

7. Create an FCIP tunnel (circuit 0 will automatically be created)

8. As needed, configure FCIP features (SACK, compression, etc.)


9. Verify configuration, enable the associated VE_Port, and validate functionality

10. Add additional circuits to defined tunnel

Commands supported on the Brocade 7800/FX8-24 platforms:


portcfg action [slot/] ge_port arguments
Use this command to configure the local IP interfaces and static routes on the Brocade 7800/FX8-24
platforms. You must configure the local IP interfaces before you can create and configure FCIP
tunnels. The procedure is essentially the same for both platforms.

portcfg ipif create src_ipaddr mtu_size


Create the local IP interface

portcfg iproute create dest_ipaddr [gateway_router] metric


Create a static route on the IP interface

portcmd --ping [slot]/port


Verify the link between FCIP end devices

portcfg fciptunnel [slot/][ge] create args [optional_args]


Optional tunnel_arguments for fciptunnel create and modify include:
-f |--fastwrite 0|1] Enables (1) or disables (0) FastWrite on the specified FCIP tunnel.
-t |--tape-pipelining 0|1 Enables (1) or disables (0) Tape Pipelining on the specified
FCIP tunnel. If Tape Pipelining is enabled, FastWrite should also be enabled.
-c |--compression compression_level Configures compression on the specified FCIP
tunnel. By default, compression is disabled (0). Specify one of the following values:
o 0 Compression disabled
o 1 Standard compression
o 2 Moderate compression (Brocade 7800 only)

28 ©2010 Brocade Communications


BCFP in a Nutshell 2010 Edition

o 3 Aggressive (Brocade 7800 only)


-T |--tperf 0|1 Enables (1) or disables (0) TPerf test mode.
-n |--remote-wwn remote-wwn Specifies the WWN of the remote FC entity.
-d |--description string Specifies a description for the specified tunnel.
-F |--ficon 0|1 Enables (1) or disables (0) FICON emulation on the specified FCIP tunnel.
Optionally, a value can be set for a minimum and a maximum committed rate to configure the tunnel
for Adaptive Rate Limiting (ARL), which allows for a more effective sharing of bandwidth between
applications. The valid range is 1544 Kbps - 1000000 Kbps. Both sides of the tunnel must have
matching configurations.
-b | --min-comm-rate minimum
Modifies the minimum committed traffic rate on the FCIP circuit 0 in Kbps.
-B | --max-comm-rate maximum
Modifies the maximum committed traffic rate on the FCIP circuit 0 in Kbps.

portcfg fcipcircuit <ve-port> <create|modify|delete> <circuitId>


[<parameters>]
Create parameters:
<remoteIp> <localIp> <commitedRate> [<optional args>]
or
<remoteIp> <localIp> --min-comm-rate <kbps> --max-comm-rate <kbps> [<optargs>]

optional circuit args:


-a, --admin-status <0|1> - enable/disable the circuit
-s, --sack - turn sack off
-k, --keepalive-timeout <ms> - set the keepalive timeout in ms
-x, --metric <metric> - set the circuit metric
-b, --min-comm-rate <kbps> - set min comm rate value in kbps
-B, --max-comm-rate <kbps> - set max comm rate value in kbps
-m, --min-retrans-time <ms> - set min retrasmit time in ms
-r, --max-retransmits <rtx> - set maximum number of retransmits
-v, --vlan-tagging <vlan-id> - set the vlan-id for the circuit
--l2cos-f-class <l2cos> - set the L2CoS value for F-Class Traffic
--l2cos-high <l2cos> - set the L2CoS value for High Priority
--l2cos-medium <l2cos> - set the L2CoS value for Medium Priority
--l2cos-low <l2cos> - set the L2CoS value for Low Priority
--dscp-f-class <dscp> - set the DSCP value for F-Class Traffic
--dscp-high <dscp> - set the DSCP value for High Priority
--dscp-medium <dscp> - set the DSCP value for Medium Priority
--dscp-low <dscp> - set the DSCP value for Low Priority

©2010 Brocade Communications 29


BCFP in a Nutshell 2010 Edition

Circuit Metric
• Each circuit will be configured with a metric of 0 (active) or 1 (standby)
• The metric will be used by the tunnel supervisor to determine which circuit or circuits will be used as active
circuits
• Metric 0 circuits have the lowest metric and will be designated the active circuits and will be used for all
data transfers
• Metric 1 circuits are classified as standby circuits. It is in standby mode in the event that all metric 0
circuits fail

Figure 9: Circuit Metric

FCIP Per-Priority TCP QoS

Provides for appropriate bandwidth usage when multiple FC QoS priorities are running over the same FCIP
circuit. It allows QoS traffic to be distributed as follows:
• F-Class: Gets what it needs. This is the highest priority
• QoS High: Gets at least 50% of the bandwidth
• QoS Medium: Gets at least 30% of the bandwidth
• QoS Low: Gets at least 20% of the bandwidth
• Extends QoS zoning that exists in fabric
• Identifies incoming virtual channels on a FC ISL
• QoS is not enforced unless there is congestion
• Lower priority flows are never limited if higher priority flows are not currently utilizing the BW
• Different TCP sessions can be treated independently

Figure 10: FCIP QoS

30 ©2010 Brocade Communications


BCFP in a Nutshell 2010 Edition

Adaptive Rate Limiting (ARL)


• Adaptive Rate Limiting (ARL) provides for an adaptive committed rate configuration on an FCIP circuit
- This is implemented by configuring a minimum and a maximum committed rate
• ARL allows the traffic rate on a circuit to float between the minimum and maximum
- If there is traffic demand from FCIP and the network connection is clean (no retransmits) then the
rate will grow
- If TCP reports retransmits, the rate will retreat back to the minimum
- If traffic demand subsides, the rate will shrink backwards
- Growth is accomplished by testing the ceiling and stepping the available bandwidth up gradually
• This provides more efficient bandwidth sharing between applications using the same network
infrastructure
• Requires the Advanced Extension license

QoS implementation over FCIP

Fabric OS versions 6.0.0 and later provide for Fibre Channel QoS through internal QoS priorities. Those
priorities can be mapped to TCP/IP network priorities. There are two options for TCP/IP network-based QoS:
• Layer three DiffServ code Points (DSCP)
• VLAN tagging and Layer two class of service (L2CoS)

DSCP quality of service - Layer three class of service DiffServ Code Points (DSCP) refers to a specific
implementation for establishing QoS policies as defined by RFC2475. DSCP uses six bits of the Type of
Service (TOS) field in the IP header to establish up to 64 different values to associate with data traffic priority.
DSCP settings are useful only if IP routers are configured to enforce QoS policies uniformly within the network.
IP routers use the DSCP value as an index into a Per Hop Behavior (PHB) table. Control connections and data
connections may be configured with different DSCP values. Before configuring DSCP settings, determine if the
IP network you are using implements PHB, and consult with the network administrator to determine the
appropriate DSCP values.

L2CoS quality of service - Devices in physical LANs are constrained by LAN boundaries. They are usually in
close proximity to each other, and share the same broadcast and multicast domains. Physical LANs often
contain devices and applications that have no logical relationship. Also, when logically related devices and
applications reside in separate LAN domains, they must be routed from one domain to the other. A VLAN is a
virtual LAN network. A VLAN may reside within a single physical network, or it may span several physical
networks. Related devices and applications that are separated by physical LAN boundaries can reside in the
same VLAN. Also, a large physical network can be broken down into smaller VLANs. VLAN traffic is routed
using 802.1Q-compliant tags within an Ethernet frame. The tag includes a unique VLAN ID, and Class of
Service (CoS) priority bits. The CoS priority scheme (also called Layer two Class of Service or L2CoS), uses only
the upper three bits of the TOS field, allowing eight priorities.

©2010 Brocade Communications 31


BCFP in a Nutshell 2010 Edition

Selective Acknowledgement
• Packet loss significantly degrades FCIP performance, lost data needs to be retransmitted
• Each lost packet requires a separate ACK response packet
• To mitigate this, Fabric OS supports Selective Acknowledgement (SACK) – (default ON)
• When SACK is enabled on a receiving VE/VEX_Port, the retransmission of multiple lost packets can be
combined in a single ACK packet

Long Distance

The connecting ports on the bookend switches must to be set to the same long distance parameters.Use the
portcfglongdistance CLI command to specify an Extended Fabric Distance Level:
• Level 0 static mode (L0) is the normal mode for a port
• Level E static mode (LE) reserves a static number of buffer credits that supports distances up to 10 km
- The number reserved depends on the port speed
• Dynamic long distance Mode (LD) calculates buffer credits based on the distance measured during port
initialization
- An upper limit is placed on the calculation by providing a desired distance value
• Static long distance mode (LS) calculates a static number of buffer credits based on a desired distance
value

Note: L0 and LE modes do not require a license. Use of LD and LS modes requires an Extended Fabric
License.

Buffer Credits

While exact calculations are possible, a simple rule of thumb is used in the calculation of the BB credit
requirement for a given link. The use of QoS causes more BB credits to be allocated to the port due to the
additional VCs (8 – 14).

Based on the speed of light in an optical cable being 5 ns/m, the rule of thumb is that a full-size FC frame
spans approximately:
• 4 km at 1 Gbps
• 2 km at 2 Gbps
• 1 km at 4 Gbps
• 500 m at 8 Gbps
• 400 m at 10 Gbps

32 ©2010 Brocade Communications


BCFP in a Nutshell 2010 Edition

For example, to keep a link that spans a distance of 10 km at full speed would require:
• 5 credits per port at 1 Gbps
• 10 credits per port at 2 Gbps
• 20 credits per port at 4 Gbps
• 40 credits per port at 8 Gbps

If your payload size is smaller than 2112 bytes, use the following formula to calculate exact number of
minimum buffer credit requirements:

buffer credits = [(distance in km) * (data rate in Gbps) * 1000] / (payload size)

Average frame size can be calculated using a Fibre Channel Analyzer or application-level data.

Long Distance Trunking

Criteria for trunking across an Extended Fabric:


• All trunking criteria must be met
• Extended Fabric and ISL Trunking licenses on supported switches at both sides of the ISL
• All trunk ports in the same trunk group must be configured to the same long distance mode
• Up to eight ports can be grouped together in one trunk group to create high performance 64 Gbps ISL
trunks between switches
• Trunk links can be 2 Gbps, 4 Gbps, or 8 Gbps depending on the Brocade platform, or individual port speed
settings
• Trunks operate best when the cable length of each trunked link is roughly equal to the others in the trunk.
For optimal performance, no more than 30 meters difference is recommended

©2010 Brocade Communications 33


BCFP in a Nutshell 2010 Edition

Protective Switching

Protection switching provides insurance against a fiber cut by optically splitting the data streams over two
diverse paths. Most WDM and SONET/SDH equipment has the ability to perform a protection switch from the
active path to the standby path in less than 50 ms in order to maintain connectivity between sites.

In the example below, assume LD mode has been set as the long distance setting, with 50 km as the desired
distance, and that data is using the 50 km side of the WDM ring. If the 50 km link fails, the WDM equipment
will switch to the 80 km side of the ring long before the loss-of-sync timer in the switch expires, resulting in no
notification of a change in path, or the need for more buffer credits. The switch is still supplying enough buffer
credits for 50 km, but our new path is 80 km, so the link is being starved. In this situation, it would be better
to specify LS long distance mode, and set the distance for the longer side of the WDM ring.

Figure 11: Protective Switching

34 ©2010 Brocade Communications


BCFP in a Nutshell 2010 Edition

6 - Troubleshooting

Gathering Information

Run Fabric OS supportsave as soon as you experience a problem in your SAN


• Run supportsave prior to beginning any troubleshooting
• Critical problem determination data will be captured if supportsave is run right away
• If unable to resolve then run supportsave again for recent activity
• Prior to Fabric OS v6.2, supportsave needs to be run on both CPs on a director

If problem escalation is required, send the escalation team all relevant supportsave files
• Baseline, prior to troubleshooting, after troubleshooting
• Maintain a baseline supportsave taken after every configuration change

For Brocade HBAs, use the bfa_supportsave command.

ISL Physical Connection Issues


• Check the cables running to and from the two switches. Verify that none of the cables are damaged,
including indentations or bent cable
• Check the SFP on both switches. Verify that they are known to be in good working condition. You can do
this by swapping the current SFP with a known good working SFP
• Clean the optics. There are many kits on the market for cleaning fiber optics. You want to find a product
that does not leave residue either from a lint-free wipe or from the solvent

Slow Drain Device


• A slow drain device returns credits slower than the sender wants to send frames to it
• Slow drain can exist at any link utilization level
• Achieved throughput into the slow drain port is lower compared to intended throughput
• Slow drain spreads into the fabric and can slow down unrelated flows in the fabric
• Bottleneck detection finds bottlenecks at the egress side of the port. It is implemented by the
bottleneckmon --enable command
• Use the portperfshow command to display throughput information for all ports on a switch or chassis.
Output includes the number of bytes received and transmitted per interval

Speed Negotiation

If there is a speed negotiation problem, complete the following:


1. Enter the portcfgshow command to display the port speed settings of all the ports.
2. Enter the switchshow command to determine if the port has module light.

©2010 Brocade Communications 35


BCFP in a Nutshell 2010 Edition

3. Enter the portcfgspeed command to change the port speed to 1, 2, 4 or 8 Gbps, depending on what
speed can be used by both devices. This should correct the negotiation by setting to one speed.
4. Enter the portlogshow or portlogdump command.
5. Check the events area of the output:
time task event port cmd args
-----------------------------------------------------------------
14:38:51.976 SPEE sn <Port#> NC 00000001,00000000,00000001
14:39:39.227 SPEE sn <Port#> NC 00000002,00000000,00000001
- In the event column sn indicates a speed negotiation
- In the cmd column NC indicates the negotiation has completed

Segmented Switches
When switches are segmented, you can use the switchshow, configshow, fabricshow, fabstatss-
how, portshow, and portcfgshow commands to help troubleshoot the segmentation. Also check zone
related commands and the license configuration. You can also use the DCFM Zone Merge Tool to compare
zones and then merge the zone configurations.

A switch is not allowed to merge with another switch that has an active effective configuration if the default
zone is set to “no access”. Before the switch can join, the default zone setting has to be set to "all access".
When the default zone no access option is enabled and the active configuration is disabled by using the
cfgdisable command, a special hidden configuration with no members is activated. This configuration will
not allow the switch to merge with switches that have an active effective configuration.

Device Connectivity Problems

Enter the fcping command, which checks the zoning configuration for the two ports specified by:
• Generates an ELS (Extended Link Service frame) ECHO request to the source port specified and validates
the response
• Generates an ELS ECHO request to the destination port specified and validates the response
Regardless of the device’s zoning, the fcping command sends the ELS frame to the destination port. A
device can take any of the following actions:
• Send an ELS Accept to the ELS request
• Send an ELS Reject to the ELS request
• Ignore the ELS request

36 ©2010 Brocade Communications


BCFP in a Nutshell 2010 Edition

Fabric Changes
Changes to the fabric can be identified by using the fabriclog -s and fabstatsshow commands. The
fabriclog command with the -s flag displays the fabric log.

Use the fabstatsshow command to display the statistics for the fabric. The following information is
displayed:
• Number of times a switch domain ID has been forcibly changed
• Number of E_Port offline transitions
• Number of fabric reconfigurations
• Number of fabric segmentations as a result of any of the following causes:
- Loopback
- Incompatibility
- Overlap
- Zoning
- E_Port segment
- Licensing
- Disabled E_Port
- Platform DB
- Security incompatibility
- Security violation
- ECP error
- Duplicate WWN
- E_Port isolated

©2010 Brocade Communications 37


BCFP in a Nutshell 2010 Edition

Taking the Test


After the Introduction Screen, once you click on Next, you will see the non-disclosure agreement:

Figure 12: Sample NDA

38 ©2010 Brocade Communications