Beruflich Dokumente
Kultur Dokumente
Brocade, the Brocade B-weave logo, Fabric OS, File Lifecycle Manager, MyView, Secure Fabric OS,
SilkWorm, and StorageX are registered trademarks and the Brocade B-wing symbol and Tapestry are
trademarks of Brocade Communications Systems, Inc., in the United States and/or in other countries.
FICON is a registered trademark of IBM Corporation in the U.S. and other countries. All other brands,
products, or service names are or may be trademarks or service marks of, and are used to identify,
products or services of their respective owners.
Notice: This document is for informational purposes only and does not set forth any warranty,
expressed or implied, concerning any equipment, equipment feature, or service offered or to be offered
by Brocade. Brocade reserves the right to make changes to this document at any time, without notice,
and assumes no responsibility for its use. This informational document describes features that may not
be currently available. Contact a Brocade sales office for information on feature and product availability.
Export of technical data contained in this document may require an export license from the United
States government.
Objective: The BCFP Nutshell guide is designed to help you prepare for the BCFP Certification, exam number
143-070.
Audience: The BCFP Nutshell self-study guide is intended for those who have successfully completed the
CFP 380 Advanced Fibre Channel Administration and Theory course, and who wish to undertake self-study or
review activities before taking the actual BCFP exam. The BCFP guide is not intended as a substitute for
classroom training or hands-on time with Brocade products.
How to make the most of the BCFP guide: The BCFP guide summarizes the key topics on the BCFP exam for
you in an easy to use format. It is organized closely around the exam objectives. We suggest this guide be
used in conjunction with our free online knowledge assessment test. To benefit from the BCFP guide, we
strongly recommend you have successfully completed the CFP 380 Advanced Fibre Channel Administration
and Theory course.
We hope you find this useful in your journey towards BCFP Certification, and we welcome your feedback by
sending an email to jcannata@brocade.com.
Table of Contents
List of Figures
List of Tables
When a switch joins a fabric several Class F frames are used to exchange various parameters:
• ELP (Exchange Link Parameters)
- Contains sender information
- R_A_TOV / E_D_TOV
- PWWN / Switch Name
- Flow control used
• ESC (Exchange Switch Capabilities)
- Vendor specific info
- Virtual Fabric support
• EFP (Exchange Fabric Parameters)
- Principal switch selection
- Principal switch priority
- Switch name
- Domain ID list
- Vendor specific info
- Virtual Fabric support
- Zoning database
NPIV
N_Port ID Virtualization (NPIV) enables a single Fibre Channel protocol port to appear as multiple, distinct
ports, providing separate port identification within the fabric for each operating system image behind the port
(as if each operating system image had its own unique physical port). NPIV assigns a different virtual port ID
to each Fibre Channel protocol device. NPIV is designed to enable you to allocate virtual addresses without
affecting your existing hardware implementation. The virtual port has the same properties as an N_Port, and
is therefore capable of registering with all services of the fabric.
Each NPIV device has a unique device PID, Port WWN, and Node WWN, and should act the same as all other
physical devices in the fabric; in other words, multiple virtual devices emulated by NPIV appear no different
than regular devices connected to a non-NPIV port.
NPIV is enabled for every port on the Brocade 300, 4100, 4900, 5000, 5100, and 5300 switches, the
Brocade 5410, 5424, 5450, 5480 embedded switches, Brocade HBAs, the Brocade 48000 Director, the
Brocade DCX and DCX-4S enterprise-class platforms, and the FA4-18 blade.
Note: The FC10-6 port blade and the CEE ports on the Brocade 8000 do not support NPIV.
Interoperability
ICLs
An Inter-chassis link (ICL) is a licensed feature used to interconnect two or three Brocade DCX Backbones,
which can be a combination of either Brocade DCXs and/ora Brocade DCX-4S Backbones. ICL ports in the
core blades are used to interconnect two Brocade Backbones, potentially increasing the number of usable
ports in the Brocade DCX or DCX-4S chassis. The ICL ports on CR8 and CR4S-8 blades are internally managed
as E_Ports. These ports use proprietary connectors instead of traditional SFPs. When two Brocade Backbones
are interconnected by ICLs, each chassis requires a unique domain and is managed as a separate switch.
On the Brocade DCX there are two ICL connectors at ports ICL0 and ICL1 on each core blade, each
aggregating a set of 16 ports. Thus, each core blade provides 32 ICL ports and there are 64 ICL ports
available for the entire Brocade DCX chassis. All the ICL connector ports must be connected to the same two
Brocade DCX or DCX-4S chassis.
The Brocade DCX-4S has 2 ICL connector ports at ICL0 and ICL1, each aggregating a set of 8 ports. Thus,
each core blade provides 16 ICL ports and there are 32 ICL ports available for the entire Brocade DCX-4S
chassis. All the ICL connector ports must be connected to the same two Brocade DCX or DCX-4S chassis.
Security
Switch Connection Control (SCC) and Device Connection Control (DCC) Policies
E_Port Authentication
The authentication (AUTH) policy allows you to configure DH-CHAP authentication on switches with Fabric OS
v5.3.0 and later.
When you configure the switches at both ends of a link to use DH-CHAP for authentication, you must also
define a secret key pair—one for each end of the link. Use the secauthsecret command to perform the
following tasks:
- View the WWN of switches with a secret key pair
- Set the secret key pair for switches
- Remove the secret key pair for one or more switches
Policy Distribution
• Each switch can be set to Accept or Reject individual security policies
• Use the fddcfg command to show or set each policy for Accept or Reject
• All policies can be manually distributed to fabric switches
• The IPFILTER, PWD and AUTH policies can only be manually distributed
The Fabric Wide Consistency Policy can be configured to automatically distribute SCC, DCC, and FCS policies
automatically. Fabric-Wide Consistency has three states:
• Absent (not defined): Fabric-Wide Consistency is not defined (default); manual database distribution is
required using the distribute command
• Tolerant: Switches are not required to have the same ACL policy databases
• Strict: Switches in the fabric always have the same ACL policy databases
- If one switch in a fabric has a strict policy, all switches in the fabric must also have a strict policy
This example shows how to set a strict (S after policy name means strict) SCC and tolerant (no notation after
policy name means tolerant) DCC fabric-wide consistency policy:
switch:admin> fddcfg --fabwideset "SCC:S;DCC“
switch:admin> fddcfg --showall
Local Switch Configuration for all Databases:-
DATABASE - Accept/Reject
---------------------------------
SCC - accept
DCC - accept
PWD - accept
FCS - accept
AUTH - accept
IPFILTER - accept
Access Gateway
• Designed to connect numerous servers with minimal impact to an existing fabric
• Focus is connectivity, bandwidth is shared
• Included in the base Fabric OS – no separate license is required
• Attached F_Port devices must be Fibre Channel Protocol (FCP) initiators or targets
- Not supported: loop devices, FICON, virtual iSCSI initiators
• Access Gateway ports can be configured as N_Ports which connect to the edge fabric
• Hosts/HBAs are mapped (through NPIV) to the N_Ports, and connect to the fabric through the N_Ports
AG Port Grouping
• Port Grouping limits N_Port failover to occur within a user-defined group of ports
• Allows Access Gateway to be attached to multiple edge fabrics
• Default port group is enabled by default and requires all N_Ports to be connected to the same fabric
• User-defined port groups must be created to attach to more than one fabric
AG F_Port Trunking
• Trunking aggregates the bandwidth of the ports within the trunk group
• Available in Fabric OS v6.1.0 or later
• Needs to be configured as trunks do not automatically form
• Has the same requirements as ISL Trunking
- Trunking license on both AG and edge fabric switch
- Port group to port group
- Same speed
• Trunks use a shared port area ID/index called a Trunk Area (TA)
• Ports to be used for the F_Port Trunk are configured on the edge switch
• When configuring a TA, an index from one of the trunk port members is used for the TA
• Remaining F_Port Trunk members share the same TA and Port ID
• The porttrunkarea command is used on the edge switch to configure F_Port trunking
The Auto Port Configuration feature brings plug-and-play functionality to the Access Gateway. With Auto Port
Configuration enabled, there are changes to Access Gateway behavior:
1. No pre-determined port role – Based on the attached device or switch, each switch port automatically
configures as F_Port or N_Port
2. No pre-determined Port Map – F_Ports do not have a Primary N_Port and are evenly mapped (round-robin)
across all available N_Ports
3. No fixed port map – An F_Port is not guaranteed the same N_Port as devices/switches are added/
removed. When F_Ports and N_Ports are added or removed, the port map is automatically updated to
ensure an even distribution (round-robin) of F_Ports across N_Ports
Virtual Fabrics
Virtual Fabrics is an architecture to virtualize hardware boundaries. Traditionally, SAN design and
management is done at the granularity of a physical switch. The Virtual Fabrics feature allows SAN design and
management to be done at the granularity of a port.
Enabling Virtual Fabrics is disruptive. A reboot of the chassis is required once it is enabled. Virtual Fabrics is
supported on the following platforms:
• Brocade 5100
• Brocade 5300
• Brocade DCX
• Brocade DCX-4S
ISL Types
• ISL: Inter Switch Link - Standard ISL used to connect two non-virtual switches in the same fabric
- Can also be used to connect a virtual switch to a non-virtual switch in the same fabric
• IFL: Inter Fabric Link - IFL links are configured/enabled between an edge fabrics E_Port and an FC Router
EX_Port
• DISL: Dedicated Inter Switch Link - Used to directly connect two (non base) logical switches in two different
chassis
• XISL: Extended Inter Switch Link - Used to directly connect two base switches in two different chassis
• LISL: Logical Inter Switch Link - Used to connect two logical (non-base) switches in two different chassis via
the XISL on the base switch
This is the default mode for all the logical switches created in the Brocade DCX and DCX-4S enterprise-class
platforms. This addressing scheme is flexible to support a large number of F_Ports. In the regular 10-bit
addressing mode, the portaddress --auto command supports addresses from 0x00 to 0x8F.
The 10-bit addressing mode utilizes the 8 bit area_ID and the borrowed upper 2 bits from the ALPA portion of
the PID. Areas 0x00 through 0x8F use only 8 bits for the port address and support up to 256 NPIV devices.
This means a logical switch can support up to 144 ports that can each support 256 devices. Areas 0x90
through 0xFF use additional 2 bits from ALPA for the port address. Hence these ports support only 64 NPIV
devices per port.
CEE is an umbrella term for an Ethernet technology that has been enhanced by additional standards to meet
the requirements for transporting fibre channel frames. Some of these requirements are:
• A lossless, full-duplex Ethernet environment which provides in-order delivery and supports a minimum of
2.5 KB mini-jumbo frames
• Coexistence with classic Fibre Channel protocols
• High speed transport of Ethernet traffic
FCoE
FCoE provides a method of encapsulating Fibre Channel (FC) traffic into an Ethernet frame and is sent from
one FCoE-aware device, between CEE ports, to a second FCoE-aware device.
2 - SAN Management
Audit Log
When managing SANs you may want to filter or audit certain classes of events to ensure that you can view and
generate an audit log for what is happening on a switch, particularly for security-related event changes. These
events include login failures, zone configuration changes, firmware downloads, and other configuration
changes—in other words—critical changes that have a serious effect on the operation and security of the
switch.
Before you configure audit event logging, familiarize yourself with the following audit event log behaviors and
limitations:
• By default, all event classes are configured for audit; to create an audit event log for specific events, you
must explicitly set a filter with the class operand and then enable it
• Audited events are generated specific to a switch and have no negative impact on performance
• The last 256 events are persistently stored on the switch and are streamed to a system message log
• The audit log depends on the system message log facility and IP network to send messages from the
switch to a remote host. Because the audit event log configuration has no control over these facilities,
audit events can be lost if the system message log and IP network facilities fail
DCFM
The Management application provides easy, centralized management of the SAN, as well as quick access to
all product configuration applications. Using this application, you can configure, manage, and monitor your
networks with ease.
Fabric Binding
• The fabric binding feature enables you to configure whether switches can merge with a selected fabric.
This provides security from accidental fabric merges and potential fabric disruption when fabrics become
segmented because they cannot merge
• For M-EOS devices, enabling Fabric Binding activates Fabric Binding and enables insistent domain ID.
Disabling Fabric Binding on M-EOS devices deactivates Fabric Binding
• For Fabric OS devices, enabling Fabric Binding activates the Switch Connection Control (SCC) policy and
sets Fabric Wide Consistency Policy (FWCP) to strict and enables insistent domain ID. Disabling Fabric
Binding on Fabric OS devices deletes the SCC policy and sets FWCP to absent
The DCFM High Integrity Fabric (HIF) mode option automatically enables features and operating parameters
that are necessary in multiswitch Enterprise Fabric environments. When HIF is enabled, each switch in the
fabric automatically enforces a number of security-related features including Fabric Binding, Switch Binding,
Insistent Domain IDs, and Domain Register for State Change Notifications (RSCNs).
For Pure Fabric OS fabrics, HIF activates the Switch Connection Control (SCC) policy, sets Insistent Domain ID,
and sets the Fabric Wide Consistency Policy (FWCP) for SCC to strict mode.
HCM
The Host Connectivity Manager (HCM) is a management software application for configuring, monitoring, and
troubleshooting Brocade HBAs and Converged Network Adapters (CNAs) in a Storage Area Network (SAN)
environment.
Common HBA and CNA management software features include the following:
• Discovery using the agent software running on the servers attached to the SAN, which enables you to
contact the devices in your SAN.
• Configuration management, which enables you to configure local and remote systems. With HCM you can
configure the following items:
- Local host
- Brocade 4 Gbps and 8 Gbps HBAs
- HBA ports (including logical ports, base ports, remote ports, and virtual ports)
- Brocade 10 Gbps single-port and 10 Gbps dual-port converged network adapters (CNAs)
- CEE ports
- FCoE ports (CNA only)
- Ethernet ports (CNA only)
• Diagnostics, which enables you to test the adapters and the devices to which they are connected:
- Link status of each adapter and its attached devices
- Loopback test, which is external to the adapter, to evaluate the ports (transmit and receive trans-
ceivers) and the error rate on the adapter
- Read/write buffer test, which tests the link between the adapter and its devices
- FC protocol tests, including echo, ping, and traceroute
- Monitoring, which provides statistics for the SAN components
- Security, which enables you to specify a CHAP secret and configure authentication parameters
- Event notifications, which provide asynchronous notification of various conditions and problems
through a user-defined event filter
The Host Connectivity Manager (HCM) Port Statistics window enables you to monitor the performance of the
CNA and the traffic between the CNA and the LUNs. You can use the information to isolate and troubleshoot
areas that impact application performance.
SAN Health
• Automates documenting of a SAN and comes in different formats
• SAN Health is a free utility that aids in creating:
- Comprehensive documentation
- Historical performance graphs
- Detailed topology diagrams
SNMP
• SNMP is a standard method for monitoring and managing network devices (both Ethernet and Fibre
Channel)
• SNMP has the following components:
- SNMP Entities: Network Management Stations and Agents
- Management Information Bases (MIBs) - The agent accesses MIB information about the switch and
makes it available to a network management station
• Every Brocade switch runs an SNMP agent and Management Information Base (MIB)
Fabric Watch
• Optionally licensed per switch, it monitors the performance and status of the switch, including:
- Fabric events – Fabric reconfigs, zone changes, and new logins
- Switch status – Environmental (fans, power supplies, and temperature), SFP (Tx/Rx power, current,
& voltage), Security, resource and FRU
- Port status – Monitors F/FL/E_Port signal quality parameters
- Performance options – monitor end-to-end performance
• Fabric Watch maintains a set of counters for each of the monitored conditions
- Tracks the number of occurrences of each condition
- Each counter is compared with an upper boundary and lower boundary
Virtual Fabric Support for Fabric Watch - Fabric Watch can monitor the switch health on eight logical switches.
You can configure thresholds and alarms for ports that belong to a particular logical switch. Each logical
switch has its own Fabric Watch configuration and triggers alarms based on its local configuration.
Port Fencing
A port that is consistently unstable can harm the responsiveness and stability of the entire fabric and
diminish the ability of the management platform to control and monitor the switches within the fabric. Port
Fencing is a Fabric Watch enhancement that takes the Port class, E_Port class, and F/FL_Port class ports
offline if the user-defined thresholds are exceeded.
When a port that has exceeded its user-defined thresholds is fenced by software, the port is placed into the
disabled state and held offline, thereby removing the port’s ability to transmit or receive frames. After a port is
disabled, user intervention is necessary for frame traffic to resume on the port.
3 - Adaptive Networking
.
• Traffic that requires lower latency is placed in front of the queue
Bottleneck Detection
• The bottleneck detection feature identifies devices attached to the fabric that are slowing down traffic
(slow drain device)
• A slow drain device is slow to process received frames and send back credit returns
• Actual throughput into the slow drain port is lower compared to intended throughput
• A slow destination returns credits slower than the sender wants to send frames to it
• Slow drain can exist at any link utilization level from 0% to under 100%; not just high utilization scenarios
• Slow drain spreads into the fabric and can slow down unrelated flows in the fabric
• Commands to manage the bottleneck detection feature:
bottleneckmon --enable
bottleneckmon --show
• Does not require a license
• Is supported on Condor2/GoldenEye2 ASIC Fibre Channel ports only
• F_ and FL_Ports only
• F_Port trunks are not supported
• Bottleneck detection is disabled by default, and must be explicitly enabled for each port that is to be
monitored
Top Talkers
• Top Talkers (TT) is an enhancement to Advanced Performance Monitor (APM) end-to-end monitors
• When enabled, these monitors determine which SID-DID pairs are the major users of switch F_Port
bandwidth
• Can be enabled on specific switch E_Ports or F_Ports in the fabric
- Port Mode: enabled on an F_Port to measure the traffic between the F_Port and all other devices
that it can communicate with
- Fabric Mode: enabled on all E_Ports in the fabric to measure the data rate of all flows in the fabric
(ingress E_Port traffic only)
• Can be configured in either Port Mode or Fabric Mode, but not both simultaneously
• Determines the flows (SID/DID pairs) that are the major users of bandwidth
• Measures bandwidth usage data in real-time and relative to the port on which the monitor is installed
• Requires APM license
• Fabric Mode Top Talker monitors and end-to-end monitors cannot both exist in the fabric
• Data collected is deleted when the switch is rebooted
End-to-end monitors provide counter statistics for traffic flowing between a given SID-DID pair. Top Talker
monitors identify all possible SID-DID flow combinations that are possible on a given port and provides a
sorted output of the top talking flows. Also, if the number of flows exceeds the hardware resources, existing
end-to-end monitors fail to get real time data for all of them; however, Top Talker monitors can monitor all
flows for a given E_Port or F_Port (up to 10,000 flows).
4 - FC - FC Routing
• Fabric OS provides L3 Fibre Channel-to-Fibre Channel routing (FC-FC Routing) between fabrics
• Allows device access between two or more fabrics without merging the fabrics
- LSAN zoning must be enabled in all fabrics that share devices
o Edge-to-edge routing: edge fabrics
o Backbone-to-edge routing: backbone and edge fabrics
• FC-FC routing is supported between the following fabric types:
- Fabric OS-to-Fabric OS
- Fabric OS-to-M-EOS
- M-EOS-to-M-EOS
• M-EOS edge fabric support:
- M-EOS v07.00 and higher
- No VEX_Ports
• Only Fabric OS switches are allowed in the backbone fabric
- Backbone fabric must be set for Brocade Native Mode (Interopmode 0)
• In configurations with two backbones connected to the same edge fabric, routing is not supported between
edge fabrics that are not directly attached to the same backbone. Routing over multiple backbones is a
multi-hop topology and is not allowed.
• Every edge and backbone fabric requires a unique Fabric ID (FID)
• Enabling FC Routing services is done by using the fosconfig command
Integrated Routing
• An Integrated Routing license is required to allow configuration of FC-FC routing capable ports called
EX_Ports
- EX_Port: A type of E_Port used to connect an FC router port to an edge fabric without merging the
two
o EX_Ports on a router connect to E_Ports in an edge fabric
• License enforcement is checked on configuration and when enabling the EX_Port
- If the license is removed while EX_Ports are online, Condor2/GoldenEye2 EX_Ports will continue to
function until the next fabric rebuild, switch disable or port offline event
• Integrated Routing is currently supported on:
- Brocade FC8, FX8 and FS8 blades in a DCX and DCX-4S chassis
- Brocade 5100, 5300, 7800 and Brocade Encryption Switch (BES)
LSAN Zones
• Zones that define which devices are to be shared between fabrics
• Defined in each fabric that will share devices (edge or backbone)
• Is a traditional zone created using normal zoning tools but uses a special naming prefix “lsan”
Once the LSAN zones are active, determine which devices actually exist in the fabric, and which ones are
imported. Executed from the FC router in the backbone:
BB_B51:admin> lsanzoneshow -s
Fabric ID: 10 Zone Name: LSAN_Backbone1_Edge1
10:00:00:05:1e:57:7c:79 EXIST
22:00:00:20:37:dd:d9:29 Imported
Fabric ID: 100 Zone Name: lsan_backbone1_edge1
10:00:00:05:1e:57:7c:79 Imported
22:00:00:20:37:dd:d9:29 EXIST
• Configured - Device is configured to be in an LSAN, but the device is not imported nor does it exist in
this fabric
• EXIST - Device exists in this fabric (the fabric of the zone entry)
• Initializing -Device is in an intermediate state. It is not yet imported into the fabric
• Imported - Device has been imported (proxy created) into this fabric
Proxy devices
A FC router achieves interfabric device connectivity by creating proxy devices (hosts and targets) in attached
fabrics that represent real devices in the originating fabrics. For example, a host in Fabric 1 can communicate
with a target in Fabric 2 as follows:
• A proxy target in Fabric 1 represents the real target in Fabric 2
• Likewise, a proxy host in Fabric 2 represents the real host in Fabric 1
The host discovers and sends Fibre Channel frames to the proxy target. The FC router receives these frames,
translates them appropriately, then delivers them to the destination fabric for delivery to the target.
The target responds by sending frames to the proxy host. Hosts and targets are exported from the edge SAN
to which they are attached and, correspondingly, imported into the edge SAN reached through Fibre Channel
routing.
Phantom domains
A phantom domain is a domain emulated by the Fibre Channel router. The FC router can emulate two types of
phantom domains: front phantom domains and translate phantom domains. A front phantom domain is a
domain that is projected from the FC router to the edge fabric. There is one front phantom domain from each
FC router to an edge fabric, regardless of the number of EX_Ports connected from that router to the edge
fabric. Another FC router connected to the same edge fabric projects a different front phantom domain.
The second level of phantom domains is known as a translate phantom domain, also referred to as translate
domain or xlate domain. The translate phantom domain is a router virtual domain that represents an entire
fabric. Device connectivity can be achieved from one fabric to another—over the backbone or edge fabric
through this virtual domain—without merging the two fabrics. The EX_Ports present translate phantom
domains in edge fabrics as being topologically behind the front domains; if the translate phantom domain is in
a backbone fabric, then it is topologically present behind the FC router because there is no front domain in a
backbone fabric.
If a FC router is attached to an edge fabric using an EX_Port, it creates xlate domains in the fabric
corresponding to the imported edge fabrics with active LSANs defined. If you import devices into the
backbone fabric, then an xlate domain is created in the backbone device in addition to the one in the edge
fabric.
BB_B51:admin> fcrphydevshow
Device WWN Physical
Exists PID
in Fabric
-----------------------------------------
10 10:00:00:05:1e:57:7c:79 030100
100 22:00:00:20:37:dd:d9:29 6206e4
Total devices displayed: 2
In the command output we see the two physical devices that are currently being shared across the backbone
fabric:
- Edge fabric (FID 10) has one physical device PID 030100
- Backbone fabric (FID 100) has one physical device PID 6206e4
In the command output we see the two proxy devices that are currently being shared across the backbone
fabric:
• The format of the proxy address is XXfYYY, with XX indicating the translate domain ID, and YYY is a value
beginning at 001
FCIP
Fibre Channel over IP (FCIP) enables you to use existing IP wide area network (WAN) infrastructure to connect
Fibre Channel SANs. The Fibre Channel fabric and all Fibre Channel targets and initiators are unaware of the
presence of the IP network.
Port Types
• An FCIP tunnel is represented in a Brocade fabric as a Virtual E_Port (VE_Port)
- Just like an E_Port, except underlying transport is IP not FC
• The VE_Port emulates an E_Port on either end of the FCIP tunnel:
- The FCIP platforms at both ends of the links merge to form a single fabric
- VE_Ports do not use FC flow control mechanisms (BB Credits); they utilize TCP flow control mecha-
nisms
- VE_Ports do not support FC ISL trunking, but they do support exchange-based routing (Dynamic
Path Selection)
• Note that with FCIP Trunking, it is recommended to implement a multiple circuit trunk instead of having
multiple VE ports to the same fabric
• The Brocade 7800 supports FC-FC routing over an FCIP tunnel, creating a Virtual EX_Port (VEX_Port)
- Allows long-distance FCIP connections with fabric-to-fabric isolation
- VEX_Ports are no different from EX_Ports, except underlying transport is IP rather than FC
• There are a few connectivity rules with VEX_Ports:
- A VEX_Port connects only to a VE_Port – it may not connect to another VEX_Port
- There can be multiple VEX-to-VE port connections between a Backbone fabric and an Edge fabric
- EX-to-E and VEX-to-VE connections to the same Edge fabric can co-exist in Fabric OS v5.2 and later
fabrics
FCIP Licenses
Hardware
Brocade 7800
In a standard configuration, users have the option of using either the top two GbE ports, which are configured
for copper, or the bottom two, left-most, GbE ports (ge0 and ge1) which are configured for optical SFPs. The
remaining four GbE ports use optical SFPs. It is possible to configure ge0 as copper and ge1 as optical and
vice versa.
FX8-24
Figure 8: FX8-24
• The Brocade FX8-24 Extension Blade has:
- 2 x 10 GbE ports (license required)
- 10 x 1 GbE ports
- 12 x 8 Gbps FC ports
• One Condor2 ASIC
• Max of 2 blades supported in both the DCX and DCX-4S as of Fabric OS v6.3.0
• One Condor2 ASIC for FC ports
• Two FCIP subsystems
• 3 FC trunk groups:
- Ports 0 - 1
- Ports 6 - 7
- Ports 2 - 5 and 8 - 11
• Supported port types as of Fabric OS v6.3.0 are:
- Fibre Channel - F, FL, E, EX and Mirror
- GbE - VE
Circuit Metric
• Each circuit will be configured with a metric of 0 (active) or 1 (standby)
• The metric will be used by the tunnel supervisor to determine which circuit or circuits will be used as active
circuits
• Metric 0 circuits have the lowest metric and will be designated the active circuits and will be used for all
data transfers
• Metric 1 circuits are classified as standby circuits. It is in standby mode in the event that all metric 0
circuits fail
Provides for appropriate bandwidth usage when multiple FC QoS priorities are running over the same FCIP
circuit. It allows QoS traffic to be distributed as follows:
• F-Class: Gets what it needs. This is the highest priority
• QoS High: Gets at least 50% of the bandwidth
• QoS Medium: Gets at least 30% of the bandwidth
• QoS Low: Gets at least 20% of the bandwidth
• Extends QoS zoning that exists in fabric
• Identifies incoming virtual channels on a FC ISL
• QoS is not enforced unless there is congestion
• Lower priority flows are never limited if higher priority flows are not currently utilizing the BW
• Different TCP sessions can be treated independently
Fabric OS versions 6.0.0 and later provide for Fibre Channel QoS through internal QoS priorities. Those
priorities can be mapped to TCP/IP network priorities. There are two options for TCP/IP network-based QoS:
• Layer three DiffServ code Points (DSCP)
• VLAN tagging and Layer two class of service (L2CoS)
DSCP quality of service - Layer three class of service DiffServ Code Points (DSCP) refers to a specific
implementation for establishing QoS policies as defined by RFC2475. DSCP uses six bits of the Type of
Service (TOS) field in the IP header to establish up to 64 different values to associate with data traffic priority.
DSCP settings are useful only if IP routers are configured to enforce QoS policies uniformly within the network.
IP routers use the DSCP value as an index into a Per Hop Behavior (PHB) table. Control connections and data
connections may be configured with different DSCP values. Before configuring DSCP settings, determine if the
IP network you are using implements PHB, and consult with the network administrator to determine the
appropriate DSCP values.
L2CoS quality of service - Devices in physical LANs are constrained by LAN boundaries. They are usually in
close proximity to each other, and share the same broadcast and multicast domains. Physical LANs often
contain devices and applications that have no logical relationship. Also, when logically related devices and
applications reside in separate LAN domains, they must be routed from one domain to the other. A VLAN is a
virtual LAN network. A VLAN may reside within a single physical network, or it may span several physical
networks. Related devices and applications that are separated by physical LAN boundaries can reside in the
same VLAN. Also, a large physical network can be broken down into smaller VLANs. VLAN traffic is routed
using 802.1Q-compliant tags within an Ethernet frame. The tag includes a unique VLAN ID, and Class of
Service (CoS) priority bits. The CoS priority scheme (also called Layer two Class of Service or L2CoS), uses only
the upper three bits of the TOS field, allowing eight priorities.
Selective Acknowledgement
• Packet loss significantly degrades FCIP performance, lost data needs to be retransmitted
• Each lost packet requires a separate ACK response packet
• To mitigate this, Fabric OS supports Selective Acknowledgement (SACK) – (default ON)
• When SACK is enabled on a receiving VE/VEX_Port, the retransmission of multiple lost packets can be
combined in a single ACK packet
Long Distance
The connecting ports on the bookend switches must to be set to the same long distance parameters.Use the
portcfglongdistance CLI command to specify an Extended Fabric Distance Level:
• Level 0 static mode (L0) is the normal mode for a port
• Level E static mode (LE) reserves a static number of buffer credits that supports distances up to 10 km
- The number reserved depends on the port speed
• Dynamic long distance Mode (LD) calculates buffer credits based on the distance measured during port
initialization
- An upper limit is placed on the calculation by providing a desired distance value
• Static long distance mode (LS) calculates a static number of buffer credits based on a desired distance
value
Note: L0 and LE modes do not require a license. Use of LD and LS modes requires an Extended Fabric
License.
Buffer Credits
While exact calculations are possible, a simple rule of thumb is used in the calculation of the BB credit
requirement for a given link. The use of QoS causes more BB credits to be allocated to the port due to the
additional VCs (8 – 14).
Based on the speed of light in an optical cable being 5 ns/m, the rule of thumb is that a full-size FC frame
spans approximately:
• 4 km at 1 Gbps
• 2 km at 2 Gbps
• 1 km at 4 Gbps
• 500 m at 8 Gbps
• 400 m at 10 Gbps
For example, to keep a link that spans a distance of 10 km at full speed would require:
• 5 credits per port at 1 Gbps
• 10 credits per port at 2 Gbps
• 20 credits per port at 4 Gbps
• 40 credits per port at 8 Gbps
If your payload size is smaller than 2112 bytes, use the following formula to calculate exact number of
minimum buffer credit requirements:
buffer credits = [(distance in km) * (data rate in Gbps) * 1000] / (payload size)
Average frame size can be calculated using a Fibre Channel Analyzer or application-level data.
Protective Switching
Protection switching provides insurance against a fiber cut by optically splitting the data streams over two
diverse paths. Most WDM and SONET/SDH equipment has the ability to perform a protection switch from the
active path to the standby path in less than 50 ms in order to maintain connectivity between sites.
In the example below, assume LD mode has been set as the long distance setting, with 50 km as the desired
distance, and that data is using the 50 km side of the WDM ring. If the 50 km link fails, the WDM equipment
will switch to the 80 km side of the ring long before the loss-of-sync timer in the switch expires, resulting in no
notification of a change in path, or the need for more buffer credits. The switch is still supplying enough buffer
credits for 50 km, but our new path is 80 km, so the link is being starved. In this situation, it would be better
to specify LS long distance mode, and set the distance for the longer side of the WDM ring.
6 - Troubleshooting
Gathering Information
If problem escalation is required, send the escalation team all relevant supportsave files
• Baseline, prior to troubleshooting, after troubleshooting
• Maintain a baseline supportsave taken after every configuration change
Speed Negotiation
3. Enter the portcfgspeed command to change the port speed to 1, 2, 4 or 8 Gbps, depending on what
speed can be used by both devices. This should correct the negotiation by setting to one speed.
4. Enter the portlogshow or portlogdump command.
5. Check the events area of the output:
time task event port cmd args
-----------------------------------------------------------------
14:38:51.976 SPEE sn <Port#> NC 00000001,00000000,00000001
14:39:39.227 SPEE sn <Port#> NC 00000002,00000000,00000001
- In the event column sn indicates a speed negotiation
- In the cmd column NC indicates the negotiation has completed
Segmented Switches
When switches are segmented, you can use the switchshow, configshow, fabricshow, fabstatss-
how, portshow, and portcfgshow commands to help troubleshoot the segmentation. Also check zone
related commands and the license configuration. You can also use the DCFM Zone Merge Tool to compare
zones and then merge the zone configurations.
A switch is not allowed to merge with another switch that has an active effective configuration if the default
zone is set to “no access”. Before the switch can join, the default zone setting has to be set to "all access".
When the default zone no access option is enabled and the active configuration is disabled by using the
cfgdisable command, a special hidden configuration with no members is activated. This configuration will
not allow the switch to merge with switches that have an active effective configuration.
Enter the fcping command, which checks the zoning configuration for the two ports specified by:
• Generates an ELS (Extended Link Service frame) ECHO request to the source port specified and validates
the response
• Generates an ELS ECHO request to the destination port specified and validates the response
Regardless of the device’s zoning, the fcping command sends the ELS frame to the destination port. A
device can take any of the following actions:
• Send an ELS Accept to the ELS request
• Send an ELS Reject to the ELS request
• Ignore the ELS request
Fabric Changes
Changes to the fabric can be identified by using the fabriclog -s and fabstatsshow commands. The
fabriclog command with the -s flag displays the fabric log.
Use the fabstatsshow command to display the statistics for the fabric. The following information is
displayed:
• Number of times a switch domain ID has been forcibly changed
• Number of E_Port offline transitions
• Number of fabric reconfigurations
• Number of fabric segmentations as a result of any of the following causes:
- Loopback
- Incompatibility
- Overlap
- Zoning
- E_Port segment
- Licensing
- Disabled E_Port
- Platform DB
- Security incompatibility
- Security violation
- ECP error
- Duplicate WWN
- E_Port isolated