TECHNICAL SUPPORT BULLETIN

February 18, 2010

TSB 2010-074-A

SEVERITY: High Operational

P RO D U CT S AF FE CTE D :
The FR4-18i blade, Brocade7500 using Fibre Channel Routing (FCR) EX_Ports, and any platform that
is configured to use integrated routing, such as DCX, DCX-4S, 5300 and 5100 are affected.
CORRECTED IN RELEASE:
The fix for this issue is part of an architectural change that was included in Brocade Fabric OS version
v6.3.0 and above.

BULLETIN OVERVIEW
Fabrics that are configured with EX_Ports in a multi-switch backbone fabric utilizing edge-to-edge
routing could experience a traffic impact due to a loss of credit across the EX_Port link.

Brocade produces and publishes Technical Support Bulletins to OEMs, partners and customers that
have a direct, entitled, support relationship in place with Brocade.
Please contact your primary service provider for further information regarding this topic and
applicability for your environment

PROBLEM STATEMENT
Frame corruption resulting from faulty media, optics or other equipment that occurs within a backbone
fabric used for FC Routing can cause a loss of credit to take place at an egress EX_Port.
The trigger for this issue is a marginal cable/SFP/patch panel that is introducing CRC errors and/or
Invalid Transmission Words (ITWs) or enc_in errors. Depending on which specific bytes within a frame
are corrupted, the frames are normally dropped either by the fabric, or by the destination device.
If the cause of the frame corruption is not addressed, and allowed to persist over a long period of
time, the probability of corrupting the DID field of the routed frame increases.

2010 Brocade Communications Systems, Inc. All Rights Reserved. TSB 2010-074-A

Prior to FOS v6.3.0, EX_Port links used a simple port-area mapping to virtual channel (VC) scheme. If
the DID field has been corrupted, it is possible for the credit to be returned to the wrong VC on the
EX_Port.
Eventually, if the corruptions are allowed to continue, it is possible that all credits from a single VC will
be returned to a different VC leaving one of the data VC paths with zero credits.
Once all TX credits have been lost, frames destined for an edge fabric via the EX_Ports impacted VC
will be dropped due to the lack of available credits. The frames will be dropped when the hold timer
expires and will show up as C3 Discards due to timeout.

RISK ASSESSMENT
This issue can only occur for frames that are traversing an edge to backbone to edge route and that
cross an ISL within the backbone fabric. Backbone to edge routing configurations are not susceptible
to this issue.
Customers that do not attend to a backbone ISL link that is corrupting frames will be at risk to this
issue. Statistically, given enough time, the corruptions will eventually affect the DID of the transmitting
frame which can result in TX credit loss.
Customers that immediately block or prevent ISLs that are corrupting frames to continue transmitting
traffic will not be at risk for this issue.

SYMPTOMS
Traffic between a host and target across an EX_Port link will stop. Connectivity will be lost due to
Class-3 Discards.
Symptom 1: CRC and/or Invalid word error counts will be incrementing at one of the ISL ports within
the core backbone fabric.
Symptom 2: Class-3 Discards will appear on the switch within the backbone fabric that is attempting
to transmit frames out to a destination edge fabric via an EX_Port.

WORKAROUND
Bouncing the affected EX_Port by executing portDisable followed by portEnable will restore the lost TX
credit.
The backbone ISL that is corrupting frames should also be disabled and/or repaired, or the risk of
losing credits will remain.

CORRECTIVE ACTION
Protecting the EX_Port from loss of Credit:
Customers can upgrade to FOS v6.3.0 or higher. Starting with firmware version FOS v6.3.0, EX_Ports
use a more advanced VC credit scheme that is already employed by standard ISLs. Using this
improved credit scheme prevents corrupted frames, elsewhere in the fabric, from impacting buffer
availability.
2010 Brocade Communications Systems, Inc. All Rights Reserved. TSB 2010-074-A

2 of 3

Note: EX_Ports must be bounced (disabled and enabled) in order to activate the new VC credit
scheme after a non-disruptive upgrade to FOS v6.3.0 or higher firmware.
Protecting the fabric from corrupted frames:
Customers can utilize port fencing to shut down any port that is causing frame corruption. Utilizing
port fencing to prevent the long-term corruption of frames will also protect the EX_Port from credit
loss.
Customers running with FOS v6.1.2x or FOS v6.2.x can use the following recommended threshold
values for Invalid Words and Invalid CRC counts:
Invalid Words
Invalid CRCs

Fencing Threshold
40/min
40/min

Customers running FOS v6.3.0 and higher can use the following recommended threshold values for
Invalid Words and Invalid CRC counts:
Invalid Words
Invalid CRCs

Low Threshold
25/min
2/min

High Threshold
40/min
40/min

Set the low threshold with an action of ALERT (RASLOG, email, SNMP Trap). The alert will be
triggered whenever the low threshold is exceeded.
Set the high threshold with an action of FENCE. The port will be fenced (disabled) whenever the high
threshold is detected.
These values can be adjusted up or down based on the quality of the fabric that has been configured.
A fabric that has a high amount of noise from older cables and/or SFPs or that is configured to use
longer cable runs through multiple patch panels may wish to use higher values for port fencing:
Invalid Words
Invalid CRCs

Low Threshold
25-40/min
25-40/min

High Threshold
80-100/min
80-100/min

Customers that have very clean fabrics can select to use more aggressive port fencing values, using
only the high threshold value to fence at the first detection of trouble, going as low as the provided
low threshold values provided above:
Invalid Words
Invalid CRCs

High Threshold
25/min
2/min

2010 Brocade Communications Systems, Inc. All Rights Reserved. TSB 2010-074-A

3 of 3