Brocade8000 - GA SG 161 01

FROM THE SOLUTIONS CENTER
Solution Guide:
Deploying the Brocade 8000 Switch
to Introduce Server I/O Consolidation
into the Data Center
Describes a reference architecture as a basis for planning and deploying a
converged network using the Brocade 8000 Switch, including several typical
use case scenarios.
FROM THE SOLUTIONS CENTER SOLUTION GUIDE
CONTENTS
Introduction........................................................................................................................................................................................................................................ 3
Overview of CEE and FCoE ............................................................................................................................................ 3
Brocade 8000 Switch.................................................................................................................................................... 4
Brocade FCOE10-24 Blade ........................................................................................................................................... 5
Management Tools ........................................................................................................................................................ 6
Planning and Deployment ........................................................................................................................................................................................................... 6
Network Topology .......................................................................................................................................................... 7
SAN Integration .............................................................................................................................................................. 8
CEE and LAN Integration .............................................................................................................................................10
Configuring CEE Attributes...................................................................................................................................12
Creating the CEE Map ..........................................................................................................................................12
Spanning Tree ......................................................................................................................................................13
VLAN Membership................................................................................................................................................13
Interface Configuration ........................................................................................................................................14
Connecting Servers to the Brocade 8000..................................................................................................................15
FC Configuration...................................................................................................................................................15
Ethernet Configuration.........................................................................................................................................15
FCoE Deployments and Use Cases.........................................................................................................................................................................................16
File Server ....................................................................................................................................................................16
LAN Configuration ................................................................................................................................................18
SAN Configuration ................................................................................................................................................19
Virtual Machine Consolidation and Migration ............................................................................................................19
Ethernet Connectivity ...........................................................................................................................................20
Storage Connectivity ............................................................................................................................................23
Performing the Migration .....................................................................................................................................24
Appendix A: Sample Configuration.........................................................................................................................................................................................25
Deploying the Brocade 8000 Switch to Introduce S into the Brocade Data Center 2 of 26
INTRODUCTION
This document provides an overview of Fibre Channel over Ethernet (FCoE) and Converged Enhanced
Ethernet (CEE) and describes a reference architecture that can be used as a basis for planning and
deploying a converged network using the Brocade® 8000 Switch or the Brocade FCOE10-24 Blade in the
Brocade DCX® or DCX-4S Backbone. The reference architecture describes a typical data center network and
details how to plan, configure, and integrate the Brocade 8000 into the existing Storage Area Network (SAN)
and Local Area Network (LAN) topology.
NOTE: Although this document primarily references the Brocade 8000 Switch, the Brocade FCOE10-24
blade (see Figure 4) has recently been released for the Brocade DCX/DCX-4S Backbones.
The following use cases scenarios for the newly installed converged network are described:
 Both LAN and SAN configuration for file servers
 Virtual Machine migration
Overview of CEE and FCoE

Data centers often deploy separate physical networks in order to transport different applications and traffic
types:
 Fibre Channel (FC) SANs are deployed to support storage traffic.
 Ethernet LANs are deployed to support client-server application traffic.
 Other networks are deployed to support high-speed IPC communication between servers.
As data centers grow in size so does the cost of managing multiple interconnect technologies, including
separate cabling, patch panels, and distribution frames. Managing interconnect technologies not only adds
cost but also impacts provisioning lead time and consequently the ability to adapt to changing business
requirements and opportunities. Consolidating different traffic types onto a converged network enables use
of a single physical infrastructure, but it also requires a new technology to consolidate multiple transport
layers without compromising the properties of each protocol.
This technology, CEE, is different from legacy Ethernet; it introduces low latency, lossless, and deterministic
capabilities. CEE enables a new protocol called FCoE, which allows organizations to extend the reliability
and network services of Fibre Channel to a broader range of server environments, reducing the cost and
complexity of implementing a virtualized data center. FCoE is a new standards-based protocol that provides
a method to encapsulate and transport FC frames over CEE.
FCoE uses the proven, standard Fibre Channel protocol and allows you to scale an existing FC network
infrastructure today and leverage FCoE and CEE technology in the future for interconnectivity—avoiding a
costly and disruptive rip-and-replace migration strategy. Combining CEE and FCoE enables consolidation of
Ethernet and Fibre Channel onto a single interconnect from the server to a Top-of-Rack (ToR) switch (shown
in Figure 1), which then directs Ethernet traffic to the LAN network and FC traffic to the SAN.
2 Brocade 8000 Switches

at the top of the rack LAN
SAN
Server
Server
SAN
Figure 1. ToR deployment model
Brocade 8000 Switch

The Brocade 8000 is a Layer 2, 32-port, 1U form factor ToR switch with:
 24 x low-latency 10 Gigabit Ethernet (GbE) CEE ports that support FCoE and provide line-rate, low-
latency, lossless, and deterministic interconnect based on the Brocade 10 Gigabits per second (Gbps)
FCoE cut-through switching technology
 8 x 8 Gbps FC, auto-sensing 1, 2, 4 or 8 Gbps ports that deliver the latest Brocade FC technology
In the rack, the Brocade 8000 connects to servers with Converged Network Adaptors (CNAs), such as the
Brocade 1010/1020 single- and dual-ports CNAs (to be released in summer 2009). FCoE traffic over CEE
links between a CNA and the Brocade 8000 enables storage connectivity, while IP traffic over the same CEE
links provides connectivity to the data center LAN.
In addition to FCoE capabilities, the Brocade 8000 also implements complete Fibre Channel Protocol (FCP)
and Fiber Channel Forwarding (FCF) capabilities and delivers Advanced Fabric Services found in other
Brocade FC switches.
Figure 2. Brocade 8000 port side with CEE (blue) and FC (red) ports
There are three main functional block modules in the Brocade 8000 Switch. The FC module is a fully
functional FC switch supporting the features and services that are typically supported by Brocade FC
switches. The Ethernet block of the Brocade 8000 is made up of a CEE switch supporting traditional 10 GbE
Ethernet and CEE. The third functional block is the FCoE entity or Fibre Channel Forwarder (FCF). The role of
the FCoE entity is to extract the FC payload form FCoE traffic and forward the FC frames to the FC switch on
their way to FC storage targets. It also encapsulates FC traffic heading to the CEE switch to be transported
back over CEE links.
FC FC FC FC 8 x 8 FC
port port port port FE ports
Fibre Channel switch
FCoE
Enhanced Ethernet switch

CEE CEE CEE CEE 24 x 10 GbE
port port port port FE ports
Figure 3. Brocade 8000 architecture
Brocade FCOE10-24 Blade

The Brocade FCOE10-24 blade, shown in Figure 4, enables Brocade DCX Backbones to provide end-of-row
server connectivity, using CEE/FCoE to consolidate LAN and SAN traffic. It provides:
 Delivers line-rate 10 Gbps performance across 24 x CEE ports
 Processes FCoE traffic at wire speed for transport to Fibre Channel SANs through Brocade DCX/DCX-4S
8 Gbps Fibre Channel port blades
 Provides rich Layer 2 Ethernet functionality for LAN traffic
Figure 4. The Brocade FCOE10-24 blade
For full product details of both the standalone switch and the blade, see their respective Data Sheets on
www.brocade.com.
Management Tools
Both the Brocade 8000 Switch and the Brocade DCX run traditional Fabric OS® (FOS) software and can be
managed using the same tools traditionally used for SAN management. Using the FOS Command-Line
Interface (CLI), administrators have access to all commands and utilities common to other Brocade
switches. In addition, FOS software enables Brocade Web Tools to support the following features for
configuring and managing a Converged Ethernet Network:
 CEE interface display and configuration
 FCoE trunk display and configuration
 CEE configuration including link aggregation (LACP), Virtual LANs (VLANs), Quality of Service (QoS), and
Link Layer Discovery Protocol (LLDP)/Data Center Bridging eXchange protocol (DCBX)
 FCoE login groups
The Brocade 8000 provides a new CLI designed to support the management of CEE and L2 Ethernet
switching functionality. The CEE CLI uses a common industry hierarchical shell familiar to Ethernet/IP
networking administrators. To access the CEE CLI, enter the cmsh command from a FOS command line.
See the Brocade Converged Enhanced Ethernet Administrator’s Guide for details.
PLANNING AND DEPLOYMENT

You can deploy CEE and FCoE while continuing to use and expand previous investments in SAN and LAN
infrastructure. New servers brought into the data center can be installed with Brocade CNAs to carry both
SAN and LAN traffic through a unified interface to the Brocade 8000 at the top of the rack. One of the
significant benefits of using the Brocade 8000 with Brocade CNAs is the reduction of server adaptor cards,
cabling, and switches at the top of the rack—connectivity to both SAN and LAN infrastructure is provided by
the Brocade 8000, as shown in Figure 5.
Figure 5. Brocade FCOE10-24 in a Brocade DCX chassis with Brocade CNAs
Consider the following questions before deploying a CEE network in the data center:
 How will the network topology change with the introduction of CEE? This includes both SAN and LAN
networks in the data center.
 What networking parameters must be configured in the CEE network to enable connectivity to existing
LAN and SAN infrastructure?
 What are the application requirements for servers in the CEE network? This includes defining traffic
types (FCoE, IP, and so on) and bandwidth requirements for each traffic type. Which traffic types will
need lossless behavior?
 How will servers and applications be configured to support a converged network? This may include
enabling support for different traffic types and enabling redundant connectivity to SAN and LAN
networks.
Network Topology
Deploying a converged network into an existing SAN/LAN infrastructure can be performed on an
incremental basis preserving previous investments in both SAN and LAN network infrastructure. During the
planning stage, administrators must determine where to place the new components and how to connect
them. The traditional data center has separate networks for storage and LAN communication. A common
network design solution has FC edge switches and Ethernet access layer switches installed at the top of the
rack. FC edge switches are connected to the SAN core network and Ethernet access layer switches are
connected into the distribution or core LAN infrastructure. It is common practice in SAN design to add
redundancy so that there is no single point of failure, a practice that is followed in this configuration.
Servers
Ethernet Fibre Channel

swtiches switches
LAN SAN A SAN B
Figure 6. Typical server connectivity to existing LAN networks and SAN fabrics
During hardware upgrades or new server installations, CNAs can be installed into server platforms and
connected to the Brocade 8000 to achieve network convergence at the edge. Ethernet switches and FC
switches are replaced with a single ToR platform enabling connectivity to core SAN and LAN networks.
Figure 7 shows a typical integration into existing network infrastructure.
NOTE: In this document, connected are color-coded as follows:
 Fibre Channel in red
 Ethernet in black
 FCoE/CEE in blue
Existing server edge connectivity New server edge
FC
Ethernet switches FCoE/CEE
swtiches swtiches
LAN SAN A SAN B
Figure 7. Network convergence at the edge
SAN Integration
FC SANs are typically deployed in a core-edge topology with servers connecting to edge switches in the
fabric. Since the Brocade 8000 FC switching module operates with the same features and functionality of a
regular FC switch, this topology is preserved when the Brocade 8000 switch is introduced into the fabric.
The Brocade 8000 can be treated as just another edge switch connecting to the core FC infrastructure. The
only difference is that servers are directly attached using a CNA supporting the FCoE protocol instead of an
HBA supporting the FC protocol.
For information about installing and setting up the Brocade 8000, see the Brocade 8000 QuickStart Guide
and the Brocade 8000 Hardware Reference Guide.
This being said, connecting the Brocade 8000 to an existing FC SAN follows the same process as adding a
new FC edge switch into a SAN. Most SAN environments include redundant fabrics (A and B). A typical
installation involves connecting a Brocade 8000 to Fabric A, verifying stability, and then installing a second
Brocade 8000 into Fabric B. The process of installing a new switch includes the following steps:
1. On the Brocade 8000, verify that the Zone database is empty and change the domain ID to a unique
number. If there are any non-default fabric configuration changes in the existing fabric, ensure that these
are also configured on the new switch.
2. Power off the Brocade 8000 and connect the Inter-Switch Links (ISLs) to the core FC switch or director.
Connecting a new switch to the fabric while it is powered off ensures that reconfiguration will not occur.
3. Once connected, power on the new switch and verify that the ISLs are online and the fabric is merged.
Check to make sure the fabrics existing Zone database was copied over to the Brocade 8000.
When a CNA logs in to the fabric, it appears in the name server and can be zoned via standard
management tools including the CLI or Web Tools. The FOS CLI command nsShow displays any FCoE or FC
devices connected to the switch.
B8000-112:admin> nsshow
{
Type Pid COS PortName NodeName
TTL(sec)
N 010804; 3;21:00:00:1b:32:00:17:0c;20:00:00:1b:32:00:17:0c; na
FC4s: FCP
NodeSymb: [33] "QLE8042 FW:v4.04.00 DVR:v9.1.7.18"
Fabric Port Name: 20:08:00:05:1e:54:19:ae

Permanent Port Name: 20:08:00:05:1e:54:19:ae
Port Index: 8
Share Area: No
Device Shared in Other AD: No
Redirect: No
FCoE devices log in to one of the 24 VF_Ports on the Brocade 8000 or FCOE24-10 blade. The VF_Ports
FCoE port services provide access to FCoE VN_Port devices similar to those provided by FC F_Port to FC
N_Port devices. In the example below, there is one VN_Port connected to port 8 and another to port 10.
B8000-112:admin> switchshow
switchName: B8000-112
switchType: 76.7
switchState: Online
switchMode: Native
switchRole: Subordinate
switchDomain: 2
switchId: fffc02
switchWwn: 10:00:00:05:1e:b0:1d:80
zoning: ON (cfgA)
switchBeacon: OFF
FC Router: OFF
FC Router BB Fabric ID: 1
Index Port Address Media Speed State Proto

==============================================
0 0 020000 id N8 In_Sync FC Disabled
1 1 020100 -- N8 No_Module FC
2 2 020200 -- N8 No_Module FC
3 3 020300 id N4 No_Light FC
4 4 020400 id N8 Online FC E-Port 10:00:00:05:1e:94:0c:00
"DCX4S_bottom" (upstream)(Trunk master)
5 5 020500 -- N8 No_Module FC
6 6 020600 -- N8 No_Module FC
7 7 020700 id N4 No_Light FC
8 8 020800 -- 10G Online FCoE VF-Port 1 VN-Port(s)
10 10 020a00 -- 10G Online FCoE VF-Port 1 VN-Port(s)
11 11 020b00 -- 10G Online FCoE VF-Port 0 VN-Port(s)
12 12 020c00 -- 10G Online FCoE VF-Port 0 VN-Port(s)
13 13 020d00 -- 10G Online FCoE VF-Port 0 VN-Port(s)
14 14 020e00 -- 10G Online FCoE VF-Port 0 VN-Port(s)
15 15 020f00 -- 10G Online FCoE VF-Port 0 VN-Port(s)

26 26 021a00 -- 10G Online FCoE VF-Port 0 VN-Port(s)
27 27 021b00 -- 10G Online FCoE VF-Port 0 VN-Port(s)
28 28 021c00 -- 10G Online FCoE VF-Port 0 VN-Port(s)
29 29 021d00 -- 10G Online FCoE VF-Port 0 VN-Port(s)
30 30 021e00 -- 10G Online FCoE VF-Port 0 VN-Port(s)
31 31 021f00 -- 10G Online FCoE VF-Port 0 VN-Port(s)
In order to determine information about FCoE devices, use the fcoe command from the FOS CLI. In this
example, there is a CNA logged into FCoE VF_Port 8 and is attached to the physical CEE port
TenGigabitEthernet 0/0. Another CNA is logged into VF_Port 10 and is attached to physical CEE port
TenGigabitEthernet 0/2.
B8000-112:root> fcoe --loginshow

================================================================================
Port Te port Device WWN Device MAC Session MAC
================================================================================
8 Te 0/0 10:00:00:05:1e:9a:a6:79 00:05:1e:9a:a6:79 0e:fc:00:02:08:01
10 Te 0/2 10:00:00:05:1e:9a:9f:e9 00:05:1e:9a:9f:e9 0e:fc:00:02:0a:01
CEE and LAN Integration

Because the Brocade 8000 is an IEEE 802.1Q-compliant switch, it easily integrates into the existing LAN
infrastructure in a variety of data center network topologies. In a typical installation, the Brocade 8000 acts
as an access layer switch connecting to a distribution or core layer switch in the LAN.
This section describes a representative data center LAN and the configuration process for introducing the
Brocade 8000 into the network, shown in Figure 8, as well as configuring feature sets unique to CEE.
Common configuration topics are discussed. Unless otherwise noted, all commands are entered through
the CEE CLI. See the Brocade FCoE Administrator’s Guide for configuration details and supported L2
functionality.
Data center
core layer CORE
Aggregation layer
VLAN
trunks
Data center Brocade Brocade . . . connected

FCoE
access layer 8000 8000 to SAN fabric
VLAN 100
Data center
servers
Server Group 1 Server Group 2

VLAN 10 VLAN 20
Figure 8. Adding the Brocade 8000 to the data center LAN (SAN not shown)
Configuring CEE Attributes

The following information is needed for CEE configuration:
 The types of traffic flowing through an interface, FCoE, TCP/IP, and so on
 The minimum bandwidth required for each traffic type
 Which traffic type needs lossless behavior
Brocade uses CEE Maps to simplify the configuration of QoS and flow control. Users assign different
priorities to different traffic types and enable lossless connectivity. A CEE Map configures two features:
Enhanced Transmission Selection (ETS) and Priority Flow Control (PFC).
Enhanced Transmission Selection (ETS) is used to allocate bandwidth based on the different priority
settings of the converged traffic. For example, users may want Inter-Process Communications (IPC) traffic to
use as much bandwidth as needed, while LAN and SAN traffic share a designated percentage of the
remaining bandwidth. ETS is used to manage the traffic priorities between traffic types by regulating flow
and by assigning preset amounts of link bandwidth and relative priority to each application.
802.1q-tagged Ethernet frames contain a Priority Code Point (PCP) field, which describes the 802.1p class
of service priority. This field indicates the a priority level that can be applied to different classes of traffic on
a CEE link from 0 to 7. For example, a server administrator may assign FCoE traffic priority 3. Priorities are
then grouped into Priority Groups (PGID), which are used by the switch to schedule frame forwarding.
The Brocade 8000 supports two types of scheduling: Strict Priority (SP) and Weighted Round Robin (WRR).
An SP scheduler drains all packets queued in the highest-priority queue before servicing lower-priority traffic
classes. Use PGID 15 for strict priority scheduling. Use WRR scheduling to facilitate controlled sharing of the
network bandwidth. WRR assigns each queue a weight, which is used to determine the frequency of frame
forwarded for the queue. The round robin aspect of the scheduling allows each queue to be serviced in a
set ordering, sending a limited amount of data before moving onto the next queue and cycling back to the
highest priority queue after the lowest priority is serviced. PGIDs 0 to 7 can be used for WRR scheduling.
Priority Based Flow Control (PFC) is an enhancement to the current link-level flow control mechanism
defined in IEEE 802.3X (PAUSE) so that it can operate individually on each priority. PFC is what enables
lossless connectivity and is required for FCoE traffic.
Creating the CEE Map

The first step is to define the types of traffic carried over the CEE network. As an example, servers in
Figure 8 use the CEE network for both FCoE and IP. The administrator decides to associate FCoE traffic with
priorities 2 and 3 and IP traffic with priorities 0, 1, and 4 – 7. All the priorities used for IP traffic are grouped
into a single Priority Group ID, PGID 1, and the priorities used for FCoE are grouped into PGID 2.
Bandwidth requirements for each PGID are then chosen. The administrator decides to give IP traffic 60% of
the schedule and FCoE traffic 40%. Finally, since FCoE traffic requires lossless communication, PFC is also
enabled for PGID 1.
To create the CEE map using the CLI, define the name of the map and then specify the traffic requirements
for each PGID using priority-group-table. The priority-table is then used to specify which priorities are
mapped to which PGID. The priorities are defined from lowest to highest.
cee-map <CEE Map Name>

priority-group-table <PGID> weight <BW%> [pfc]
priority-table <PGID> <PGID> <PGID> <PGID> <PGID> <PGID> <PGID> <PGID>
For the given example, a CEE map named “srvgroup” is created using the following syntax:
cee-map srvgroup
priority-group-table 1 weight 40 pfc
priority-group-table 2 weight 60
priority-table 2 2 2 1 1 2 2 2
Data Center Bridging eXchange Protocol (DCBX) runs on CEE links and is an extension of the Link Layer
Discovery Protocol (LLDP) protocol. The primary goal of DCBX is to allow the discovery of CEE-capable hosts
and switches and allow CEE-specific parameters—such as those for ETS and PFC—to be sent before the link
is shared. DCBX parameters use a type-length-value (TLV) format. By default, DCBX is turned on, but there
are two TLVs that must be enabled to support FCoE on a CEE link.
 dcbx-fcoe-app-tlv: IEEE Data Center Bridging eXchange FCoE Application TLV
 dcbx-fcoe-logical-link-tlv: IEEE Data Center Bridging eXchange FCoE Logical Link TLV. The presence of
this TLV declares that the FCoE part of the converged link is up.
Enter the following into the CEE CLI shell to configure LLDP:
protocol lldp
no disable
advertise dcbx-fcoe-app-tlv
advertise dcbx-fcoe-logical-link-tlv
Spanning Tree
Spanning Tree Protocol is a mechanism to detect and avoid loops in Ethernet networks by establishing a
fixed path between all the switches in a LAN. The Brocade 8000 supports three spanning tree variations:
 Standard Spanning Tree (STP)
 Rapid Spanning Tree (RSTP)
 Multiple Instance Spanning Tree (MSTP)
It is best practice that an access layer switch, such as Brocade 8000, does not become the root switch.
Changing the bridge or STP priority helps to ensure that this does not occur. The example below performed
from the CEE CLI configures the Brocade 8000 for RSTP and sets the bridge priority to the highest value,
thus ensuring it will not become the root switch in an existing LAN:
protocol spanning-tree rstp

bridge-priority 61440
VLAN Membership
IEEE 802.1q Virtual LANs (VLANs) provide the capability to overlay the physical network with multiple virtual
networks. VLANs allow network traffic isolation into separate virtual networks reducing the size of
administrative and broadcast domains.
A VLAN contains end stations that have a common set of requirements which can be in independent
physical locations. You can group end stations in a VLAN even if they are not physically located in the same
LAN segment. VLANs are typically associated with IP subnets and all the end stations in a particular IP
subnet belong to the same VLAN.
In the sample network shown in Figure 8, there are three VLANs, VLAN 100, 10 and 20 . VLAN 10 and 20
are used to isolate the L2 traffic from the two server groups. These VLANs carry IP traffic from the servers to
the data center LAN. Any routing between these VLANs is performed at the distribution layer of the network.
VLAN 100 is a special VLAN used for FCoE traffic between the servers and storage connected to the Fibre
Channel fabric and must be configured as an FCoE Forwarder (FCF). Only FCF-capable VLANs can carry
FCoE traffic.
Create the VLAN interfaces on the Brocade 8000 using the CEE CLI.
interface vlan 10
description server group 1
interface vlan 20
description server group 2
interface vlan 100
description FCoE VLAN
fcf forward
In addition to creating a special VLAN for FCoE traffic, VLAN classifiers are applied to incoming Ethertypes
for FCoE Initiation Protocol (FIP) and FCoE. VLAN classifiers are rules used to dynamically classify Ethernet
packets on an untagged interface to VLANs. The example below creates VLAN rules and a VLAN classifier
group for these two Ethertypes. The VLAN classifer group can then be applied to any CEE interface.
vlan classifier rule 1 proto fip encap ethv2

vlan classifier rule 2 proto fcoe encap ethv2
vlan classifier group 1 add rule 1
Interface Configuration
Traffic on CEE interfaces can be assigned to a VLAN using several methods: the VLAN tag contained in the
incoming frame, the VLAN classifiers, or the Port-VLAN ID (PVID). Because uplink ports from the Brocade
8000 to the distribution layer switches will carry traffic for multiple VLANs, they are configured as 802.1q
trunk ports. In this example, the CEE switch port was configured to carry VLANs 10 and 20. This step
assigns those VLANs to an uplink Ethernet port. This step is repeated for all uplink interfaces.
interface TenGigabitEthernet 0/1

switchport
switchport mode trunk
switchport trunk allowed vlan add 10
no shutdown
The downstream CEE ports connected to the server CNAs are typically configured for access or converged
mode. An access mode port only accepts Ethernet frames that are not VLAN tagged while a converged
mode port can accept both tagged and untagged frames. When receiving an untagged frame, the VLAN is
classified based on any protocol or MAC based classifiers installed on the port or the PVID. In the example
below, the switch port is defined as access mode with a PVID of 10 and is connected to a Qlogic QLE8042
CNA In addition, the VLAN classifier group created for the FIP and FCoE Ethertypes is applied to the
interfaces in order to place FCoE traffic on the correct VLAN. The CEE map is also applied to the interface.

switchport
switchport mode access
switchport access vlan 10
vlan classifier activate group 1 vlan 100
no shutdown
cee srvgroup
An example of an interface configured for converged mode is described below. In this case the native PVID
is 20. For most second generation CNAs, including the Brocade 1010 and 1020, Brocade recommends
using converged mode on CEE interfaces.

switchport
switchport mode converged
switchport converged vlan 20
no shutdown
cee map1
Connecting Servers to the Brocade 8000

Converged Network Adapters (CNAs) support FCoE and Ethernet LAN communication over the same cable
from the server to a CEE switch such as the Brocade 8000, as shown in Figure 9 (or the Brocade FCOE10-
24, shown in Figure 5). The CNA is presented to the host operating system as both an Ethernet NIC and a
Fibre Channel HBA so that network configuration and server management practices do not change.
SCSI
MPIO
TCP FC
IP FCoE
CNA
CEE Brocade
8000
Figure 9. CNA protocol stack
The CNA supports CEE features required to support lossless connectivity and QoS of different traffic types.
Although modification of parameters is possible with some CNAs, most adapters are set up in a “Willing”
mode, meaning that they automatically accept CEE configurations for QoS and PFC from the connected
switch using the DCBX protocol.
The Brocade 8000 supports most Converged Network Adapters on the market today including those from
QLogic and Emulex.
FC Configuration
The CNA discovers storage on the FC SAN and presents LUNs to the operating system in the same manner
as an HBA. The same multipathing software needed for high availability in a traditional SAN can be used in
a converged network.
Ethernet Configuration
Most CNAs support some type of Network Teaming or Link Aggregation protocol to allow the use of multiple
ports in parallel, to improve performance or create redundancy for higher availability. For highest availability
it is always recommended that you install two CNAs into a server and connect each to a different CEE
switch.
FCOE DEPLOYMENTS AND USE CASES

This section describes several deployment examples and use cases that demonstrate the features and
capabilities of network convergence with the Brocade 8000. All examples utilize the configuration described
in the above sections for the Brocade 8000.
File Server
A typical CIFS or NFS File Server receives requests from clients on the LAN and retrieves or writes data to
storage on a Fibre Channel SAN. In a traditional networking topology, the file server would have a Fibre
Channel HBA for connection to the SAN and an Ethernet Network Interface Card (NIC) for connection to the
data center LAN. Two of each would be required for high availability.
File server
Top-of-rack
Fibre Channel Top-of-rack
switches LAN switches
Clients on LAN
Fabric A Fabric B LAN
Figure 10. Typical configuration before network convergence
When network convergence is introduced at the network edge using the Brocade 8000, the number of
cables and interconnects in a rack are reduced by half. Figure 11 illustrates the components and data flow
from clients accessing file data through the IP network and Brocade 8000 to a CNA on a Windows CIFS file
server. The file system shared by the file server is located in the FC SAN and is again accessed through the
Brocade 8000 using the FCoE protocol.
Redundant CNAs provide
multipatihng to FC storage
and switch fault tolerance
to the Ethernet LAN over CEE
IP tr
affic
File server
c
f fi
t ra
e
ag
or
Brocade 8000
St
enables network
convergence
at the top
of the rack
Clients on LAN
Fabric A Fabric B LAN
Figure 11. Network convergence at the network edge using the Brocade 8000
LAN Configuration
Two CNAs are installed in the file server and are presented to the operating system as two physical NIC
interfaces that are teamed together for failover protection. Switch Fault Tolerance Teaming mode is used to
provide a fault-tolerant network connection in the event that the first adapter, its cabling, or the switch fails.
Each CNA is connected to a diferent Brocade 8000 providing redundant access into the LAN.
Figure 12. NIC teaming on the CNA
SAN Configuration
The two CNAs installed in the file server are also presented to the host operating system as two physical
FCoE adapters. Storage for the network share is located in the FC SAN and is multipathed to Fabrics A
and B. The port World Wide Name pWWN of the CNA in each fabric is then zoned to the storage to allow the
host access to the LUN. In the tested example, Symantec Volume Manager Dynamic Multi-Pathing (DMP) is
used to provide active/active access to the storage on the SAN through the Brocade 8000. In the event of
cable or component failure in one fabric, access to storage remains available.
Figure 13. Multipathed access to storage through two CNAs
Once networking configuration is complete and storage is provisioned, a file system can be created on the
LUN and its contents shared to clients in the LAN.
Virtual Machine Consolidation and Migration

Server virtualization is one of the drivers for greater I/O needs in servers. CNAs with 10 GbE CEE links
provide virtualized applications with the IP links they expect and allow data centers to deploy multiple
applications on individual servers. In addition to consolidating server I/O links over CEE ports, virtualization
gives data centers the opportunity to consolidate applications over a smaller number of servers while
benefiting from added application mobility. IT managers can also take advantage of the 10 GbE CEE links to
consolidate slower Ethernet and FC connections onto the much faster connections offered by the new
generations of CNAs. The combination of reducing server and I/O link counts results in noticeable savings
and greater efficiencies in data centers.
In this use case, a VMware ESX Server Virtual Machine (VM) is migrated from a server with traditional FC
HBAs and Ethernet NICs to a new server deployed on a converged network. Migration is performed without
downtime using VMware VMotion. VMotion is a feature of VMware ESX Server that allows the migration of a
powered-on VM from one physical server to another, while continuous service availability and transaction
integrity is maintained.
Migration with VMotion has three stages:
1. When migration with VMotion is requested, the ESX Server host verifies that the existing VM is in a stable
state with its current host.
2. VM state information (memory, registers, and network connections) is copied to the target host.
3. The VM resumes activities on the new host.
VMotion
VM
NICs HBAs CNAs
LAN SAN A SAN B
VMware guest
image datastore
Figure 14. Virtual Machine migration reference architecture
Ethernet Connectivity
Network connectivity on both of the ESX Servers is provided by a Virtual Switch (VS). The Virtual Switch is an
abstract networking device that can route traffic internally between Virtual Machines or link to an external
network via connection to physical network adapters.
On each of the servers, the VS enables connectivity to the service console, VMKernel, VMs, and to the
external LAN using two physical interfaces working together. The service console is used for ESX Server
management of the environment, the VMKernel runs services for VMotion, and Virtual Machine Network
handles VM network traffic.
Figure 15. Original VMware ESX Server with NIC and HBA adapters
The CNAs in the new server are presented to VMware ESX Server as NICs in the Network Adapters
Configuration window and provide the same functionality as traditional Ethernet NICs. A new VS is created
and configured identical to the original ESX Server except that the 10 GbE CNAs are used as the physical
adapters.
Figure 16. New server with converged 10 GbE adapters before migration
The Virtual Switches in the new server use NIC teaming and each of the physical interfaces is connected to
a separate Brocade 8000 for redundancy. VMs share the two VM NICs with active/active Port ID-based load
balancing. In the case of link failure on one of the VM NICs, the other handles all traffic to the external
network.
Figure 17. Virtual Switch teaming configuration
Storage Connectivity
Guest images residing on the original server are installed on a datastore located on a LUN in the FC SAN.
The storage exposes the LUN on multiple ports and is connected to both Fabric A and Fabric B for high
availability.
Figure 18. Datastore for guest images
On the new server, VMware identifies the CNAs as FC storage adapters (in this case, two Emulex LP21000
CNAs). Once the adapters are zoned to the storage in the FC fabric, the guest image data store appears as
shown in Figure 19.
Figure 19. ESX Storage Adapter Configuration
Performing the Migration

Migration of the VM using VMotion is performed using the Migrate Virtual Machine wizard in the VMware
Infrastructure client. After validating the configuration, all VM state information is copied to the new server
and host activities are resumed.
Figure 20. Network configuration of new server with CNAs after migration
APPENDIX A: SAMPLE CONFIGURATION

hostname B8000-112-cmsh
!
protocol spanning-tree rstp
bridge-priority 61440
!
vlan classifier rule 1 proto fip encap ethv2
vlan classifier rule 2 proto fcoe encap ethv2
!
cee-map srvgroup
priority-group-table 1 weight 40 pfc
priority-group-table 2 weight 60
priority-table 2 2 2 1 1 2 2 2
!
interface Vlan 1
!
interface Vlan 10
description server group1
!
interface Vlan 20
description server group2
!
interface Vlan 100
fcf forward
description FCoE VLAN
!
switchport
no shutdown
!
switchport
no shutdown
!
switchport
switchport mode access
switchport access vlan 10
no shutdown
cee srvgroup
!
switchport
switchport mode converged
switchport converged vlan 20
no shutdown
cee srvgroup
!
protocol lldp
advertise dcbx-fcoe-app-tlv
advertise dcbx-fcoe-logical-link-tlv
© 2009 Brocade Communications Systems, Inc. All Rights Reserved. 09/09 GA-SG-161-01
Brocade, the B-wing symbol, BigIron, DCX, Fabric OS, FastIron, IronPoint, IronShield, IronView, IronWare, JetCore, NetIron,
SecureIron, ServerIron, StorageX, and TurboIron are registered trademarks, and DCFM, Extraordinary Networks, and SAN Health
are trademarks of Brocade Communications Systems, Inc., in the United States and/or in other countries. All other brands,
products, or service names are or may be trademarks or service marks of, and are used to identify, products or services of their
respective owners.
Notice: This document is for informational purposes only and does not set forth any warranty, expressed or implied, concerning
any equipment, equipment feature, or service offered or to be offered by Brocade. Brocade reserves the right to make changes
to this document at any time, without notice, and assumes no responsibility for its use. This informational document describes
features that may not be currently available. Contact a Brocade sales office for information on feature and product availability.
Export of technical data contained in this document may require an export license from the United States government.

Brocade8000 - GA SG 161 01

Hochgeladen von

Dokumentinformationen

Originalbeschreibung:

Originaltitel

Copyright

Verfügbare Formate

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

Verfügbare Formate

Brocade8000 - GA SG 161 01

Hochgeladen von

Copyright:

Verfügbare Formate

FROM THE SOLUTIONS CENTER

 Both LAN and SAN configuration for file servers

 Virtual Machine migration

Overview of CEE and FCoE

 Fibre Channel (FC) SANs are deployed to support storage traffic.

 Ethernet LANs are deployed to support client-server application traffic.

2 Brocade 8000 Switches

Figure 1. ToR deployment model

Brocade 8000 Switch

Enhanced Ethernet switch

Figure 3. Brocade 8000 architecture

Brocade FCOE10-24 Blade

 Delivers line-rate 10 Gbps performance across 24 x CEE ports

 Provides rich Layer 2 Ethernet functionality for LAN traffic

Figure 4. The Brocade FCOE10-24 blade

 CEE interface display and configuration

 FCoE trunk display and configuration

 FCoE login groups

PLANNING AND DEPLOYMENT

Figure 5. Brocade FCOE10-24 in a Brocade DCX chassis with Brocade CNAs

Ethernet Fibre Channel

LAN SAN A SAN B

NOTE: In this document, connected are color-coded as follows:

 Fibre Channel in red

Existing server edge connectivity New server edge

LAN SAN A SAN B

Figure 7. Network convergence at the edge

Fabric Port Name: 20:08:00:05:1e:54:19:ae

Index Port Address Media Speed State Proto

19 19 021300 -- 10G Online FCoE VF-Port 0 VN-Port(s)

B8000-112:root> fcoe --loginshow

CEE and LAN Integration

Data center Brocade Brocade . . . connected

Server Group 1 Server Group 2

Configuring CEE Attributes

 The types of traffic flowing through an interface, FCoE, TCP/IP, and so on

 The minimum bandwidth required for each traffic type

 Which traffic type needs lossless behavior

Creating the CEE Map

cee-map <CEE Map Name>

 dcbx-fcoe-app-tlv: IEEE Data Center Bridging eXchange FCoE Application TLV

 Standard Spanning Tree (STP)

 Rapid Spanning Tree (RSTP)

 Multiple Instance Spanning Tree (MSTP)

protocol spanning-tree rstp

vlan classifier rule 1 proto fip encap ethv2

interface TenGigabitEthernet 0/1

interface TenGigabitEthernet 0/10

interface TenGigabitEthernet 0/11

Connecting Servers to the Brocade 8000

Figure 9. CNA protocol stack

FCOE DEPLOYMENTS AND USE CASES

Fabric A Fabric B LAN

Figure 10. Typical configuration before network convergence

Fabric A Fabric B LAN

Figure 12. NIC teaming on the CNA

Figure 13. Multipathed access to storage through two CNAs

Virtual Machine Consolidation and Migration

Migration with VMotion has three stages:

3. The VM resumes activities on the new host.

NICs HBAs CNAs

LAN SAN A SAN B