Connectivity Protocols SRG

Welcome to Connectivity Protocols.
Copyright ©2015 EMC Corporation. All Rights Reserved. Published in the USA. EMC believes the information
in this publication is accurate as of its publication date. The information is subject to change without notice.
THE INFORMATION IN THIS PUBLICATION IS PROVIDED “AS IS.” EMC CORPORATION MAKES NO
REPRESENTATIONS OR WARRANTIES OF ANY KIND WITH RESPECT TO THE INFORMATION IN THIS
PUBLICATION, AND SPECIFICALLY DISCLAIMS IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS
FOR A PARTICULAR PURPOSE.
Use, copying, and distribution of any EMC software described in this publication requires an applicable
software license. The trademarks, logos, and service marks (collectively "Trademarks") appearing in this
publication are the property of EMC Corporation and other parties. Nothing contained in this publication
should be construed as granting any license or right to use any Trademark without the prior written
permission of the party that owns the Trademark.
EMC, EMC² AccessAnywhere Access Logix, AdvantEdge, AlphaStor, AppSync ApplicationXtender,

ArchiveXtender, Atmos, Authentica, Authentic Problems, Automated Resource Manager, AutoStart,
AutoSwap, AVALONidm, Avamar, Bus-Tech, Captiva, Catalog Solution, C-Clip, Celerra, Celerra Replicator,
Centera, CenterStage, CentraStar, EMC CertTracker. CIO Connect, ClaimPack, ClaimsEditor, Claralert
,cLARiiON, ClientPak, CloudArray, Codebook Correlation Technology, Common Information Model, Compuset,
Compute Anywhere, Configuration Intelligence, Configuresoft, Connectrix, Constellation Computing, EMC
ControlCenter, CopyCross, CopyPoint, CX, DataBridge , Data Protection Suite. Data Protection Advisor,
DBClassify, DD Boost, Dantz, DatabaseXtender, Data Domain, Direct Matrix Architecture, DiskXtender,
DiskXtender 2000, DLS ECO, Document Sciences, Documentum, DR Anywhere, ECS, elnput, E-Lab, Elastic
Cloud Storage, EmailXaminer, EmailXtender , EMC Centera, EMC ControlCenter, EMC LifeLine, EMCTV,
Enginuity, EPFM. eRoom, Event Explorer, FAST, FarPoint, FirstPass, FLARE, FormWare, Geosynchrony, Global
File Virtualization, Graphic Visualization, Greenplum, HighRoad, HomeBase, Illuminator , InfoArchive,
InfoMover, Infoscape, Infra, InputAccel, InputAccel Express, Invista, Ionix, ISIS,Kazeon, EMC LifeLine,
Mainframe Appliance for Storage, Mainframe Data Library, Max Retriever, MCx, MediaStor , Metro,
MetroPoint, MirrorView, Multi-Band Deduplication,Navisphere, Netstorage, NetWorker, nLayers, EMC
OnCourse, OnAlert, OpenScale, Petrocloud, PixTools, Powerlink, PowerPath, PowerSnap, ProSphere,
ProtectEverywhere, ProtectPoint, EMC Proven, EMC Proven Professional, QuickScan, RAPIDPath, EMC
RecoverPoint, Rainfinity, RepliCare, RepliStor, ResourcePak, Retrospect, RSA, the RSA logo, SafeLine, SAN
Advisor, SAN Copy, SAN Manager, ScaleIO Smarts, EMC Snap, SnapImage, SnapSure, SnapView,
SourceOne, SRDF, EMC Storage Administrator, StorageScope, SupportMate, SymmAPI, SymmEnabler,
Symmetrix, Symmetrix DMX, Symmetrix VMAX, TimeFinder, TwinStrata, UltraFlex, UltraPoint, UltraScale,
Unisphere, Universal Data Consistency, Vblock, Velocity, Viewlets, ViPR, Virtual Matrix, Virtual Matrix
Architecture, Virtual Provisioning, Virtualize Everything, Compromise Nothing, Virtuent, VMAX, VMAXe, VNX,
VNXe, Voyence, VPLEX, VSAM-Assist, VSAM I/O PLUS, VSET, VSPEX, Watch4net, WebXtender, xPression,
xPresso, Xtrem, XtremCache, XtremSF, XtremSW, XtremIO, YottaYotta, Zero-Friction Enterprise Storage.
Revision Date: 11/2015
Copyright 2015 EMC Corporation. All rights reserved. Connectivity Protocols 1

Revision Number: MR-1WP-NSSSP.0.3.0
Copyright 2015 EMC Corporation. All rights reserved. Connectivity Protocols ‹#›
This course covers basic networking concepts and protocols associated with storage area
networks (SAN).

This course takes about four hours to complete and consists of eight training modules. Here we
have an outline of the major sections of the course. (In the launched version of the course, these
sections are live links to aid navigation through the course.)

This module focuses on basic networking concepts including switching, routing and the OSI 7-
layer model.

A network provides a way to share computer data and resources. In the old days we used to
transfer data from one desktop computer to another using floppy disk. This method is sometimes
affectionately called “sneaker-net”, because you had to literally walk the data from one user
desktop to another. Now we have a much better way. It is called the computer network.
The computer network is a collection of cables switches and routers that connect computers to
other computers and to peripheral equipment. Modern computer devices may use either
hardwired or wireless connections to talk to one another.
We are surrounded by computer networks in both our places of work and in our homes. Networks
allow us to connect with others and share instant messages, video, email, photos, documents,
and many other things. Networks also allow us to share resources such as a databases, file
systems, and printers. This course will describe the basic protocols and equipment used in today’s
modern business networks.

Here we show typical network components used in a home network or small business. End-user
devices include desktop PCs, laptops, smartphones or tablets, and other devices.
These end-user devices are connected together using wireless access points, cables, switches,
and routers. Through this network, multiple users can share common resources such as printers,
servers, network storage, and file systems.
The network can also connect users to the outside world through the internet and worldwide web.
Security is always a concern when connecting private networks to the outside world, so firewalls
and other security mechanisms are put in place to provide a level of protection.

When building a network, we start locally. Our local area network, or LAN, allows a group of
devices to communicate within a small geographical area, such as within the same building or
office.
A LAN can be as small as two PC’s connected to a shared printer in a small home office, or it can
extend to 200+ devices within an office building. We can connect these individual LANs with a
router. Each LAN connected to the router is a stand-alone switched network called a subnet.
If a network stretches across multiple buildings in a city or metropolitan area, it is called a MAN
(Metropolitan Area Network).
We often use a cloud icon to represent a network and hide the complex details showing how it is
built.

A WAN (or Wide Area Network) is used to connect a LAN in one geographical location another
LAN in a remote location. WANs cover distances measured in miles. A data center normally owns
its LAN infrastructure, but leases the WAN infrastructure from a service provider (ISP or Internet
Service Provider).
• LANs use routers to communicate across the WAN.
• Home office LANs often connect to the WAN using DSL (phone line) or cable modem
provided by the ISP. Businesses can purchase bandwidth via high capacity leased lines
from a service provider or use dark fiber leased from a third party.
Common components of a WAN include:

• Switches with routing capabilities
• Routers
• Firewalls
• NAT devices (Network Address Translation)
• Encryption and other security components and protocols (e.g., VPN - Virtual Private
Network)
The internet is a global WAN internetwork, which includes hundreds of millions of machines and
users worldwide.
Note: The difference between leased lines and dark fiber is that with leased lines, the ISP
provides equipment and operates the network to provide bandwidth to each of its customers.
Dark fiber is unused fiber optic infrastructure that is sold or leased by a supplier to an operator for
private long-distance network connectivity. The operator has complete flexibility and provides the
equipment and defines the protocols used to attach to dark fiber.

In the early days of networking, desktop personal computers were connected together using coax
cable and T-connectors (with impedance matching terminators at each end of the cable). This was
a simple, low cost approach to networking that was easy to implement. However, there were
problems. For example if the cable or T-connector broke on PC 2 in the diagram, then PC 1 would
not be able to communicate with PC 3 or PC 4.
An improvement was to use a hub. The hub had internal bypass connections, so you could unplug
a host and the network would still be operational. However a hub also has major drawbacks.

The problem with a hub is that it can ONLY have one conversation at a time. All ports share the
same loop, as shown on this drawing. So each port will see the data intended for all other ports.
If a device is not plugged into a port, an internal port bypass (B) circuit maintains the integrity of
the loop. Each conversation is half duplex (one direction at a time). All network protocols that
use a hub must have some way of detecting and handling data collisions, in case two ports try to
transmit at the same time, or a way of arbitrating, so only one port at a time transmits
information. Hubs are much slower than switches because of these constraints.
A switch is much better than a hub because it can have multiple conversations at the same time.
There is a separate circuit enabled for each data conversation providing a collision free
environment. All conversations through a switch are full duplex, meaning a device can transmit
and receive data at the same time (bi-directional).
As switches have come down in cost, they have replaced hubs for most applications. Hubs are
seldom used now for networking.

Each frame of information that travels in a network contains both a source and destination
address along with the data. The switch reads the destination address and uses it to direct the
frame to an outbound port that will send the frame further down the network to its destination.
Sometimes the destination is unknown, so the switch will broadcast the frame out all ports other
than the receiving port.
Some protocols also support multicast addresses, where the frame is sent out multiple ports, but
not all ports.
Broadcast require special handling so that loops between switches don’t cause broadcast storms
and network congestion. Spanning Tree Protocol (STP) is one example of a way to prevent
loops between switches and is discussed later in this course.

A router is a device that operates at a higher layer in the network protocol stack than a switch.
This is why routers are called Layer 3 devices and switches are called Layer 2 devices. We will talk
more about the network layers in a later lesson.
The purpose of a router is to forward packets of information from one network to another. Shown
are two subnetworks (subnetwork 1 and subnetwork 2). All devices attached to switches in
subnetwork 1 are in the same broadcast domain. Subnetwork 2 is a different broadcast domain.
The reason we create subnetworks is that we want to limit the scope of a broadcast and reduce
the amount of traffic in the overall network.
The router acts as a gateway from one subnetwork to another. If a host on subnetwork 1 sends
a packet of information to a host on subnetwork 2, it will be sent to the gateway address on the
router first. The router will forward the packet to subnetwork 2, where it will be switched through
the appropriate switches and arrive at the destination host.

There are several network topologies used for interconnecting network devices. The oldest and
the least commonly used now is the bus topology. In a bus topology, all devices share the same
transmission media (usually coax cable). Only one device at a time can transmit and all devices
receive the data. Impedance matching terminators are required at each end of the cable to
prevent signal reflection noise. To prevent multiple devices from transmitting at the same time,
each device must first listen before transmitting. If two devices do transmit data at the same
time, there is a data collision. The collision is detected by the transmitting devices and they each
wait a random amount of time before trying to re-transmit. This mode of operation is called
carrier sense multiple access with collision detection (CSMA/CD).
The star topology is the most common topology in Ethernet LANs today. The star topology has a
central connection point, and all devices have a private cable to the central device. The central
device is either a switch or a hub, although switches are preferred (as discussed previously).
Devices communicate with one another through the central point. Cable or connector problems
only affect one device while the rest of the devices remain operational. Because all data must flow
through the central connection device, it must be robust. If the central device goes down, the
entire network goes down.
An extended-star topology is a way to expand the star topology. Devices in a local part of the
network may not need to access the central connection point in order to access other local
devices. Again, the central connection point must be very robust in this topology.

A ring topology is similar to a bus topology, except that there is no beginning or end point that
must be terminated. All devices in the ring share a common cable, and data flows in one direction
only. Data frames are passed from device to device around the ring. Ring topologies often use a
token passing protocol to prevent data collisions. The token is passed around the ring (hence the
name Token-Ring). A device is not allowed to transmit unless it holds the token. The token is
shared in a fair manner to allow all devices to have an equal opportunity to obtain network
bandwidth. The disadvantage to ring topology is that a single station or cable failure can disrupt
the entire network. Troubleshooting can be more difficult than some other topologies as it may be
necessary to trace each cable to find a faulty connection.
Dual-Ring topology provides a redundant ring with data traveling in the opposite direction. This
design is more robust than single-ring topology. FDDI (Fiber Distributed Data Interface) is a
protocol that takes advantage of this topology using fiber optic cables. FDDI is often found in
backbone Metropolitan area networks.

A full-mesh topology connects each device to every other device in the network. This is done for
redundancy and fault tolerance. Full-mesh topologies are expensive because so many ports must
be used to make all the connections between devices. It also doesn’t scale well and is limited to
fairly small implementations. Full-mesh topologies are the most robust and resistant to failures
because if one device goes down, it does not affect the ability of other devices to communicate.
A partial-mesh topology is a lower cost way to provide some redundancy, but has less fault
tolerance. Normally a central device maintains connectivity with all other devices. A network
designer may decide which device is the most critical, and give it the most connections.
Fibre Channel networks often use a form of mesh topology.

Be aware that a network is often designed to use a hierarchy of switches and routers. For
example, here we show access layer switches, distribution layer switches and core (or backbone)
switches and routers.
Access Layer switches are where end devices, like hosts and printers, connect to the network.
Access layer switches provide uplinks to the distribution layer. They are usually dual attached to
the distribution layer switches for high availability.
Distribution Layer switches may be multilayer switches that provide a boundary between the
access and core layers. Multilayer switches may act as a switch attached to the access layer
switches and as a router when attached to the core. These switches can summarize addresses
and routes and relieve some of the processing burden from the core switches.
Core (Backbone) Layer switches are mainly concerned with very fast and efficient data
transport. For this reason, these switches are simple and typically provide minimal services. The
core layer is usually designed with redundant switches in a full mesh configuration for highest
availability.

The networking protocol that the average consumer is familiar with is Ethernet and TCP/IP. This
is the networking protocol that is used in home networks and to access the internet and the world
wide web. There are other network protocols used for various applications, and this course will be
discussing protocols especially associated with storage area networking.
The Fibre Channel protocol is used to attach host servers to storage in both small to medium
businesses and in large enterprises. Fibre Channel networks are used to increase the number of
servers that can access a storage array, and to increase the amount of storage that can be
accessed by each host. The Fibre Channel protocol is a serial protocol used to transport SCSI
(Small Computer Systems Interface) commands and data that is used to access raw block-level
storage. SANs, or Storage Area Networks are built out of Fibre Channel switches. As these
networks grow into multi-switch networks, they form Fibre Channel fabrics. These fabrics use
mesh or partial mesh topology.
iSCSI protocol is also used to attach servers to storage devices. The “i” in iSCSI stands for
“internet”. This protocol uses the internet protocol to transport SCSI commands and data. Since
iSCSI makes use of standard internet hardware, you will see the same switches and routers used
for iSCSI, as are used for other TCP/IP applications.

InfiniBand is a network protocol that was designed to be a high speed replacement for a server’s
internal PCI bus protocol and other data center networking protocols including Ethernet and Fibre
Channel. The idea is to replace multiple protocols with a single protocol. This vision has not been
fully accepted in the marketplace yet. The one area where InfiniBand has become popular is as a
high performance clustering interconnect for high speed computer applications.
FCIP (Fibre Channel over IP) is a network protocol that has found popularity in SAN distance
extension applications. If we have two sites with Fibre Channel storage that we want to connect,
we can connect them through an IP network. The Fibre Channel protocol is encapsulated into IP
packets. This distance extension solution can take advantage of existing CWDM/DWDM technology
and SONET/SDH networks between the two sites, which saves the cost of extending the Fibre
Channel network.
FCoE (Fibre Channel over Ethernet) is the newest protocol designed to converge multiple
protocols onto a single cable. It is starting to get some traction in the marketplace because it
provides an upgrade path for data centers that already have a significant Fibre Channel
investment. With FCoE, both Fibre Channel and Ethernet protocols are transported on the same
cable using enhanced (lossless) Ethernet. This is done by encapsulating Fibre Channel frames in
Ethernet frames for transportation. FCoE has an advantage over FCIP in that there is no TCP/IP
overhead.

Various organizations exist to develop and publish the standards used in modern computer
networking.
Go to the web pages shown here for additional details and standards publications for the protocols
discussed in this course.

To simplify the complex task of networking, a layered architecture is used. Each layer is
responsible for handling one or more of the tasks required for networking. There is no standard
number of layers that must be included in a network architecture. However most network
architectures are a variation of the theoretical OSI 7-Layer Reference Model.
TCP/IP is a 4-layer protocol that is an example of an implementation of the OSI model.
The architectural layers form a hierarchy and items are listed in order by rank. Higher layers
depend upon services from lower layers and lower layers provide services for upper layers.
There are many good reasons for implementing network architectures in layers. They include the
ability to make changes at one layer (or to re-design a layer) and have minimal impact on the
other layers. Learning the complexities of networking is easier if we concentrate on one layer at a
time. Troubleshooting efforts can also be focused on the specific layer that carries out the task
where a problem occurs.
Perhaps the most important reason for using a layered architecture is to provide standards for
how each layer interfaces with the layers immediately above and below. This standardization
allows all vendors to create devices and software that can interoperate with devices and software
from other vendors.
Note: The Open Systems Interconnection reference model was developed by the International
Organization for Standardization (ISO).

Each layer in the OSI reference model only interacts directly with the layer immediately beneath
it, and provides facilities for use by the layer above it. The following layers make up the OSI
model:
• Layer 1 Physical Layer - defines the electrical and physical specifications for devices.
• Layer 2 Data Link Layer - provides the functional and procedural means to transfer data
between network entities and to detect and possibly correct errors that may occur in the
Physical Layer.
• Layer 3 Network Layer - provides the function and procedural means of transferring
variable length data sequences from a source to destination via one or more networks while
also maintaining a quality of service requested by the Transport Layer.
• Layer 4 Transport Layer - provides transparent transfer of data between end users
providing reliable data transfer services to the upper layers.
• Layer 5 Session Layer - controls the connections between computers. It establishes,
manages, and terminates the connections between the local and remote application.
• Layer 6 Presentation Layer - establishes a context between Application layer entities in
which the high-layer entities can use different syntax and semantics.
• Layer 7 Application Layer - the layer closest to the end user, which means that both the
OSI application layer and the user interact directly with the software application.

Each source layer in this network model communicates logically with the same layer in the
destination device. Building our network protocol stack by layers allows us to replace a particular
layer with a new protocol, and not have to re-engineer the entire stack.
In this model, the only layer where physical bits (1’s and 0’s) are transferred is at the bottom
physical layer.

In a layered network model, both data and protocol information for the layer is placed in a PDU
or Protocol Data Unit.
The protocol information for the layer is put in a header, and the header is attached to the front
of the data.
The entire PDU is then passed down to the next layer where it becomes the data for that layer.
Again, protocol information for the layer is put into a header and attached at the beginning of the
data to form a PDU for that layer.
This new PDU is passed down to the next layer. Thus, the information grows as it is passed from
upper layers to lower layers. Finally, at Layer 1, the final PDU is encoded into a bit stream and
sent out by the transmitter as bits (electrical or optical on/off pulses).
The reverse process happens when the data is received. Layer 2 removes the Layer 2 header and
extracts the Layer 2 protocol information. The remaining information is sent to Layer 3, where the
Layer 3 header is removed and the Layer 3 protocol information is extracted. This continues up
the stack.
The example we have shown here applies to the TCP/IP/Ethernet protocols. The Layer 4 PDU is
called a TCP segment. The Layer 3 PDU is called an IP packet, and the Layer 2 PDU is called an
Ethernet frame. Layer 1 is always bits.
The header has layer specific information like addressing, formatting, and various switches and
flags.

This module covered introductory networking concepts including definitions, device descriptions
and an introduction to network protocols and the OSI 7-layer model.

This module focuses on the Ethernet protocol including frame format, MAC addressing, STP,
commonly used cables and connectors and virtualization.

This lesson shows where Ethernet fits in the OSI model and describes the Ethernet frame format.

The Ethernet protocol resides between the Network Layer and the Physical Layer. Ethernet is
responsible for providing an address scheme and delivery mechanism that facilitates the transfer
of data to devices on the same network as the sender. Ethernet provides no guaranteed delivery
mechanism and instead relies on higher level protocols to provide that functionality.
There are several versions of Ethernet. Not all of them are in use today. The most common type
is called Ethernet Type II.
The Ethernet Data Link layer is divided into two sublayers: Logical Link Control (LLC) and
Media Access Control (MAC).
The LLC layer is responsible for determining what Layer-3 protocol is being used and respond
appropriately.
The MAC layer defines how the information is placed on the media. This layer includes addressing
and channel access control.

Shown here is the frame format for the Ethernet protocol. All data and upper level protocol
information is placed into the data payload portion of the PDU. A header is added which specifies
the frame’s destination and where it came from, along with other information including the upper
level protocol type and an optional 802.1Q tag which includes a VLAN ID.
A unique 6-byte MAC address is assigned to every Ethernet device. When a device sends a frame
it puts its own address into the source MAC address portion of the header, and it puts the MAC
address of where the frame is being sent, into the destination MAC address portion of the header.
A FCS (Frame Check Sequence) is added to the end of the frame and is used to verify that the
frame is transmitted and received without error.

The Ethernet MAC address is 48 bits.
Bit 47 is the Individual/Group bit. This bit is set to 1 for broadcast.
Bit 46 is the Global/Local bit. This bit is set to 1 for a Local address.
Bits 45-24 are the OUI (Organizationally Unique Identifier) which is set for each vendor by IEEE.
Bits 23-0 are assigned by the vendor.
We have given you a couple of example MAC addresses. The first example is a broadcast
address as it might be displayed on a network analyzer. Each byte (8 bits or octet) is displayed in
hexadecimal and separated by colons.
The second address comes from a typical laptop computer where the user typed the ipconfig /all
command to find all of the network addresses assigned to the laptop. Here the bytes are
separated by dashes.

Here is an example showing how the source and destination MAC addresses are placed into the
Ethernet frame header.
We have abbreviated the addresses in the frame header for simplicity.

This lesson covers how MAC addresses are used to switch and forward frames.

Network layered architectures use different addresses at each protocol layer. For example, IP
addresses are used at the network layer, while if Ethernet is the protocol used at the Data Link
layer, then MAC addresses are used. ARP (Address Resolution Protocol) is the tool that
automatically maps between IP and MAC addresses. Every computer that uses IP and Ethernet
will use ARP. For example, you can type the command arp -a on your Windows PC and see the
table that maps between IP addresses and MAC addresses.

Here we show an example of how MAC address tables are built in the Layer 2 switch, and how
those MAC table entries are used to switch the frame to the proper outbound port on the switch.
This example has been simplified to illustrate how the basic concept works. We begin with a new
switch with no MAC address entries for any of its ports.
PC A, which is attached to port 1 on the switch, sends a frame to PC D. The frame header includes
the MAC address for PC A as the source of the frame, and PC D as the destination for the frame.
When the frame arrives at port 1 on the switch, if there is no MAC entry for port 1, the switch
records the MAC address for PC A in the MAC table, next to port 1.

The switch searches its table for the destination MAC address corresponding to PC D, but can’t
find a match.
When the switch does not know which port to forward the frame out of, it floods the frame to all
of the ports except for the port the frame was received on.

Sometime later, PC D will send a frame to PC A in response to the frame it just received. The
source and destination addresses are now reversed when sending the reply frame.
The switch receives the frame and creates a MAC address entry next to port 4 for PC D.
The switch checks the destination address in the frame and finds the MAC address for PC A. It
does a search of its table and finds the PC A MAC address in the entry for port 1. So the frame is
sent out port 1 to PC A.

This lesson covers how Spanning Tree Protocol is used to prevent network loops.

One issue that has to be solved in networking is the problem of loops forming that result in
broadcast storms. Take the example shown here. We have three switches: A, B, and C. They
are all connected together in a full mesh topology.
The problems start when a host sends a broadcast frame into this switched network. What is a
switch supposed to do with a broadcast frame? It is required to send the frame out every port on
the switch except the one it was received on. Switch A receives the broadcast frame from the host
and sends it out the other ports to switches B and C.
Both switch B and C receive the broadcast frame and send it out all other ports except the one
the frame was received on. You can see what begins to happen. You get multiple copies of the
frame going in all directions throughout the network causing congestion and poor performance.
This is what is referred to as a broadcast storm. This problem can be exacerbated when there are
multiple links between switches for redundancy.
Similar problems occur when a switch doesn’t have the destination MAC address in its tables and
must flood a unicast frame out all ports. This issue also affects the stability of the MAC address
tables in the switches, since frames with the same MAC address arrive on different ports.

The Spanning Tree Protocol (STP) resolves this problem by managing the physical paths between
switches and eliminating loops. Physical path redundancy is allowed, but loops are prevented by
blocking certain ports.
Blocked ports can pass BPDU (Bridge Protocol Data Unit) frames, which are used to communicate
the spanning tree protocol between switches, but data frames are not allowed to pass through a
port in blocking or discarding state.
In our example, the port on switch B that connects to switch A is in blocking state.
If the link between switch B and C goes down, STP protocol will unblock the port on switch B so
that it can use that port as an alternate communication path with switch A and switch C.

When switches are first connected, or whenever there is a network configuration change, the STP
protocol holds an election for a Root Bridge. The switch with the lowest Bridge ID wins the
election. The Bridge ID is comprised of a switch priority value and the switch MAC address. The
network administrator can ensure that a particular switch is elected as root by changing the
priority for the switch. The default priority is 32768. A lower number gives the switch higher
priority. If all switches are left at the default priority setting, then the switch with the lowest MAC
address will win the election.
We see in this example that Switch C will win the election because all switches have the same
priority, but Switch C has the lowest MAC address.

After the election Switch C is the root bridge. Ports on the root bridge that connect with other
switches are put into forwarding state and are given the port role of designated port.
Ports on the switches designated as a nonroot bridge that have the best path to the root will be
put in forwarding state and given the port role of root port. Other ports on the nonroot bridge
that are put in forwarding state are also given the role of designated port.
Any ports on a nonroot bridge that should be blocked to prevent a loop is put into discarding
state and the port will have the port role of alternate. In this example the alternate port that is
in discarding state is on Switch B because it has a higher Bridge ID. Alternate ports discard all
data frames in either direction. BPDU frames are still passed however.

Spanning tree protocols have evolved over the years into faster and more feature rich protocols.
The initial Spanning Tree Protocol has been replaced by the 802.1d-2004 standard which includes
RSTP (Rapid Spanning Tree Protocol). The original version of STP took 50 seconds or longer to
rebuild the network after a topology change. RSTP can do it in less than 10 seconds, or within a
few milliseconds of a link failure.
PVST+ and Rapid PVST+ are Cisco proprietary protocols that are based on the IEEE standards
but have Cisco proprietary extensions to support a separate spanning tree on each VLAN. The
default spanning tree mode on Cisco switches is PVST+
MSTP is the IEEE 802.1s standard that was merged into the IEEE 802.1Q-2005 standard. This
standard also supports a separate spanning tree for each VLAN.
Note on standards:
STP  Spanning Tree Protocol (IEEE 802.1d-1998 standard)
RSTP  Rapid Spanning Tree Protocol (IEEE 802.1w standard)

• Incorporated into IEEE 802.1d-2004 standard
PVST+  Per-VLAN Spanning Tree (Cisco proprietary)

• Based on 802.1d with Cisco proprietary extensions
Rapid PVST+  Rapid Per-VLAN Spanning Tree (Cisco proprietary)

• Based on 802.1w standard with Cisco proprietary extensions
MSTP  Multiple Spanning Tree Protocol (IEEE 802.1s standard)

• Incorporated into IEEE 802.1Q-2005 standard.

Today’s virtualization and converged network environments need something more advanced than
STP. TRILL (Transparent Interconnection of Lots of Links) is a protocol that is gaining favor in
many network implementations.
TRILL applies the Intermediate System-to-Intermediate System protocol (IS-IS) layer 3 routing
protocol to Layer 2 devices. This allows Layer 2 switches to route Ethernet frames.
Because TRILL does routing at layer 2, redundant links can be used without forming network
loops. This means we don’t need to block paths, resulting in fewer network hops and more
available network bandwidth.
TRILL is used with scalable NAS applications and is becoming an essential protocol in high
performance iSCSI and FCoE protocol environments.

STP blocks certain ports in the network to prevent loops. The result is that there are some
network paths that are not used. This is inefficient and leads to longer I/O paths and latency.
If we compare that to the way TRILL works - TRILL routes frames, and can thus use all available
paths, and bandwidth. This also leads to fewer hops and less network latency.

This lesson covers the most popular hardware used in Ethernet networking.

Ethernet is the most common Layer 2 network architecture in the data center. Along with
Ethernet switches and hubs, other common hardware you should be familiar with include network
interface controllers (or NIC cards) for connecting computers to the network, and copper and fiber
optic cables with their associated connectors.
Most end devices support connection speeds of 100 Mbps. Newer and higher end devices can
support speeds of 1 Gbps and even 10 Gbps (which used to be reserved for the backbone).
Ethernet standards have also defined 40 Gbps and 100 Gbps connection speeds which are being
adopted for connecting core data center equipment.

To attach a PC or server to a network we need a Network interface controller, or NIC. This used to
be a stand alone card, but most desktop PCs and servers now include this as part of the
motherboard hardware.
Laptop and mobile devices come with built-in wireless capability that allows them to connect to a
network wireless access point. Wireless is slower than hardwired connections supporting speeds
of up to 54 Mbps
High end systems can use a feature called NIC teaming to combine multiple NICs into a single
virtual NIC. This provides better performance and fault tolerance. Multiple NICs are essential on
computers that host multiple virtual machines.

Several forms of cabling have been used with Ethernet. Each type of cable varies in its supported
maximum cable distance and transmission speed. An exhaustive listing of the various cabling and
connection options is beyond the scope of this course. However, here we present some of the
most popular and interesting solutions.
Category 5 (Cat 5) Unshielded Twisted Pair cable is used to connect between an end user
device and a network device such as a hub, switch or router. A cable consists of four twisted pairs
(eight wires) in an insulated jacket. An RJ45 connector is typically used. There are 8 wires, but
only 4 wires (2 pairs) are used for most Ethernet connections.
There are several categories of twisted pair cable, such as Cat 5, Cat 6 and Cat 7. These
categories of cable vary in characteristics that affect the distance, quality and speed of
transmission. Higher category cables generally support higher transmission speeds than lower
category number cables.
Twinax cable is similar to coaxial cable, except that there are two inner conductors surrounded
by shielding instead of one. One conductor carries data in the transmit direction and the other
conductor carries data in the receive direction. The 10 Gb Ethernet implementation for this cable
specifies that the SFP+ is directly attached to the cable. These cables are popular for their low
cost, low power, and low latency. Their 5 meter maximum length makes them perfect for
providing high speed connections within racks of devices and networking equipment.
Fiber optic cables come in both single mode and multimode cable.

Fiber-optic cabling has advantages over copper cabling in that it can carry more bandwidth over
greater distances. It is also immune to electronic interference and eavesdropping. One
disadvantage of fiber optic cabling is that the connectors, transmitters, and receivers are more
expensive. Fiber optic cable is most often found at the core of the data center, and in long
distance communication applications.
The two basic forms of fiber-optic cabling are multi-mode and single-mode. Multi-mode fiber is
usually limited to distances of less then 500 meters. This is because a pulse of light will have
multiple modes (or paths) of propagation down the fiber. Because some of the light from the
pulse takes a longer path down the fiber, the pulse of light spreads out over distance. This is
called modal distortion. Multi-mode fiber is often paired with LED transmitters. LEDs emit light
at different wavelengths around a center wavelength. Since different wavelengths of light travel
at different speeds down the fiber, this also adds to distortion of the signal.
Single-mode fiber systems use lasers to generate light. The laser and single-mode cable
combination permit signals to travel much greater distances with less distortion than LED and
multi-mode cable systems. Single mode fiber can carry signals tens of kilometers without a
repeater.
The fiber-optic core is the portion of the cable through which the light travels. The cladding
surrounds the core. A protective outer jacket covers the core and cladding.
Fiber-optic cables are normally specified with two numbers representing the core and cladding
diameter in microns. The first number represents the core diameter and the second number
represents the cladding diameter. For example, 62.5/125 multi-mode fiber is frequently used for
Ethernet connections, 50/125 multi-mode fiber is frequently used for Fibre Channel (FC)
connections, and 9 micron, single-mode fiber is used for long distance Symmetrix Remote Data
Facility (SRDF) connections.

Several forms of fiber-optic connectors are used in the networking industry. When Gigabit
Ethernet was introduced in the early 2000’s, rather bulky GBICs (gigabit interface converter) with
SC style connectors were used for fiber optic cabling. As hardware shrank and network equipment
with dense port configurations were introduced, the smaller SFP (small form-factor pluggable) and
LC style connectors became popular. Both of these styles support both single mode and
multimode fiber.

This lesson covers basic VLAN theory of operation.

Without virtualization, we build flat networks, meaning there is only one broadcast domain (also
commonly called a collision domain). If a host sends a broadcast frame, the frame is propagated
out every port on every switch except the port the frame is received on. As larger networks are
built, more devices will be sending broadcast frames, leading to network congestion.
Also, in a flat network, there is no security between end devices. By default, all devices see all
other devices in the broadcast domain.

VLANs (Virtual LANs) are a solution to the scalability and security problems inherent in flat
networks. Each switch can be divided into several VLANs. Each VLAN is a separate broadcast
domain.
Switch ports can only be assigned to one VLAN. Each VLAN is treated as a separate network. If a
device wants to communicate with another device in a different VLAN, it must go through a
router.
VLAN trunks can carry the traffic from multiple VLANs between switches.
Ports on the switch that are designated as trunk ports can carry traffic from multiple VLANs (or all
VLANs). Ports on the switch that are designated as access ports can only carry the traffic of a
single VLAN.

Here we can see a simple example of how VLANs work. We have setup two VLANs: VLAN 100 and
VLAN 200. If a device in VLAN 100 sends a broadcast frame, it is forwarded to all ports associated
with VLAN 100. This is represented by the red arrows. If a device in VLAN 200 sends a broadcast
frame, it is forwarded to all ports associated with VLAN 200. The trunks carry traffic for both
VLANs. Devices in VLAN 100 cannot communicate with devices in VLAN 200, in this example,
even if the two devices are on the same switch.
So we can see that broadcast domains have been made smaller (reducing total broadcast traffic),
and we have established security between devices by assigning them to ports in different VLANs.
One practical use for VLANs is that we can take a single network for a large organization and
subdivide it between departments in the organization. Using VLANs we can minimize the size of
broadcast domains and provide security between the different departments in the organization.

When traffic from multiple VLANs is traveling down a trunk cable, how does the receiving switch
know which frame is for which VLAN? The answer is: by the frame’s embedded VLAN tag. The
VLAN tag is defined by the IEEE 802.1Q standard and is an optional bit of information that can be
added to each Ethernet frame. The tag is four bytes. Two bytes contain the VLAN ID.
Each access port on the switch can be assigned to a single VLAN only. With some exceptions,
administrator created Ethernet VLANs are given VLAN IDs between 0002 and 4094. Unassigned
ports are automatically included in VLAN 1 (VLAN ID = 0001) by default. This is why VLAN 1 is
often called the default VLAN. Untagged frames received by the switch are directed to the default
VLAN.
When a switch receives a frame, it will compare VLAN ID to filter table. If valid, and the MAC is on
another switch, the frame is forwarded through the trunk port to the next switch. If the
destination MAC is on an access port on the switch, the switch will remove the VLAN tag from the
frame and forward it on to the device.

If we think of each VLAN as a separate network, it should be easy to see that communication
between VLANs must go through a router. This is true even if two hosts are attached to the same
switch as shown in this example. Each VLAN will have it’s own Layer 3 gateway address on the
router. In this example, all hosts on VLAN 100 will have to go through the VLAN 100 Layer 3
gateway port on the router, in order to communicate with hosts in VLAN 200. Conversely, all
hosts in VLAN 200 will have to go through the VLAN 200 Layer 3 gateway port on the router, in
order to communicate with hosts in VLAN 100.

This module covered basic concepts of the Ethernet protocol.

This module focuses on IP and TCP / UDP protocols. It covers basic operations, addressing, and
subnetting concepts.

This lesson covers IP, TCP, and UDP protocols, and their place in the OSI 7-layer model.

TCP and UDP protocols operate at Layer 4, the transport layer. IP, or Internet Protocol operates at
Layer 3, the Network layer. The currently implemented versions of Internet Protocol are version 4
(IPv4) and version 6 (IPv6).

IPv4 was developed in 1978. At the time, a 32-bit address field (which allows for 4 billion
addresses), was thought to be more than enough to meet world-wide demand. However, the
developers didn’t envision that in addition to computers, there would be an “Internet of things”.
Today smart phones, tablets, appliances, cars, tracking chips, environmental controllers, sensors,
and many other “things” need internet addresses.
In 1993 an industry task force set out to develop IPv6. It was tested in development
environments for many years and finally introduced in a world-wide launch in June of 2012. IPv6
addresses are 128 bits, which means we can now have 340 million-trillion-trillion addresses!
IPv4 addresses use dotted decimal notation to specify an address. Each group of eight bits
(called an octet) in the address is translated into a decimal number from 0 - 255. The octets are
separated by dots, so we call this dotted decimal notation. This gives us addresses like the
sample shown of 10.127.80.10.
IPv6 addresses use hexadecimal notation, with the address broken up into eight quartets
separated by colons. Each quartet is four hexadecimal numbers.
IPv4 is still popular, especially in the United States, and the address space has been increased
using NAT (Network Address Translation). Europe and Asia have led the way with IPv6
implementations. It is predicted that by 2018, IPv6 will be the dominate IP version used
throughout the world. Many devices being sold now support both protocols.

The IPv4 packet carries Layer 4 data and protocol in the payload. The header is a variable size
depending on options. The important fields we want to focus on are the version, (for IPv4 the
version number is 4), the protocol field (which specifies the protocol being carried in the payload),
and the source and destination IP addresses. The header also carries a checksum for the header
only. There is no checksum for the entire packet. The header has length descriptors to specify the
length of the header and the entire frame.
Many protocols can be carried in the payload. Among them are ICMP, TCP, UDP, and IPv6
encapsulation.
Notes on acronyms:
• ICMP: Internet Control Message Protocol
• TCP: Transmission Control Protocol
• UDP: User Datagram Protocol

The IPv6 packet carries Layer 4 data and protocol in the payload just like IPv4 packets. The
difference is in the header. IPv6 addresses are 128 bits in length, so the address fields in the
header are much larger.
Instead of having a protocol field in the header, there is a next header field. This is the field that
specifies if there are optional header extensions and what the protocol carried in the payload is.
We can see the most common protocols supported by IPv4 are also supported by IPv6.

It is anticipated that it will take many years to change from IPv4 addressing to IPv6 addressing.
So we must have a way to let both types of addresses exist concurrently. There are two ways to
do this. One way is to upgrade software on current devices to support a dual IP address stack.
This is called the dual stacks approach. This means that a host or router has both IPv4 and IPv6
addresses associated with each NIC or port. The device can send IPv4 packets to other IPv4
devices, and it can also send IPv6 packets to other IPv6 devices.
The second way is by tunneling. In the tunneling approach, an IPv6 packet is encapsulated
inside an IPv4 packet. The IPv4 packet can then be forwarded over an existing IPv4 network. At
the other end of the network a dual stack device removes the IPv4 header and extracts the IPv6
packet.

TCP and UDP are Layer 4 network protocols. Here we can see the differences between these two
protocols.
TCP is a connection oriented protocol. This means that a virtual circuit is established at the
beginning of a connection and windowing is used to establish flow control. The sender can only
transmit the amount of data that has been requested, according to the TCP window value.
Each TCP segment must be acknowledged. Sequence numbers are tracked. The combination of
sequence numbers and acknowledgements allow the devices to verify that all segments were
delivered, or if not delivered, which segments didn’t arrive and must be retransmitted. Examples
of applications that use TCP are FTP (File Transfer Protocol), Telnet, SSH (Secure Shell), and
SMTP (Simple Mail Transfer Protocol, or email).
UDP on the other hand is a best effort protocol. There is no setup time to pass any flow control
values between sender and receiver. Since there are no acknowledgements, the protocol is fast
and efficient. Applications that use UDP for the transport layer protocol can loose a data
transmission every once in a while and not have much affect on the end user. Examples of UDP
applications are voice, video and audio streaming. If a frame of video is lost every once in awhile,
it doesn’t have much effect on the end user.
There is a related protocol called RUDP (Reliable UDP) which is like UDP except that it attempts
in-order packet delivery using time stamps in each packet.

Here we compare the TCP header information with the UDP header. We can see the difference
between an acknowledged reliable protocol - TCP, and an unacknowledged unreliable protocol -
UDP. Notice how the TCP header, in addition to the source and destination port numbers, also
has a sequence number to track where this frame fits in a sequence of frames, an acknowledge
number to specify which frames are being acknowledged, and a window size to specify how much
more data the receiver is able to accept.
The UDP segment doesn’t have any of that extra information in the header, and is therefore a
much more efficient protocol than TCP.

Applications that use TCP and UDP protocols communicate with one another through port
numbers, sometimes called sockets. Well-known port numbers are set aside for certain
applications by the Internet Assigned Numbers Authority (IANA) . Here we show some of the
more popular well-known port numbers for applications that use TCP and UDP.

Over the years, several creative ways have been devised to get around the limited IPv4 address
space. One way is to perform Network Address Translation, NAT, and Port Address Translation or
PAT.
Address translation allows us to take a private network address that is not internet ready and
translate it to a public address that is used to communicate across the internet.
PAT allows us to perform address overloading. What this means is that if an ISP assigns us only a
single public internet address, for example on our home network, we can use a router to translate
that single address into several private addresses for all of our home devices. The router can
distinguish between communications targeted for our desktop and information for our laptop,
even though they share the same public address. This is done using TCP port numbers. Port
numbers are written by adding them with a colon to the end of the IP address. For example:
68.85.1.6:1024 (where 1024 is the TCP port number).
The router keeps a NAT table that allows it to do the translation. In our simple example, we have
three host with private addresses on the 10.1.1.0 network. There is one public address to share
between these three devices. It is 68.85.1.6. By adding a port number to this address, we can
come up with different IP Address/port combinations to correspond with each of our private
addresses. Theoretically, we could have thousands of private addresses that share a single public
address.
To illustrate, let us see what happens when the host with private address 10.1.1.2 sends a packet
to the host at 200.1.2.3. When this packet arrives at the router, which sits on the boundary
between the private and public address space, a NAT translation will take place. The router will
reference its NAT table and replace the private address with the public address, before sending
the frame on to the internet. The reverse translation happens when frames are received from the
internet.

This lesson covers IPv4 addressing, address classes, and subnetting.

Before your computer can communicate with other computers on a network, it must have an
address. This address is known as an IP Address. Users are probably most familiar with IPv4
addresses which consist of four octets of the form: n.n.n.n. Each octet represents 8 bits and is
expressed as a decimal number from 0 to 255. This form of IP address is known as IPv4, denoting
IP Version 4.
To improve performance and manageability, networks are divided into subnetworks, or

subnets. IP addresses are used for routing packets from one subnet to another. Each device on
the network has an IP address. To get to other networks, each device must point to a gateway
address on the router. Routers are used to manage traffic and constitute borders between
subnets. Once a packet arrives at a router, the router has tables that enable it to direct the
packet to the lowest cost (or shortest) path to the destination network.

An IPv4 address is 32 bits wide and is composed of two parts: the network ID and the host ID. It
is expressed as four decimal numbers separated by periods, such as 200.1.2.3.
The valid addresses range from 0.0.0.0 to 255.255.255.255, a total of about 4.3 billion addresses.
The first few bits of the address indicate the Class that the address belongs to.
Class A addresses can be recognized by an address from 1 to 126 in the first octet of the
address. A Class A network can support over 16 million hosts.
We have shown the classful network ID part of the address in red, to make it stand out from the
host ID part of the address. The network ID, sometimes called the prefix, is the part of the
address used by a router to route from one network to another. Routers don’t care about
destination hosts, they only care about destination networks. Once a packet gets to the correct
destination network a switch will direct it to the correct host.
Class B addresses can be recognized by a first octet ranging from 128 to 191. A Class B network
address can have up to 65534 hosts.
A Class C network address begins with a first octet ranging from 192-223. Class C networks can
have up to 254 hosts.
There are two other address classes that will not be discussed in this course. Those are Class D
addresses for multicast, and reserved Class E addresses.

IPv4 addresses are divided into public and private addresses. A public address is a worldwide
unique range of addresses assigned to an organization or internet service provider. ICANN
(Internet Corporation for Assigned Network Numbers) owns the process and assigns ranges of
addresses to a regional Internet registry (RIR) where they are divided up and assigned to ISPs
and smaller registries. ISPs can then assign address ranges to their customers.
There are only so many of these addresses to go around, as can be quickly seen by the fact that
there are only 126 unique Class A network address ranges, each having 16 million hosts.
Because most network traffic in an organization stays within the local organization, a solution was
devised that allows a nearly unlimited number of networks. This solution involves using private
addresses and network address translation (or NAT).
An organization can use a private range of addresses for internal communications. Private IP
addresses are not routable on the Internet, so NAT is used to translate between these private
addresses and a few public address supplied by an ISP.
As an example, Class A networks have an address range of 10.0.0.0 to 10.255.255.255 set aside
as private addresses. Multiple organizations can use these same addresses because they are
private, and are not used outside of the organization’s network.

How do we know if two IP device addresses are in the same subnetwork? This is an important
question, because, if they are not, then to communicate, all frames must pass through a router. If
they are on the same subnet, a router is not necessary.
A 32-bit subnet mask is used to determine if two addresses are in the same subnetwork. The
subnet mask specifies which part of the address contains the network ID, and which part of the
address contains the host ID.
This example shows how this is done. The net mask is specified in dotted decimal notation just
like the address. Our subnet mask for our Class A address is 255.0.0.0. Each octet where the net
mask is set to 255 corresponds to the part of the address representing the network ID. Each octet
where the net mask is zero corresponds with the host ID part of the network address. For this
Class A network address, the network ID is 68, and the host ID is 214.99.72. This is what is
known as a classful network address, because it matches the default Class A network address
division.
We can also see the default classful subnet mask for Class B and Class C network addresses.

In this example we demonstrate classless addressing. Our address has 10 as the first octet, so
we know that this is a Class A private address. Normally a Class A address defines the network ID
with the first octet and the last three octets are for the host ID.
However with classless addressing, we can have a subnet mask that changes the default
addressing scheme.
In this example, we choose a mask of 255.255.255.0. This means that the first three octets will
identify the network and subnet ID. Only the last octet will define the host ID.
With this method, we can divide up our Class A addresses into many subnetworks. Each
subnetwork can have up to 254 devices.
An alternate way to define a classless IP address is by using CIDR notation. CIDR stands for
Classless Inter-Domain Routing.
With CIDR, we specify the number of the subnet mask bits that are set to 1 with a forward slash
followed by a number. In this example the subnet is 10.127.82.0 and the subnet mask is
255.255.255.0. 255 in decimal is equivalent to 8 binary ones. Since we have 3 octets of 255, we
have 3 octets with all eight bits set to 1. Three times eight is 24, therefore the CIDR notation is
/24.
Knowing this information we can calculate the subnet address, host address range for the subnet,
broadcast address, and the next subnet number.

If an organization is using worldwide unique public addresses, they will get those through their
ISP as described previously. Private address ranges are assigned by the organization that uses
the addresses. But how does a particular device get it’s address?
There are two ways a device can get its IP address. The first is to have an administrator assign a
static address. This might work if there are only a few computers which need addresses. But it is
much easier if we use the second method, which is to automatically assign the addresses. This is
done by setting up a DHCP server.
DHCP is Dynamic Host Configuration Protocol. An administrator will set up a DHCP server and
give it either a public or private range of IP addresses that can be assigned, as well as the subnet
mask and address of the default gateway for each subnetwork. It also has pointers to DNS
(Dynamic Name System) servers which map hostnames to IP addresses.
A device that is setup to dynamically get its IP address will send out a broadcast when it first joins
the network. This broadcast will make its way to the DHCP server. The DHCP server responds with
an IP address that is leased to the device for a certain amount of time, and the other information
the device needs to communicate on both the internal network (intranet) and internet.
Once a device has this information it knows enough to make internal routing decisions. For
example, it will know if a host it wants to communicate with is on the same subnet and can be
addressed directly, or a different subnet, thus requiring a router gateway.

This lesson covers IPv6 addressing, subnetting and autoconfiguration.

IPv6 addresses are 128 bits long. That is eight quartets of 16 bits each, specified in hexadecimal
notation. Each quartet is separated by a colon.
Since IPv6 addresses are so long, there are two conventions used to shorten the way we write an
address. The first convention allows us to omit leading zeros in any quartet. This is the same as in
decimal notation. Leading zeros are not required, but are understood to be present to fill out a
quartet. For example the hexadecimal number 0015 can be written as 15.
The second convention is to represent 1 or more quartets of all zeros with a double colon (::).
The double colon shorthand may only be used once for a given address.

IPv6 has most of the same address types as IPv4 with a few exceptions. Instead of private
addresses, there are Link-local addresses. There is also Anycast addressing, where a packet is
only delivered to the closest interface - the one with the least routing distance.
One other thing to note is that there are no broadcast addresses in IPv6.

IPv6 uses a classless view of addressing with a CIDR type notation to specify the prefix and host
ID parts of an address. In this example, the /64 means that the first 64 bits are the prefix bits.

Here we see an example where an ISP provided a 48-bit prefix to an organization. The
organization can then use the next 16 bits for subnetting. The network administrators can use
this information to program the ports on the organization’s router interfaces. Each would have a
/64 prefix.
With 16 bits, the organization can create over 64,000 subnets. Each subnet can have 1.8 x 1019
unique addresses. This demonstrates the power of having a 128-bit address space in IPv6.

How does a device obtain an IPv6 address? IPv6 gives a device the ability to configure its own
address using a process called stateless autoconfiguration. There is a second process called
stateful autoconfiguration which uses a DHCP server similar to IPv4.
In stateless autoconfiguration a device will use its MAC address for the lower 64 bits of the
address. Since the MAC address is only 48 bits we supplement it by putting the hexadecimal value
FFFE right in the middle of the MAC address.
In our example, the MAC address for the PC is 0024.E899.BED7, so the lower 64 bits of our IPv6
address is 0224.E8FF.FE99.BED7.
To get the upper 64 bits of the address, a device will use ICMP (Internet Control Message
Protocol) to send a Router Solicitation to the router’s multicast address. The router will return
an ICMP Router Advertisement message that contains the IP prefix.
This type of address is called an EUI-64 address. If this is a unique worldwide address, the U bit
will be set. This is why we see 0024 in the MAC address become 0224 in the IPv6 address.
Note: EUI is Extended Unique Identifier.

This lesson covers the basic network concepts of how a computer communicates with a computer
on another subnet.

Here we have created an example where we have a host, called myhost1, that has already
received its IP address information from a DHCP server, and now wants to communicate with a
host called fshost1. Notice that fshost1 is on a different subnet than myhost1.
In the next several slides we will look at the major steps required to communicate between these
two hosts.

To begin, myhost1 has an application that request FTP (File Transfer Protocol) data from fshost1.
The application asks the transport layer to set up a session to 192.168.20.44. The data for the
request is passed to Layer 4 where TCP will add a Layer 4 TCP header with port 21 as the
destination and the TCP SYN flag set, indicating we want to initiate a TCP session.

IP Layer 3 takes the PDU and adds the Layer 3 IP header information. This includes the
destination IP address and source IP address. Type is set to protocol type 6 for TCP. The IP packet
is then passed to Layer 2.

The destination IP address is compared to the subnet mask. It is discovered that the destination
address is on a separate network. We still need to keep the destination IP address, but we also
need to know the IP address of the default gateway. Once we know the IP address of the default
gateway, we can direct the Layer 2 frame to the MAC address corresponding to the default
gateway. Remember frames are transported on the local network using Ethernet MAC
addresses. However IP packets go from subnetwork to subnetwork using IP addresses.
Ethernet Layer 2 asks ARP (Address Resolution Protocol) to get the destination MAC address
corresponding to the IP address for the default gateway at 192.168.17.254. But in this case ARP
does not currently have a map entry for that address. So the frame is held by Layer 2 while ARP
gets the MAC address.
ARP provides the service to resolve IP addresses to MAC addresses, and it maintains a cache of
the mappings.

So while Ethernet is waiting for the destination MAC address, ARP builds an ARP request frame.
The destination MAC is FFFF.FFFF.FFFF, which is the broadcast address. The source MAC is the
one belonging to the NIC card on myhost1. The data in the frame basically says “If you are IP
address 192.168.17.254, send me your MAC address. I’m including my MAC address and IP
address so you know where to send the information.”

The ARP data frame is sent on myhost1’s physical NIC interface. When Switch1 gets the frame, it
will do two things with it. First, it will record the MAC address 0800:0111:1111 in its MAC address
table for port 2. Since the destination MAC address is a broadcast address, the switch will also
forward the frame out every port except the port it was received on.
When the IP router gets the frame, it will look at the ARP data in the frame and recognize its own
IP address. This will cause the router to respond with an ARP reply unicast frame and include its
MAC address as the source and the myhost1 MAC address as the destination.

When Switch1 receives the ARP reply it updates its MAC address table for port 3. It also does a
MAC lookup for the myhost1 MAC address and finds an entry for port 2. So it forwards the frame
out port 2.
The ARP protocol on myhost1 receives the MAC Reply frame with the MAC address for the default
gateway.

The ARP protocol on myhost1 returns the MAC address associated with the default gateway to the
link layer that has been waiting for the address. The link layer inserts the address into the Layer 2
header. The L2 header will now have the destination MAC address of 0100:5555.5555 . The
encapsulated Layer 3 header still contains 192.168.20.44 as the destination IP address. The MAC
address will be used by the switch to forward the frame to the router, and the IP address will let
the router direct the packet to the correct network.

Switch1 receives the frame and looks at the destination MAC address. It looks up the address in
its MAC table and directs the frame out port 3. It also refreshes its MAC table entry for port 2.

The IP Router receives the frame and strips off the Layer2 header. Then it looks at the Layer 3
header and compares the destination IP address with its routing table entries.
We’ll assume that there is an entry for the 192.168.20.0/24 network. The router notes the
outbound interface, but it is still missing a critical piece of information. Once the frame is on the
next subnet, it will be forwarded on that subnet by its MAC address.
So the router must access its own internal ARP process to get a MAC address corresponding to IP
address 192.168.20.44.

In this example the IP Router internal ARP has an ARP table entry for 192.168.20.44. So Layer 2
on the router is able to create the Layer 2 header. The Source MAC address is 0100:0777:7777
and the destination MAC address is 0800:0222:2222.

Switch2 receives the frame and saves the default gateway MAC address for port 7 in its MAC
address table. It checks its MAC table for an entry for 0800:0222:2222, but there isn’t one. This
could be because the cached entries age out of the table over time.
Since the switch doesn’t know where to send the frame, it floods the frame out all ports except
the port it received it on.

At Ethernet Layer 2, fshost1 receives the frame and uses the FCS (frame check sequence) bits to
detect any errors. The Layer 2 header is removed and read. The header indicates that the payload
is for the IP protocol on this host. So the PDU is passed up to Layer 3 IP. The other host will
discard the frame because the IP address doesn’t match that host.

At IP Layer 3, fshost1 takes the packet and strips off the Layer 3 header. It sees that the PDU is
for TCP, so the segment is passed up to the TCP layer of the protocol stack.

At TCP Layer 4, fshost1 receives the frame and looks at the port in the header and sees that the
payload is for the FTP application. The TCP header is stripped off and the data is given to the FTP
application.

The FTP application on fshost1 receives the FTP request and formulates a response.

This module covered many details required to understand TCP/IP networking including IPv4 and
IPv6 addressing, subnets, ARP, and routing.

This module focuses on Fibre Channel protocol layers, topologies, addressing, name services, and
deployment models.

This lesson covers the Fibre Channel layered model and describes Levels FC-0 through FC-4, as
well as possible topologies.
Fibre Channel is the major networking protocol used in enterprise Storage Area Networks or
SANs. Fibre Channel has a lot of capability, but for the purposes of this training, we will only focus
on capabilities used in SAN deployments. Fibre Channel was influenced by the work done on the
OSI 7-layer model, so you can see that it is a layered protocol. However the layers are called
levels in the Fibre Channel standard, and consist of levels FC-0 through FC-4 plus an Upper Layer
Protocol. No FC-3 common services are deployed in production SANs, so we will skip that level
and only talk about levels 0, 1, 2, and 4.
Fibre Channel is a standard managed by the InterNational Committee for Information Technology
Standards (INCITS) T11 technical committee.
The FC-0 level describes the physical Fibre Channel link and connectors. The FC-0 level includes
both copper and fiber optic media and the associated transceivers capable of operating at 1, 2, 4,
8, 10, or 16 Gbps. Standards are continually evolving to support higher and higher speeds.
Fibre Channel SANs are normally deployed in switched fabric topologies. Communications are full
duplex, meaning you can simultaneously transmit and receive between devices. There are two
conductors in each cable: one for transmission in one direction, and one for transmission in the
other direction.
Although Fibre Channel supports copper connectivity for short distances (within a rack for
example), most SAN applications use optical transceivers and cable.
Today, the LC style is by far the most popular Fibre Channel connector. SFP (Small Form-factor
Pluggable) transceivers support speeds up to 4 Gbps. SFP+ transceivers support 8 Gbps, 10 Gbps
and 16 Gbps top speeds. These come in versions to support standard and extended distances.
The mini-SFP and mini-LC transceiver and cable are a specialty solution designed by Brocade to
squeeze 64 connectors onto a single blade. Other than that specific application, you probably
won’t see the mini LC connectors used.
Distance is a consideration while implementing Inter-switch links (ISL) between Fibre Channel
switches. This is especially important when a fabric spans campus distances. For example, two
datacenters a few miles apart would use longwave laser (LWL) instead of shortwave (SW).
The three media options that are available while implementing an ISL are Multimode ISL, Single-
mode ISL, and DWDM ISL.
Some variables that affect supportable distance are propagation and dispersion losses, buffer-to-
buffer credit, and optical power.
Using this chart, we can see that for shortwave SFPs and multimode fiber, distances are limited to
500 meters or less.
For longer distances, longwave laser (LW, LWL) or extended longwave laser (ER, ELWL) over
single-mode 9-micron fiber optic cable is required. This solution is least susceptible to modal
dispersion, thereby enabling distances between 10 km and 40 km, depending on the vendor and
SFP type used.
For greater distances than shown here, CWDM or DWDM (Coarse Wavelength Division Multiplex /
Dense Wavelength Division Multiplex) may be used between switches. DWDM supports distances
up to 3000+ km.
Each device on a Fibre Channel network will use a port and cable to make the connection. Ports
will come online in one of the modes shown here. NL_Ports and FL_Ports (which are used in loop
environments) are not used much in SANs anymore. Some switches have ports that come up in
generic mode when they are not connected to anything.
Vendors may expand on the standard port types. For example, Cisco introduced the TE_Port to
extend the E_Port for VSAN capabilities.
End devices like computers and storage devices will have N_Ports.
F_Ports are switch ports that connect to N_Ports.
E_Ports are switch ports that connect through an ISL to another switch E_Port.
The Host Bus Adapter, or HBA, is a card that plugs into a bus slot on a server and provides Fibre
Channel connectivity. It is similar to the NIC card that provides connectivity to the Local Area
Network. The HBA will operate in N_Port mode to attach to a fabric. If attached directly to a
storage device, the HBA may operate in NL_Port mode.
HBAs come in single port and multi-port versions. HBAs support various Fibre channel speeds up
to 16 Gbps.
EMC qualifies HBAs and associated software from various vendors to work with different hosts,
operating systems and switches. An HBA should not be used that has not been qualified by EMC.
See the EMC support matrix for qualified HBAs and host systems.
FC-1 defines the transmission protocol including serial encoding and decoding rules, special
characters, and error control. At less than 10 Gbps, each information byte (8 bits) is encoded into
a 10-bit Transmission Character. This is fairly inefficient as 20% of the bits effectively are
overhead) For interfaces operating at 10 Gbps and 16 Gbps, every 8 bytes (64 bits) is encoded
into a 66 bit transmission character. This results in only 3% overhead, so the actual clock speeds
don’t have to be as fast to achieve the same effective data rate.
The primary rationale for using a transmission code is to improve the transmission characteristics
of the serial stream of bits on the cable. The bit stream must be DC balanced to support the
electrical requirements of the receiving units. The Transmission Characters ensure that enough
transitions are present in the serial bit stream to make clock recovery possible. This is also the
first level of error checking, as most single bit errors will be detected as an ‘invalid character’.
The encoding process results in the generation of Transmission Characters. Two types of
Transmission Characters are defined: Data characters and Special characters. Certain
combinations of Transmission Characters, referred to as Ordered Sets, have special meaning.
The Ordered Sets are used to identify frame boundaries (for example start of frame characters,
SOF), transmit primitive function requests (like R_RDY to replenish buffer-to-buffer credit), and
maintain proper link transmission characteristics by sending idle characters during periods of
inactivity.
The FC-2 level serves as the transport mechanism of Fibre Channel. It includes data framing,
frame sequencing, flow control, and class of service. There are several defined classes of service,
but SANs usually use class 3, which is a connectionless class of service. Class 3 frames are sent
without verification that the frame is received.
Frames contain the information to be transmitted, the address of the source and destination
ports, and link control information. Frames are broadly categorized as data frames and link
control frames.
It is the FC-2 layer's responsibility to break the data to be transmitted into frame size chunks and
reassemble the received frames into sequences and exchanges for the upper level protocol.
Frames are the basic building blocks of a Fibre Channel connection. The frames contain header
information, data, and CRC (Cyclic Redundancy Check) for error checking. The frame is delineated
by Start of Frame (SOF) and End of Frame (EOF) special sequences. All information in Fibre
Channel is passed in frames. The maximum amount of data carried in a frame is 2112 bytes with
the total frame size of 2148 bytes.
The header contains the Source and Destination Addresses, which allows the frame to be routed
to the correct port in the Fibre Channel network. The ‘Type’ field interpretation is dependent on
whether the frame is a link control or a Fibre Channel data frame. For example, if the frame is a
data frame, a ‘08’ in the ‘Type’ field indicates SCSI FCP (Fibre Channel Protocol) information in
the Data field.
Notice that the header also keep track of the sequence number, originator exchange ID (OX_ID)
and receiver exchange ID (RX_ID) so each frame of data can be placed in its proper context.
FC-2 also provides flow control for buffer management and to prevent data overruns. Buffer-to-
buffer credit (BB_Credit) is used to control the flow of data across a link. This is especially
critical when data is transmitted over long distance cables and DWDM. In those cases, extra
buffering is usually required. Credit is negotiated when each link first comes up and reflects the
number of receive buffers that are available to receive Fibre Channel frames. A device may only
send a frame when it has credit to do so. This avoids overrunning the receive buffers and
consequently losing data frames.
Every time a frame is sent, credit is decremented by one. Every time a receiver makes a frame
buffer available, it sends an R_RDY primitive to the sender replenishing a credit.
Most data center Fibre Channel links are well under 500 meters. So we normally don’t have to
worry about BB_Credit because the default allocation is sufficient. However, on long distance
links, there may be several frames in-transit on the link, and BB_Credit is used up before R_RDY’s
can be sent back. Performance is impacted while the transmitter waits to receive credit.
FC-4 is the highest level in the Fibre Channel structure. It defines the application protocols that
can be transported over Fibre Channel. It specifies how upper layer protocols map to the lower
Fibre Channel levels.
The purpose of an FC-4 protocol mapping is to make a logical connection between ULP and Fibre
Channel’s transport facilities. It provides a logical connection between two architectures. For SANs
carrying block storage traffic, SCSI is the ULP that is mapped to FCP (Fibre Channel Protocol).
Upper layer operations are divided into information units that map into Exchanges, sequences
and frames. The Information Units are the data which is passed between the FC-4 level and ULP
(Upper Layer Protocol).
As an example of how all of this comes together, let us look at a single SCSI I/O operation. A
SCSI Read command is executed in a three step operation. First, the SCSI initiator sends the
SCSI Read command, which includes the block address on the LUN and how many blocks to read.
Second, the SCSI target sends the requested read data. A Fibre Channel frame holds
approximately 2 Kilobytes, so if reading 100 blocks of data (around 50 KB), it will take many
frames to complete the sequence. Finally, after the data has been sent, the target reports the
status of the operation in a SCSI response.
These three operations are each turned into an information unit that is passed to the FC-4 level.
Each information unit creates a sequence at the FC-2 level, and each sequence is split into one or
more frames. The entire SCSI Read operation is known as a Fibre Channel Exchange. Each SCSI
I/O Read or Write operation is a separate Fibre Channel exchange. Each I/O will have an
Exchange ID (OX_ID) which is imbedded in the Fibre Channel frame. Also imbedded in each
Fibre Channel Frame will be the Sequence ID (SEQ_ID).
There are three basic topologies in Fibre Channel:
The point-to-point topology provides a dedicated full-duplex link between two nodes.
Fibre Channel Arbitrated Loop or FC-AL provides the ability to connect between at least 2, and
up to 126 nodes that share a common loop. Practical SAN implementations seldom have more
than 8 nodes before the loop becomes too congested with traffic to add more devices. FC-AL is
usually implemented through a FC-AL hub. In the early days of Fibre Channel these were quite
popular in SANs, but now hubs have been almost entirely replaced by switches.
Fibre Channel Switched Fabric or FC-SW provides a dynamic switched fabric with an address
space of more than 15 million nodes. Nearly all data centers that require Fibre Channel
communication between more than two devices use the switched fabric topology. It is very
scalable, with some data centers running fabrics with more than 4000 ports.
This lesson covers Fibre Channel node addressing and World Wide Names.
A Fabric is a physical or virtual space in which all authorized storage nodes may communicate
with each other. It can be created with a single switch or a group of switches connected together.
The primary function of the fabric is to receive data frames from source N_Ports and forward
them to the destination N_Ports. Each frame has a destination address so the fabric knows where
to send it. Each frame also has a source (or return address) so the receiver knows where to send
a response.
How do end nodes get their Fibre Channel addresses? That’s the subject of the next few slides.
Each N_Port must be assigned a 24-bit address, called an FCID (Fibre Channel ID), before it is
allowed to communicate on a Fibre Channel network. With a switched fabric, the switch itself
automatically assigns an address to the N_Port. The switch will assign an FCID that is unique in
the fabric. This address is assigned when the N_Port logs into the fabric.
The address is specified as six hexadecimal digits and is divided into three fields: Domain_ID,
Area_ID, and Port_ID.
Each switch is assigned a unique Domain_ID between 1 and 239. The switch Domain_ID becomes
the first byte of the address for every N_Port attached to that switch.
The Area_ID and Port_ID bytes are assigned in different ways by different switch vendors. One of
the reasons there are incompatibilities between different switch vendors is because they handle
address assignment differently. This is why you will normally find a datacentre has Fibre Channel
switches from only one vendor.
Each switch runs fabric services at the FC-2 level to manage the fabric environment. Each fabric
service is assigned a standard well known address. Three of them are shown here.
The F_Port server login service is assigned well known address FFFFFE. This is the address
that an end node connects to when performing a FLOGI operation. Each end node must have an
FCID to operate. Each end node sends a FLOGI frame to address FFFFFE and receives a response
that includes the FCID the node will use for all other Fibre Channel communications.
The node must then login and register with the fabric Name Service. A port login (PLOGI)
operation is used to do this. When a node sends a PLOGI frame to the fabric name server, it uses
destination ID FFFFFC.
The Name Service is used to store information about all devices attached to the fabric. After
performing a PLOGI to the name service, each node registers its identifying information and
capabilities. The name service stores all these entries in a local database on each switch and
distributes the information to other switches in the fabric. Devices can query the name service to
find other devices logged into the fabric.
The Fabric Controller provides state change notifications to all registered nodes in the fabric. A
state change is when a link in the fabric transitions from up to down, or down to up. Hosts require
notification when storage targets have link state changes. A node registers for state change
notifications by sending an SCR (State Change Registration) frame to address FFFFFD.
Each Fibre Channel Port has a unique World Wide Name (WWN). Each is a 64-bit address used in
Fibre Channel networks to uniquely identify each element in the network. The name is assigned to
a host bus adapter or switch port by the vendor at the time of manufacture. It is similar to the
MAC address in an Ethernet network.
There are two designations of WWN – World Wide Port Name or (WWPN) and World Wide
Node Name (WWNN). Both are globally unique 64-bit identifiers. The difference lies in where
each value is ‘physically’ assigned. For example, a server may have dual port HBAs installed. Each
port would receive a unique WWPN, and may share a WWNN.
Shown are three examples of World Wide Port Names (WWPN or PWWN) for different products
that may connect to a Fibre Channel SAN.
This lesson covers Fibre Channel zoning, login and name services.
Zoning is a Fibre Channel fabric function that allows devices attached to the fabric to be logically
separated into groups. Each group, or zone, facilitates communication between devices assigned
to the same zone. Devices are not allowed to communicate with devices in other zones. A zoning
database contains one or more individual zones. The database is distributed to all switches in the
fabric. Each zone will have one initiator and one or more target devices.
When a frame arrives at a switch port the destination address is read and compared to the zone
ACL for the port. If the destination is allowed by the ACL, the frame is forwarded. Otherwise the
frame is dropped.
A collection of zones that can be activated throughout the fabric is called a “Zone Configuration”
or a “Zone Set”.
Each device that connects to a Fibre Channel switch goes through a process to log into the fabric
and register with the fabric name server. Once logged in and registered, each host device queries
the name server to discover storage target devices.
As an example: Initiator A logs into the fabric and queries the name server about logged in
devices that it may communicate with.
The name server checks the logged in devices, and determines which are zoned to talk to Initiator
A. The zoning function controls this process by permitting only ports in the same zone to be
discovered. In our example, Target A is returned to the initiator as an available logged in device.
Zoning prevents unauthorized devices from communicating with other devices. If Initiator A tries
to send a frame to Target B, the frame is dropped at the ingress port of the switch and is not
forwarded because Target B is not in the same zone as Initiator A.
There are two ways to define the members of a zone.
One way is by using World Wide Port Names (WWPN or PWWN). This type of zoning is sometimes
called soft zoning. In this type of zoning, the administrator controls connectivity by putting the
WWNs of devices that are allowed to communicate into a common Zone. WWNs defined as part of
a zone ‘see’ each other regardless of the switch port they are plugged into. With this type of
zoning, if a host HBA is replaced, the zone must be modified with the WWN of the replacement
HBA. This type of zoning is recommended by EMC.
The other way to define zone members is by using the Domain_ID and port number. This type of
zoning is often called port-based zoning, or hard zoning. Each port is only allowed to see ports
that are in the same zone. If a cable is moved to a different port, the zone has to be modified
before communication can take place. Some administrators like this type of zoning because if
they are changing host and HBAs, they don’t have to change the zoning because of WWN
changes. All they need to do is plug the correct devices into the correct ports.
With both types of zones, zone members can be part of multiple zones. Port zoning is not
recommended because any device regardless of its WWN can be granted access to unauthorized
devices simply by plugging it into a zoned port on the switch.
– A variation of these two types of zones is the hybrid zone which defines some members
by WWN and others by their domain and port.
– Since WWNs and FCIDs are not very user friendly, human readable names called
aliases may be defined to represent the zone members.
Now that we know something about Fibre Channel addresses, WWNs, zoning, and fabric services,
let’s put everything together and see how an HBA port on a host logs into the fabric and discovers
target LUNs on fabric attached storage arrays.
The steps that a Fibre Channel N_Port must go through are listed here. The first step is to bring
the link between the N_Port and the switch F_Port online. This is done by exchanging several
primitive sequences until both ports are synchronized and sending Idle sequences to one another.
After the link comes online, the N_Port needs an address. So it sends a Fabric Login (FLOGI)
frame to the well known address FFFFFE of the F_Port Server. The F_Port Server responds with a
link services Accept frame which includes the new FCID that is being assigned to the N_Port. In
this example, the FCID will begin with 03, because the HBA is connected to a switch with
Domain_ID 3.
Once our HBA N_Port has a Fibre Channel address, it can login to the Fabric Name Server. Each
switch in the fabric has a copy of the name server database, and when new N_Ports are
registered, that information is passed to all of the other switches in the fabric.
When logged into the name server, our HBA port will register important information about itself
by transmitting several Fibre Channel services frames. These frames will specify which Fibre
Channel classes of service are supported by the HBA, the FC-4 protocols supported, and its mode
of operation (N_Port). HBAs from different vendors, and storage device ports, may register
additional information such as WWNs and symbolic node names that include text that describes
the manufacturer.
Once the HBA has sent all the frames with information it wants registered with the name server, it
then begins a series of queries. This step allows the HBA to find out what storage devices are
registered with the name server. Each query is sent with a Fibre Channel services frame and
responded to with an Accept frame.
The first query asks the name server for a list of FCIDs that have registered as SCSI FCP devices.
Once it gets this list of FCIDs, the HBA will ask the name server for relevant information about
each registered FCID. For example, it will get port WWNs associated with each FCID.
Now, this is where zoning is important. The name server will only return information about
registered devices that are in the same zone as the HBA port. So if there are ten storage array
ports registered in the fabric, but only one of them is in the same zone as the HBA port, then the
HBA will only learn about that one storage device.
When the HBA is done asking the name server for information, it will log out of the name server.
In step 6, the HBA will register for state change notifications. Some HBA vendors may actually
perform this registration earlier in the initialization process.
A state change registration (SCR) frame will be sent to well known address FFFFFD of the Fabric
Controller, and an Accept frame will be returned in response.
The reason an HBA registers for state change notifications is that it wants to be notified when a
device that is in the same zone has a link come online, or a link goes offline. The HBA is
responsible for querying the name server, when it receives a registered state change notification,
to check for changes in target device status.
Once the HBA has a list of storage device port IDs, it will login to those ports. So in our example,
the HBA discovered that FCID 040600 was logged in with the name server, so it sends a PLOGI
(port login) Extended Link Services frame to that address. The storage array port responds with
an Accept frame.
Once the HBA port has logged in to the storage array port at a Fibre Channel level, it must then
login at a SCSI process level. Since the HBA wants to exchange SCSI data over Fibre Channel, it
must perform a Process Login by sending a PRLI (Process Login) Extended Link Services frame to
the storage array port. The storage array port responds with an Accept frame.
Information carried in these frames specifies which port is the SCSI Initiator and which port is the
SCSI target, and whether to use XFER_RDY on read operations (usually XFER_RDY is only used
for write operations).
Finally, after all of the previous steps have been completed, the HBA can start sending frames
with SCSI information. Usually, the first SCSI frame sent will be a SCSI INQ command to get
information about the SCSI LUNs on the target device.
This lesson covers Fiber Channel Deployment scenarios including dual fabrics, core edge topology,
SAN Routing and SAN Virtualization.
The highest SAN availability comes when fabrics are mirrored.
Each topology we discuss in this course can (and should) be mirrored to achieve the highest
availability. EMC recommends mirrored fabrics be identical for easier management and
monitoring. However, an advantage of mirroring is that one side of the mirrored fabric can be
brought down for maintenance or upgrades, or to change switch vendors, while the other side
continues production operations.
A fabric with a Core-Edge topology is fairly simple to design and implement. There are two
variations: two-tier or three-tier.
In a two-tier Core-Edge topology (as shown here), all hosts are connected to edge switches and
all storage is connected to the core switch.
With a Core-Edge three-tier topology, there are two edge tiers connected to a central core. All
hosts are connected to one edge, and all storage is connected to the other edge. The core tier
would then only be used for ISLs.
The Edge tier, usually small low cost departmental switches, offers an inexpensive approach to
add more hosts into the fabric.
The Core or backbone tier usually consist of enterprise directors, which are higher cost and have
higher availability. For the highest availability, we should have redundant edge switches and core
directors.
There are two types of mesh topologies used to build a SAN: full mesh and partial mesh. A full
mesh topology (shown here) has each switch connected to every other switch. In a partial mesh
topology, each switch is connected to several other switches, but not to every switch. A partial
mesh topology is more practical to build when there are a large number of switches.
Full mesh topology provides maximum availability. However, this is done at the expense of
connectivity, which can become prohibitively expensive with an increasing number of switches.
Compound Core-Edge topology is a combination of the full mesh and core-edge three-tier
topologies.
Routing between fabrics is done by a Fibre Channel switch with SAN routing enabled. Fibre
Channel switches that perform a routing service do not allow the two fabrics to merge. Special
vendor-dependent zone types will allow certain devices in one fabric to communicate with devices
in another fabric. The SAN administrator will configure which devices are allowed to communicate
across fabric boundaries.
NPIV (N_Port ID Virtualization) is a T11 standard that provides a means to assign multiple FCIDs
to a single N_Port. This feature allows multiple VMs sharing an N_Port to use different FCIDs and
allows access control, zoning, and port security to be implemented at the application level.
You must globally enable NPIV for the switch to allow the NPIV-enabled applications to use
multiple N_Port identifiers. Only NPIV capable HBAs support this feature. As each VM powers up,
it creates a VPORT (Virtual Port) on the HBA. Each VPORT has its own WWNN and unique WWPN.
With NPIV, the first time the physical HBA port logs into the fabric, it does it in the normal way,
with a FLOGI frame, and receives an FCID from the F_Port server. Subsequent initialization logins
by VMs using the same HBA port will use an FDISC (Fabric Discovery) login frame instead of a
FLOGI to get an FCID assignment.
Fibre Channel switches can be virtualized. This means that a switch or chassis may be partitioned
into multiple logical switches, each belonging to a different fabric.
As an example, we have taken sixteen ports from the physical switch on the left, and created
three logical switches in three separate fabrics on the right. Each fabric is logically isolated from
the other fabrics. The logical switches in these fabrics will each have its own Domain_ID, and can
use its ports to attach to Fibre Channel devices or to other physical switches (and their logical
switches if they have them).
An administrator creates a logical switch and assigns it physical ports and a fabric or virtual SAN
identifier. Physical ports can only be part of one logical switch at a time.
This feature allows us to take a single physical fabric infrastructure and virtualize it, so that we
can segregate functions, applications, or groups within an organization.
This module covered Fibre Channel architectural layers, switched fabric topology, addressing,
login and name services, and deployment options.
This module covers basic iSCSI protocol concepts including interaction with other network layers,
addressing, operation and deployment.
This lesson covers the iSCSI layered architecture, frame format, and host hardware.
The Internet Small Computer Systems Interface protocol provides a means of transporting SCSI
packets over TCP/IP. iSCSI works by encapsulating SCSI commands into TCP segments and
transporting them over an IP network. iSCSI can be routed or switched on standard Ethernet and
TCP/IP equipment.
iSCSI is a technology that is used as an alternative to Fibre Channel to create Storage Area
Networks. It first gained popularity when attaching entry level servers to shared block storage
arrays because of its perceived lower costs and ease of management for those shops that had no
expertise in Fibre Channel. Since iSCSI uses the familiar IP protocol that is used for most other
networking in the data center, it requires little training for the IT staff.
Best practice is to create a separate network for iSCSI traffic. This is because storage traffic can
quickly impact performance for other traffic if sharing the same network.
iSCSI solutions work with direct attached storage, or the host can connect to storage using
standard TCP/IP network equipment as represented by the network cloud shown here.
The iSCSI protocol, like other protocols, is influenced by the OSI 7-Layer Reference model. iSCSI
sits on top of the standard TCP/IP/Ethernet protocol stack and takes SCSI commands, data, and
responses and encapsulates them into TCP segments for transportation. When receiving iSCSI
TCP segments, the iSCSI layer removes the SCSI information and passes it to the SCSI driver
software.
Within iSCSI, a Node is defined as a single Initiator or Target. These definitions map to the
traditional SCSI Initiator/Target/LUN (or ITL) model. iSCSI Names are assigned to all Nodes and
are independent of their associated IP address.
The great thing about iSCSI is that it doesn’t require any special hardware for your host. Most
major operating systems have iSCSI software drivers that can utilize standard network interface
cards or NICs. These should be at least 1 Gbps capable (which are common on most high end
servers).
For higher performance, 10 Gbps NICs are recommended. But you should be aware that because
of inefficiencies in TCP/IP protocol, an 8 Gbps FC link has better performance than a 10 Gbps
iSCSI connection. One drawback to using standard NICs is that host system performance could be
impacted.
There are two types of hardware adapters that can be installed in a server to improve
performance. One hardware solution is to install a specialized NIC that includes a TCP/IP offload
engine - called a TOE card for short. This card will offload the iSCSI TCP/IP operations from the
host processors and free them up for other workloads. This solution still requires host processing
for the iSCSI protocol layer.
An iSCSI HBA is also available. This card combines a NIC, TOE, and iSCSI bus adapter. The iSCSI
HBA offloads TCP/IP and iSCSI and SCSI processing. This card may include PCI option ROM to
allow booting from an iSCSI SAN device.
An iSCSI PDU contains SCSI data and the iSCSI header, which is created by the iSCSI initiator
and is then encapsulated or wrapped in other protocol layers to facilitate its transport. The iSCSI
header contains control information, and may optionally contain a SCSI command.
The Ethernet Header is used to provide addressing for the physical layer while the IP Header
provides packet routing information used for moving the information across the network (from
one subnet to another), and the TCP Header contains the information needed to guarantee
delivery to the target destination. The iSCSI Header defines how to extract SCSI commands and
data.
As in most networking protocols, the data is broken up and encapsulated within packets for
transmission. In iSCSI, the SCSI commands are issued and broken into iSCSI PDUs. These in turn
are broken into IP packets and transmitted across the physical network. The receiving node must
then reassemble the data and pass it up the network stack until a SCSI command is extracted. As
can be seen here, alignment of iSCSI PDUs with IP packets varies.
There are two major network components associated with iSCSI.
The first is the Network Entity which follows the client server model. Hosts act as network
clients using iSCSI initiators to initiate I/O operations. Storage devices act as network servers and
receive read and write commands as iSCSI targets.
The second network component is the Network Portal.
The Network Entity must have one or more Network Portals available for use. Administrators
assign an IP address to each network portal. TCP listening port numbers for host ports are usually
assigned by the operating system. The target always listens on TCP port 3260.
iSCSI supports multiple connections within the same session. Some implementations have the
ability to combine connections in a session across multiple Network Portals.
A Portal Group defines a set of Network Portals within an iSCSI Node that collectively supports
the capability of coordinating a session with connections that span these portals.
Portal Groups are identified within an iSCSI Node by a portal group tag, an integer value
between 0 and 65,535. Both iSCSI Initiators and iSCSI Targets have portal groups, though only
the iSCSI Target Portal Groups are used directly in the iSCSI protocol.
This lesson covers how iSCSI devices are addressed.
iSCSI Names enable iSCSI storage resources to be managed regardless of the IP address. An
iSCSI node name is also the SCSI device name of an iSCSI device. The iSCSI name is the
principal object used in authentication between initiators and targets. iSCSI names are associated
with iSCSI nodes and not with iSCSI network adapter cards. So, the replacement of a NIC does
not change the name. iSCSI names should be worldwide unique.
There are two iSCSI naming formats as shown: EUI names and IQN names. IQN names are the
most popular. They include the reverse DNS fully qualified name of the organization, the year the
name was assigned, and a unique identifying string.
iSCSI addresses define the path between iSCSI nodes. To enable iSCSI I/O operations, there
must be connectivity at the lower network layers including the TCP/IP layers of the network
protocol stack. It is at this layer that TCP/IP addresses are used.
Then an iSCSI session must be initiated. iSCSI sessions typically use iSCSI iqn addresses to
establish a connection between iSCSI client and iSCSI server at the upper layer of the network
protocol stack.
Not only does an initiator need to know the IP address of the iSCSI target; it also needs the TCP
port number of the target. By default, iSCSI targets listen on port 3260. If there is a network
connectivity issue between an iSCSI initiator and target, check that the iSCSI port is open on the
network firewall.
Initiators use various TCP port numbers (one for each open iSCSI session). These are not
advertised because initiators do not listen for connections.
This lesson covers how iSCSI initiators login with iSCSI targets, and how iSCSI devices are
discovered.
Before sending any SCSI commands, the iSCSI initiator must login to the iSCSI target.
The Login Phase starts with a login request from the iSCSI initiator to the iSCSI target. A login
may create a new session or it may add a connection to an existing session between a given
initiator and target. Each session has a session ID associated with it.
Frames are passed between the two that negotiate session parameters like iSCSI names,
negotiation state, security keys and so forth. Once all login negotiations have completed, then
both initiator and target go to Full Feature phase.
It is only in Full Feature phase that SCSI commands can be sent and data transferred between
the hosts and the various LUNs on the target.
All SCSI commands carried over iSCSI will operate the same as SCSI commands carried over
Fibre Channel. The SCSI commands and protocol do not change when the underlying
transportation protocol changes.
Before a host can login to a storage device and establish an iSCSI session, it must know which
devices to log into. How does an iSCSI host discover iSCSI storage devices?
iSCSI Discovery is the process by which the iSCSI initiator finds iSCSI targets. An initiator uses
one of three methods for the discovery: Manual Configuration, using the SendTargets command,
or by making use of an iSNS (internet Storage Name Service).
The ability to define a target manually places the control of the discovery process into the
administrator’s hands. This might work OK in a small installation. However, it is the most labor
intensive method.
The Administrator must go into each target and record the node name, along with the IP Address
and port number. Then the admin must go into the iSCSI driver on each initiator and manually
enter each target’s addressing information. Node names can be up to 255 characters long, so
there is a lot of room for typing errors in this process.
The SendTargets discovery method is a semi-automated method to discover iSCSI targets.
Usually the administrator will configure the iSCSI initiator to discover an IP address or range of IP
addresses.
For each IP address within the specified range, the initiator will begin a discovery session.
If the discovery session is successful, it will issue a SendTargets=All command. The target will
return its iSCSI node name, IP address and port number, and any other target nodes it knows
about with any port grouping information.
In the example shown, we see the SendTargets Discovery process between an initiator and a
target – the storage array. Starting this session looks like a normal login process. The difference
is the SessionType=Discovery. With a discovery session type, the initiator does not need to
know the target’s node name (that’s usually the purpose of doing a discovery). Once a successful
acknowledgment is received, the initiator issues the SendTargets command. The target replies
with the target’s information.
The most scalable solution for discovering iSCSI targets in large applications uses an Internet
Storage Name Service (iSNS). iSNS follows a client server model. The iSNS clients (including
iSCSI initiators, targets, and management stations) register with the iSNS server. The iSNS
server may send state change notifications to registered clients.
A client, such as an iSCSI initiator, can query the iSNS server to find iSCSI targets with which it is
allowed to communicate. The initiator may find the address of the iSNS server from DHCP, once
an administrator sets this up.
The iSNS server is configured with discovery domains to limit discovery to targets within the
same domain. Discovery domains are similar to zones in the Fibre Channel protocol. They are a
security and management mechanism used to partition storage resources. Discovery Domains
limit the discovery process to the administrator-configured subset of relevant storage devices.
This prevents initiators from attempting to login into devices that they should not have access to.
This module covered basic iSCSI protocol concepts including interaction with other network
layers, addressing, operation and deployment.
This module covers how to extend the distance of a Storage Area Network and focuses on FCIP
protocol.
This lesson covers alternatives to FCIP for long distance network technologies including CWDM,
DWDM, and SONET/SDH.
Fibre Channel started out as a way to connect servers to more storage. Typically the storage was
right next to the server and there was no need to cover any great distance. But data centers
evolve, and so has the need for long distance between SAN storage devices. One need is data
replication for disaster recovery operations. Disaster recovery sites may be located hundreds of
miles away to be outside of a potential disaster zone. Another reason for long distance Fibre
Channel connections is that as a business grows and consolidates with other businesses, SAN
islands develop.
What are the best options for connecting block storage devices over long distances?
Native Fibre Channel supports distances up to 10 km. Some types of Fibre Channel Transceivers
can go up to 40 km. But what if we are replicating storage for a disaster recovery solution from
one city to another city that is 100 km away. What are the options?
One option is DWDM or CWDM, which can extend the reach of Fibre Channel to 200 kilometers
and beyond. This is an ideal solution for transporting SAN data across a metro area.
For longer distances, SONET is a good solution, and is offered by many service providers.
Also, a company could utilize unused bandwidth on its long distance IP connections and
implement a FCIP (Fibre Channel over IP) solution.
The purpose of this lesson is to look at some of these options. Then, in the following lessons, we
will discuss the details of the FCIP solution.
Dense Wavelength Division Multiplexing (DWDM) is a process in which different channels of data
are carried at different wavelengths over one pair of fiber-optic links. This is in contrast with a
conventional fiber-optic system in which just one channel is carried over a single wavelength
traveling through a single fiber.
Using DWDM, several separate wavelengths (or channels) of data can be multiplexed into a
multicolored light stream transmitted on a single optical fiber. Service providers will lease unused
long distance fiber optic cables (known as dark fiber) that can be connected to DWDM
equipment. Current DWDM technology can multiplex over 100 different signals for transmission
down a single fiber optic cable.
Available DWDM solutions include both point-to-point and ring topologies.
Coarse Wave Division Multiplexing (CWDM), like DWDM, uses similar processes of multiplexing
and de-multiplexing different channels by assigning different wavelengths to each channel. CWDM
is intended to consolidate environments containing a low number of channels (less than 16) at a
reduced cost. DWDM is much more expensive, but can consolidate many more channels.
Synchronous Optical NETwork, (SONET), and Synchronous Digital Hierarchy (SDH) are essentially
the same standard. SONET is used in the United States and Canada, While SDH is used in the rest
of the world. These standards define a technology for carrying different capacity signals through a
synchronous optical network. The standard defines a byte-interleaved multiplexed transport
occupying the physical layer of the OSI model.
SONET is useful in a SAN for consolidating multiple channels into a single higher-speed
connection. This can reduce DWDM wavelength requirements in an existing SAN infrastructure. It
can also allow a distance solution to be provided from any SONET service carrier, saving the
expense of running private optical cable over long distances.
The basic SONET building block is an STS-1 (Synchronous Transport Signal) carried on an OC-1
(optical carrier) signal.
An STS-1 operates at 51.84 Mb/s, so multiple STS-1s are required to provide the necessary
bandwidth for Fibre Channel, and high speed Ethernet, as shown in the table. Multiply the rate by
95% to obtain the usable bandwidth in an STS-1 (reduction due to overhead bytes). One OC-192
can carry approximately four Gigabit Ethernet data streams plus one 4Gbps Fibre Channel data
stream.
This lesson covers FCIP concepts and benefits when used in a SAN environment, layered
architecture, addressing, and topology options.
Sometimes the answer for a distance extension problem is already in the data center. Consider
the existing local or wide area network. If we already have a long distance TCP/IP network that
has extra capacity, why not use it to carry Fibre Channel data for the SAN. That is the premise
behind the FCIP (Fibre Channel over IP) protocol.
As we have studied before, TCP provides a reliable transport, and if the network infrastructure
with the needed capacity is already in place, FCIP may be the ideal solution for SAN distance
extension.
FCIP is a manageable, cost-effective way to blend the best of both Fibre Channel block data I/O
and proven, widely deployed IP infrastructure. Extended distance is not a problem with IP
infrastructure, and adding FCIP is transparent to existing Fibre Channel deployments.
Note that some data storage, replication, and backup applications require large amounts of
bandwidth. Therefore, IP networks should be provisioned for the amount of bandwidth that is
required to support those applications.
FCIP extended distance solutions for SANs rely on multi-protocol switches (sometimes called
FCIP gateways) to translate between Fibre Channel and FCIP.
When data is received at the Fibre Channel port on a switch, the FC-0 through FC-2 layers
process it and extract the Fibre Channel frame just as they normally would for any Fibre Channel
frame. When the destination of the frame is discovered to be a switch at the other end of the FCIP
link, the frame is processed through the FCIP layer of the protocol stack and then passed down to
the TCP/IP layers and encapsulated in an Ethernet frame, where it is sent out the physical GigE
port.
Since each Fibre Channel frame is encapsulated and sent without modification through the TCP/IP
network, the Fibre Channel protocol is said to “tunnel” through the WAN.
The incoming Ethernet frame is processed through the TCP/IP layers and the original Fibre
Channel frame is passed on through the FC layers to the next Fibre Channel port.
The unmodified Fibre Channel frame complete with Start of Frame (SOF) and End of Frame (EOF)
delimiters, Header and CRC is encapsulated in the FCIP frame. Only the FCIP gateway is aware of
frame encapsulation. The Fibre Channel part of the SAN only sees Fibre Channel, while the
LAN/WAN only sees TCP/IP traffic. The frame encapsulation is transparent to end devices.
Multi-Protocol FCIP capable switches have both Fibre Channel Ports and Gigabit Ethernet ports.
Virtual E_Ports (VE_Ports) are created that emulate standard E_Ports for the purpose of creating
an ISL connecting two switches across the WAN. The VE_Ports use the underlying TCP/IP and
gigabit Ethernet technology for FCIP Frame transport.
FCIP frames are routed across the WAN following normal IP routing procedures.
When configuring a switch for FCIP, the FCIP initiator must be configured with both the local
TCP/IP address and the TCP/IP address of any FCIP targets.
Virtual E_Ports communicate end-to-end between Fibre Channel switches, and function just like
interconnected E_Ports. The normal result of implementing VE_Ports at each end of an FCIP
tunnel, is to connect two SAN islands and create a fully merged Fibre Channel fabric.
However we also have the option of routing between fabrics without allowing them to merge into
a single fabric.
One way this is done is with VEx_Ports (or routed VE_Ports). Having a VEx_Port at one end of
the link is like installing a Fibre Channel router. The VEx_Port will pass control traffic across the
link, but will not let the fabrics merge. Data traffic will be limited by special routing zones setup
by the administrator.
An example use of FCIP routing would be to allow long distance data replication traffic between
two remote storage arrays, but keep all other data traffic local to each SAN.
We can take advantage of the fact that IP sessions may carry multiple TCP connections on a link
carrying encapsulated Fibre Channel frames. For example, some vendors automatically create two
TCP connections for each FCIP link. One TCP connection is used for data frames. The second
connection is used only for Fibre Channel control frames (such as switch-to-switch protocol
frames). This arrangement is used to provide low latency for the control frames.
Multiple VE_Ports may share a physical GbE interface. Each VE_Port would have a unique TCP/IP
address and would point to one remote VE_Port. Separate FCIP tunnels would be created for each
VE_Port interface.
This module covered popular choices used by data centers to extend the distance of a Storage
Area Network. Particular attention was given to FCIP solutions implemented by multi-protocol SAN
switches.
This module covers basic concepts for FCoE (or Fibre Channel over Ethernet) protocol, including
enabling technologies, theory of operation, and example applications.
This lesson covers the benefits of FCoE and describes how FCoE fits into the layered network
model.
Fibre Channel over Ethernet (FCoE) is a technology protocol defined by the T11 standards
committee. It expands Fibre Channel into the Ethernet environment. At a physical level, servers
are equipped with Converged Network Adapters (CNA) that carry both Ethernet and Fibre Channel
traffic. Multi-Protocol switches keep the protocols separate and forward frames to the correct
destinations.
Without FCoE, a host server may connect to different networks using different hardware for each.
For example, a host may connect to storage through Fibre Channel, other applications through
TCP/IP and other high performance computers through InfiniBand. This solution results in
connections to three different networks, using three different types of network adapters, cables,
skill sets, and management tools.
Many IT companies, including EMC, got together to solve this problem. Various international
standards committees participated, such as the INCITS/ANSI T11 committee for Fibre Channel
standards, and the IEEE committees that developed enhanced Ethernet standards to support data
center bridging. The result is the Fibre Channel over Ethernet (FCoE) solution.
With FCoE, we can merge different types of networks into a single “converged” network. This
results in less data center cabling, and fewer host adapters and networking switches.
Management is also centralized and simplified. All of this means there is less cost and better
utilization of resources.
The first thing we notice when comparing FCoE to other protocol stacks is efficiency. Here we see
that FCoE protocol eliminates two protocol layers when compared to Fibre Channel over IP
protocol.
FCoE is a protocol that supports the direct mapping of Fibre Channel over Ethernet. The problem
with this is that Fibre Channel expects a lossless underlying network layer, and standard Ethernet
is not lossless. FCIP works because TCP acts as an underlying lossless network.
Generic Ethernet networks may lose frames due to congestion and buffer overflow. FCoE requires
a new kind of Ethernet called Enhanced Ethernet. We will talk more about this later.
The FCoE protocol encapsulates the complete Fibre Channel frame with SOF (Start of Frame) and
EOF (End of Frame) sequences directly into Ethernet frames without any TCP/IP overhead. The
simple encapsulation of Fibre Channel into Ethernet frames results in great benefits, such as
greater efficiency and performance, due to less overhead.
As previously mentioned, because Fibre Channel is considered a lossless protocol, meaning frame
delivery is never expected to fail, Ethernet must be enhanced to also be a lossless protocol.
Ethernet switches that carry FCoE traffic must support Enhanced Ethernet protocols such as
DCBX (Data Center Bridging eXchange), PFC (Priority Based Flow Control), Congestion
Notification, and ETS (Enhanced Transmission Selection).
FCoE utilizes virtual ports such as VN_Ports, VF_Ports, and VE_Ports, which emulate the
behavior of regular Fibre Channel N_Ports, F_Ports and E_Ports. The only difference is the
physical link, since Fibre Channel is encapsulated over Ethernet.
A VN_Port is identified in the FCoE standard as part of an ENode (FCoE End Node). The ENode
contains a single Fibre Channel node with one or more FCoE entities, one per Ethernet port. Each
FCoE entity has an Ethernet MAC address that is used as a source or destination address when
FCoE frames are transmitted through the Ethernet network.
When the ENode first comes online, the Fibre Channel protocol will attempt a FLOGI or FDISC.
This process creates a Virtual N_Port (VN_Port), which receives an FCID from the fabric.
The FCoE Link End Point is the entity that handles translation between the Fibre Channel protocol
and the Ethernet protocol. It encapsulates Fibre Channel Frames into Ethernet frames and adds
the FCoE header. It also retrieves the Fibre Channel frames from the received Ethernet frames
and gives them to the VN_Port.
An FCoE capable switch is similar to an ENode. The major difference being that it has two
different switching elements. On one side of the FCoE switch is the Ethernet bridge (switch), and
on the other side is a standard Fibre channel switch, known as a Fibre Channel Forwarder (FCF) in
the FCoE environment. An FCoE switch contains either Virtual E_Ports, or Virtual F_Ports,
depending on the type of connection being made at the Ethernet port. If the external Ethernet
port is connected to an ENode, then a VF_Port is instantiated. If the Ethernet port is attached to
another FCoE switch, then a VE_Port is created.
The E_Ports and F_Ports connect to other Fibre Channel switches and devices.
The FCoE functional model calls for the ENode to communicate with the FCF (Fibre Channel
Forwarder).
From the ENode, the data is passed from the application to Fibre Channel for frame construction.
It is then passed to the VN_port. From there it goes to the FCoE_LEP where the Fibre Channel
frame is encapsulated into an Ethernet frame. The Ethernet frame is passed to the MAC and
transmitted out the Ethernet port to the network. The frame will be switched through the
Ethernet layer 2 network like any other Ethernet frame. When it arrives at the switch, the frame
will be passed to the MAC and then to the FCoE_LEP for the port. The Ethernet header and FCoE
header are stripped away and the Fibre Channel frame is passed to the VF_Port, where it is sent
to the FCF to be forwarded to a Fibre Channel switch or device.
Virtual links are the connections between different points within the fabric. In native Fibre
Channel, a link is not virtual, but is the physical cable connecting a switch to a host HBA, resulting
in a Point-to-Point connection. FCoE provides virtual links, which mean the same physical port can
have multiple virtual connections.
FCoE Link End Points are located between the MAC and the virtual ports. They are in charge of
Fibre Channel Frame encapsulation and reverse encapsulation, and of transmitting and receiving
the encapsulated frames through a single virtual port.
FCoE Link End Points have virtual ports associated with them. Notice the different virtual ports in
the diagram. End ports on the hosts are VN_Ports, fabric ports are VF_Ports, and ports for
switch ISL are VE_Ports.
In our example, the lower host has two virtual N_Ports. One connects to the upper switch, and
the other connects to the lower switch as shown.
This lesson covers FCoE enabling technologies including hardware and lossless Ethernet.
FCoE capable hosts require CNAs (Converged Network Adapters). This is where ENodes are
implemented. A CNA combines the functions of both a 10 Gigabit per second Ethernet NIC and a
Fibre Channel HBA, and allows the two protocols to be converged onto a single cable. FCoE
protocol overhead is handled by the CNA hardware so no processing overhead is added to the
host.
What does the CNA card look like to the host operating system? It looks like an Ethernet NIC and
a separate Fibre Channel HBA. The host operating system does not know about virtual N_Ports
and FCoE Link End Points. An administrator installs network and Fibre Channel drivers the same
as if the CNA were a stand-alone NIC and HBA. HBA management tools also stay the same.
If an I/O request is a network transaction, it is delivered to the NIC where it is forwarded to the
Lossless Ethernet MAC.
In the case of Fibre Channel I/O requests, they are sent to the HBA part of the CNA where the
frames are encapsulated into FCoE by the FCoE encapsulation engine. Then they are sent to the
lossless Ethernet MAC for delivery.
Received traffic is processed by the lossless MAC. Ethernet frames with type=FCoE are sent to
the FCoE side of the CNA, and other frames are sent to the NIC side.
The Fibre Channel Forwarder, or FCF, is an FCoE capable switch, or a blade that is part of a
modular Fibre Channel switch. The purpose of the FCF is to service FCoE login requests and
provide the services typically associated with a Fibre Channel switch. FCFs may also optionally
provide the means to:
• De-encapsulate Fibre Channel frames that are coming from a host CNA and forward them to
storage devices in the SAN,
• Encapsulate Fibre Channel frames that are coming from storage devices in the SAN and
forward them to a host CNA.
Ethernet is a “best-effort” protocol that is subject to dropped frames during times of network
congestion. This is not suitable for Fibre Channel, which depends on a lossless lower layer
protocol. To get the performance and reliability required by the Fibre Channel protocol, FCoE
requires Enhanced Ethernet. Enhanced Ethernet is actually a collection of standards that fall
under the term Converged Enhanced Ethernet (CEE) also called Data Center Bridging (DCB).
Data Center Bridging includes:

• Priority-based Flow Control (PFC)
• Enhanced Transmission Selection (ETS)
• Transparent Interconnections of Lots of Links (TRILL)
• DCB Capability Exchange (DCBX)
PFC improves Ethernet reliability by creating a Lossless Ethernet connection for FCoE traffic. By
default, Ethernet is designed to drop packets when a node cannot sustain the pace of the
incoming traffic. Packet drops make Ethernet very flexible in managing random traffic patterns
injected into the network, but they effectively make Ethernet unreliable.
Using a pause frame to stop transmission, before receive buffers overflow and drop frames,
makes a link more reliable. The problem with the standard pause mechanism is that it stops
everything on the link. What PFC does is offer a point-to-point flow control mechanism for
Ethernet traffic based on the IEEE 802.1Q priority field in the Ethernet frame header. Priority Flow
Control gives FCoE no-drop reliable behavior, while allowing other classes to retain traditional
best-effort Ethernet service.
The flow control function is used by the receiving node to signal its peer when no more frames
can reliably be accepted. In the example, notice the PAUSE is sent from the receiving node. Also
notice there are eight virtual lanes, all with their assigned priority. FCoE is assigned priority=3.
Fibre Channel Frames with headers, and other overhead bytes have a maximum size of 2148
bytes. Standard Ethernet frame payloads have a maximum Transmission Unit (MTU) size of 1500
bytes. This means if we were to use Ethernet frames with the standard MTU size, two frames
would be needed to transmit each FCoE frame. This extra frame overhead is not efficient.
For this reason, FCoE requires Ethernet jumbo frames with an MTU size of 2240 bytes. Using
jumbo frames with FCoE gives a one-to-one relationship between a Fibre Channel frame and the
Ethernet frame that encapsulates it, and frame fragmentation is eliminated. Multiple FCoE frames
are never consolidated into single Ethernet frames, so using frames larger than 2240 bytes
provides no added benefit.
This lesson covers FCoE addressing and logins.
When working with FCoE, we need to be aware of both Ethernet and Fibre Channel addresses. For
a CNA to build an I/O frame to send to a storage device, it must have the Ethernet MAC address
of the neighboring Ethernet port (in our example, that is the port on the FCoE switch. The CNA
must also know the Fibre Channel ID (FCID) assigned to the storage array port.
As an example, the CNA first builds a Fibre Channel frame which includes the Fibre Channel
Destination ID and Source ID (D_ID, S_ID). The Fibre Channel frame is then encapsulated into
an Ethernet frame with a header that includes the destination and source Ethernet MAC
addresses. The Ethernet frame is sent to the MAC destination address (MAC B) of the FCoE
switch Ethernet port. The FCoE switch strips off the Ethernet frame header to retrieve the Fibre
Channel frame. The Fibre Channel Frame is sent without modification to the switch with Domain
ID = 1, because 01 is the first byte in the destination ID. Once there, the frame will be forwarded
out the switch port attached to the storage array.
We already know that MAC addresses are used at the Ethernet layer. But there are actually three
different MAC addresses assigned to a CNA Ethernet port.
The first is the Physical MAC. This is the Burned In Address (BIA) of the CNA.
The second is the ENode MAC. This is the MAC assigned by the CNA to the FCoE Controller. The
FCoE Controller initiates the FIP (FCoE Initialization Protocol) and uses the ENode MAC as the
Source MAC address in those frames.
The third is the FPMA (Fabric Provided MAC Address). This address is assigned by the FCoE
switch to the VN_Port and is used for transmission of Fibre Channel frames.
The CNA VN_Port is created and the FPMA (Fabric Provided MAC address) is assigned during the
Fibre Channel Login process. FPMA addresses are not universal, but instead are addresses local to
the SAN. [These addresses have OUIs with the U/L (universal / local) bit set to 1].
The 48-bit FPMA is made up of two parts. The 24-bit FC-MAP (usually set to 0E-FC-00; which is
“FCOE” backwards), and the 24-bit FCID that is assigned by the connected switch during FLOGI.
When a CNA first comes online, its Ethernet port follows DCBX protocol. It then must follow the
Fibre Channel initialization protocol and login to the fabric to get an FCID. However there are a
few extra steps at the beginning of the Fibre Channel initialization process called the FCoE
Initialization Protocol or FIP. The main purpose of FIP is to discover if there are any FCoE
switches (FCF) in the converged network, and if the path to them can handle full size Fibre
Channel frames.
To do this, the CNA ENode FCoE Controller will send a multicast frame called FIP VLAN Request.
The purpose is to discover which VLANs provide Fibre Channel services. Each FCF responds with a
FIP VLAN Notification frame.
Then the ENode multicasts a FIP Solicitation frame to find which FCFs are available for login.
Each FCF responds with a FIP Advertisement frame.
At this point the FIP discovery is complete and now the CNA FCoE Controller can login to the Fibre
Channel fabric by sending a FIP FLOGI to an FCF that has advertised that it is accepting logins.
The reason it does a FIP FLOGI instead of a regular FLOGI is that the FIP FLOGI ACC frame has a
field for the FCF to assign the VN_Port FPMA MAC address. Once the reply to the FIP FLOGI is
received, the VN_Port is instantiated and has both a MAC address and a Fibre Channel ID.
The VN_Port continues with the normal Fibre Channel initialization by sending a PLOGI to the
fabric name server, and registering information, querying the name server, registering for state
change notifications and logging into discovered storage devices.
This lesson covers popular FCoE datacenter Implementations.
Here we show a converged network implementation that combines the LAN and SAN networks for
the first network hop (from host to first network switch). This converged switch is either at the
top of a rack of servers or at the end of a row of server racks.
The converged networking switch can then separate the Ethernet traffic, which is sent to the LAN,
and FCoE traffic, which is sent to the SAN.
This is a solution that allows New high performance hosts to attach to existing Fibre Channel
Storage, while reducing the amount of cables, I/O cards, and switches used in traditional
methods.
Current EMC recommendations specify that each host should have two CNAs (for redundancy) to
process FCoE frames.
For new implementations where there is no existing Fibre Channel storage, Cloud architects may
consider designing a solution that uses FCoE data paths all the way from the host to the storage
array. This is available on selected EMC CLARiiON, VNX, and VMAX storage arrays, and does not
require Connectrix Fibre Channel Switches.
Here we show a typical Top-of-Rack solution. As you can see here, using the typical configuration
of separate NIC and Fibre channel interfaces, there are multiple sets of cabling, adapters and
switches for standard Fibre channel attached rack servers.
However, if we implement an FCoE solution, we can reduce the number of adapters, cables and
switches that must be managed. The savings is significant as can be seen in the table on the
right.
Note that copper coax cables are preferred as a low cost solution for in-cabinet cabling. Then
Fiber optic cabling can be used to get from the top of the rack to the core switches.
The first implementations of FCoE were all single-hop point-to-point links between the CNA and
the FCF. Now, we are starting to see multi-hop implementations, where there is a switch of some
sort between the CNA and FCF. In these implementations, the switch between the CNA and the
FCF must be able to support both lossless Ethernet, and be able to snoop on FIP frames. This is
called a FIP Snooping Bridge (FSB). In addition to FIP, a FIP snooping bridge must support PFC,
ETS, DCBX, and Dynamic ACLs. The reason for this is that Fiber Channel assumes all connections
are point-to-point. If we put a switch (or bridge) in the path, it opens up the possibility for all
sorts of mischief. For example, the SAN would then become vulnerable to Denial-of-Service (DOS)
or man-in-the-middle attacks.
A FIP Snooping bridge can read FIP frames and adjust ACLs (Access Control Lists) to only allow
ENodes and FCFs that have passed the FIP protocols to communicate. FCoE cannot be guaranteed
to function properly when a non-FIP-aware Ethernet Bridge is used anywhere in the data path.
This module covered FCoE Protocol and examined the enabling technologies (such as Enhanced
Ethernet), and also covered addressing and popular implementations.
This module covers InfiniBand basic concepts, components, and illustrates deployment scenarios
and use cases for InfiniBand.
This lesson covers basic concepts of InfiniBand architecture including layers, messages, packets,
basic operation, flow control, and addressing.
InfiniBand is an open protocol promoted by the InfiniBand Trade Association. All of the details for
the standard are found at www.infinibandta.org.
The protocol is a reliable, lossless protocol. It is similar to Fibre Channel protocol in some
ways, including the fact that it transmits a serial stream of bits, uses credit based flow control,
8b/10b (and 64b/66b) encoding, and supports a switched fabric type of topology.
There are a few areas where InfiniBand is very different from Fibre Channel. InfiniBand has
direct access to a computer’s memory, and messages are passed from the application
memory space of one system directly to the application memory space of another system,
bypassing the operating system.
Also, InfiniBand can aggregate several links into one physical link, and send serial data bytes in
parallel down multiple wires or optic fibers. Links are serial high bandwidth links. Links may be
a single transmit and receive pair (1x), or they may be an aggregation of four (4x), eight (8x), or
twelve (12x) pairs of transmission lines. A quad data rate (QDR) 4x link has 40 Gbps of raw
bandwidth in each direction.
The main application of InfiniBand is currently in the area of High Performance Computing (HPC),
to enable high-speed message passing between clustered systems.
The InfiniBand standard defines components including channel adapters, switches and routers.
All transmissions begin or end at a channel adapter - either a host channel adapter (HCA) or
target channel adapter (TCA). Channel adapters execute transport-level functions and support
the verbs interface. The InfiniBand verbs interface is similar to an API - Application
Programming Interface.
InfiniBand switches relay packets from one link to another link on the same subnet.
Although not very common, InfiniBand routers are also available which route packets from one
subnet to another.
With traditional I/O operations, an application must make a system call to the operating system.
The operating system owns the I/O stack and all I/O resources.
With InfiniBand, a messaging service is provided that allows applications to bypass the
operating system. A private protected channel is created that allows secure communication
between application virtual memory address spaces. Each end of the channel has a Queue Pair
(QP), an addressable entity that consists of two work queues: a transmit queue and receive
queue. More than one QP can be assigned to an application as needed.
The InfiniBand channel directly connects an application’s virtual address space with another
application’s virtual address space.
Let’s consider an example where the “initiator” wishes to pass a block of data to another
computer. To do so, it places the block of data in a buffer in its virtual address space and uses a
SEND operation to send a request message to the “target” system. The target, in turn, uses
RDMA READ operations to fetch the block of data from the initiator’s virtual buffer. RDMA is
Remote Direct Memory Access.
Once it has completed the operation, the target uses a SEND operation to return an ending status
message to the initiator. Notice that the initiator, having requested service from the target, was
free to go about its other business while the target asynchronously completed the storage
operation, notifying the initiator on completion.
We have shown what logically happens in this data transfer, but remember, each message will be
sent across the InfiniBand interface represented at the bottom of the graphic. Also note that the
operating system is not involved in the data transfer.
InfiniBand architecture is defined in four layers: Physical, Link, Network, and Transport.
The Physical layer specifies signal levels and frequency, cables (media) and connectors.
The Link layer specifies symbols and framing, credit-based flow control, and how packets are
relayed from source to destination.
The Network layer describes how packets are routed between subnets.
The Transport layer delivers packets to the appropriate Queue Pair (QP). This is where message
assembly and de-assembly occurs, and remote memory access rights are managed.
Upper layer protocols and applications communicate using “verbs” with the InfiniBand
operations layer.
The InfiniBand standard has several different data rates as shown here. Currently, a popular
implementation is QDR (Quad Data Rate) with four serial data paths (4x) which has an aggregate
signaling rate of 40 Gbps in each direction. Because this method uses 8b/10b encoding, the
effective data rate is 32 Gbps.
InfiniBand uses industry standard cables and connectors. Both copper and fiber optic cables are
supported.
Copper cables will span a distance of up to 20 m for SDR, 10 m for DDR, and 7 m for QDR.
Fiber optic cables support 300 m for SDR, 150 m for DDR and 100 to 300 m for QDR.
QSFP (Quad SFP) cables support four transmit and four receive paths in each cable. 12x cables
have 12 transmission paths and 12 receive paths in one cable.
InfiniBand’s primary unit of communication is the message. There may be several messages in a
single transaction. A message can be up to 2 Gbytes. Messages are divided into convenient size
chunks of information called packets. Each packet has a header with addressing information for
getting the frame to the correct end destination.
The maximum packet payload is governed by the MTU (Maximum Transfer Unit). InfiniBand
allows MTUs from 256 Bytes to 4K Bytes.
Only packets smaller than or equal to the MTU are transmitted. A large MTU is more efficient
because there is less overhead, however a smaller MTU gives less jitter. A small MTU is preferable
since segmentation and reassembly is performed by hardware in the HCA and requires little
software overhead.
Note: In electronic communications, “jitter” is a term that describes the undesired deviation from
a periodic signal such as a clock.
InfiniBand can provide separate channels or streams of data on a single physical link. Each stream
is called a Virtual Lane and has its own send and receive buffers as well as its own credit-based
flow control.
The InfiniBand specification allows a total of 16 VLs. VL0 - VL14 are for data and VL15 is used
exclusively for management traffic. A minimum of 1 Data and 1 management VL is required on all
physical links
The port maintains separate flow control over each data VL such that excessive traffic on one VL
does not block traffic on another VL.
Flow control is credit based and assures no packets are lost due to buffer overruns during times
of link congestion.
Link receivers grant credit to link transmitters as they move data from their buffers and free up
space for additional packets. Credits are granted for each virtual lane and indicate the number of
data packets that the receiver can accept on that VL. A virtual lane cannot transmit a packet
unless it has credit to do so.
Using virtual lanes alleviates congestion due to head of line blocking. Congestion and latency on
one VL does not impact traffic with guaranteed QoS on another VL, even though they share the
same physical link.
VL15 (the management VL) is not subject to flow control.
The Link Physical layer provides an interface between the packet byte stream of upper layers and
the serial bit stream(s) of the physical media. The physical media may be implemented as 1, 4, 8,
or 12 physical lanes. The packet byte stream will be byte striped across the available physical
lanes.
Shown here is byte striping on a 4x link (four physical lanes of traffic in each direction). In this
example, the packet byte stream is striped across all four physical paths. The byte stream on
each physical lane is encoded using 8b/10b coding.
For higher data rates above QDR, 64b/66b encoding is used and the striping method sends 8
consecutive bytes down each lane.
Each packet has a Local Routing Header (LRH) that will get the packet to the correct node on a
subnet. The local addressing information is in the form of a Local ID (LID), a16-bit field used to
route packets in an InfiniBand subnet. Each subnet may contain up to: 48K unicast addresses and
16K multicast addresses. This address is assigned by the Subnet Manager at Initialization and
when the subnet topology changes.
The transport layer adds the Global Routing Header (GRH) to a packet. This header includes
the GID (Global Identifier): A 128-bit unicast or multicast identifier used to identify an endport
or a multicast group. A GID is a valid 128-bit IPv6 address with additional properties and
restrictions defined by the InfiniBand standard to facilitate efficient discovery, communication,
and routing. Every node must have a GID.
This lesson covers possible deployment scenarios and use cases for InfiniBand.
InfiniBand has many applications, but it seems particularly well-suited for High Performance
Computing (HPC) applications, especially large parallel computing scientific applications. This is
because it enables high performance storage and low latency InterProcess Communication (IPC).
Some examples of HPC applications include atmospheric modeling, genomics research,

automotive crash test simulations, oil and gas extraction models and fluid dynamics.
One of the goals of InfiniBand is the same as for FCoE, and that is to converge and consolidate
datacenter networking interfaces. For example, in the host on the left, we have three applications
that each have their own adapter to connect to a different network. This is very inefficient.
InfiniBand provides the architecture to combine these networks and their switches and adapters
into a consolidated network using an HCA and an InfiniBand fabric. There are upper layer
protocols defined that allow us to run SCSI over InfiniBand, IP over InfiniBand, NAS over
InfiniBand, just to name a few. With Virtual Lanes, InfiniBand has the tools to keep each
application secure and separate from other applications.
Gateway devices are also available to connect a Fibre Channel SAN or an IP LAN to an InfiniBand
fabric.
So far, datacenter consolidation has not been a very popular application for InfiniBand, and FCoE
seems to have much greater traction in this area.
Big data requires highly scalable solutions with high computational density and massive data
movement. InfiniBand has excelled in this type of environment for years with HPC applications.
InfiniBand is already being used as the internal network on the back end of products like EMC’s
Isilon and Oracle’s Exadata products.
Big data hadoop-type clusters would also benefit from the speed and flexibility of InfiniBand on
the front end. Replacing 10 Gbps Ethernet connections with InfiniBand could greatly improve the
speed of data analytics in these environments.
With data continuing to grow at an exponential rate, high-performance data analytics is in

demand, and InfiniBand’s high throughput and low latency could make it the network of choice.
This module covered InfiniBand architectural layers, memory access, physical and virtual traffic
lanes, addressing, and deployment options.
This course covered layered networking protocols including, Ethernet, TCP/IP, Fibre Channel,
iSCSI, FCIP, FCoE and InfiniBand.
This concludes the training. Thank you.

Connectivity Protocols SRG

Hochgeladen von

Dokumentinformationen

Copyright

Verfügbare Formate

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

Verfügbare Formate

Connectivity Protocols SRG

Hochgeladen von

Copyright:

Verfügbare Formate

Welcome to Connectivity Protocols.

EMC, EMC² AccessAnywhere Access Logix, AdvantEdge, AlphaStor, AppSync ApplicationXtender,

Revision Date: 11/2015

Copyright 2015 EMC Corporation. All rights reserved. Connectivity Protocols 1

Copyright 2015 EMC Corporation. All rights reserved. Connectivity Protocols 2

Copyright 2015 EMC Corporation. All rights reserved. Connectivity Protocols 3

Copyright 2015 EMC Corporation. All rights reserved. Connectivity Protocols 4

Copyright 2015 EMC Corporation. All rights reserved. Connectivity Protocols 5

Copyright 2015 EMC Corporation. All rights reserved. Connectivity Protocols 6

Copyright 2015 EMC Corporation. All rights reserved. Connectivity Protocols 7

Common components of a WAN include:

Copyright 2015 EMC Corporation. All rights reserved. Connectivity Protocols 8

Copyright 2015 EMC Corporation. All rights reserved. Connectivity Protocols 9

Copyright 2015 EMC Corporation. All rights reserved. Connectivity Protocols 10

Copyright 2015 EMC Corporation. All rights reserved. Connectivity Protocols 11

Copyright 2015 EMC Corporation. All rights reserved. Connectivity Protocols 12

Copyright 2015 EMC Corporation. All rights reserved. Connectivity Protocols 13

Copyright 2015 EMC Corporation. All rights reserved. Connectivity Protocols 14

Fibre Channel networks often use a form of mesh topology.

Copyright 2015 EMC Corporation. All rights reserved. Connectivity Protocols 15

Copyright 2015 EMC Corporation. All rights reserved. Connectivity Protocols 16

Copyright 2015 EMC Corporation. All rights reserved. Connectivity Protocols 17

Copyright 2015 EMC Corporation. All rights reserved. Connectivity Protocols 18

Copyright 2015 EMC Corporation. All rights reserved. Connectivity Protocols 19

TCP/IP is a 4-layer protocol that is an example of an implementation of the OSI model.

Copyright 2015 EMC Corporation. All rights reserved. Connectivity Protocols 20

Copyright 2015 EMC Corporation. All rights reserved. Connectivity Protocols 21

Copyright 2015 EMC Corporation. All rights reserved. Connectivity Protocols 22

Copyright 2015 EMC Corporation. All rights reserved. Connectivity Protocols 23

Copyright 2015 EMC Corporation. All rights reserved. Connectivity Protocols 24

Copyright 2015 EMC Corporation. All rights reserved. Connectivity Protocols 25

Copyright 2015 EMC Corporation. All rights reserved. Connectivity Protocols 26

Copyright 2015 EMC Corporation. All rights reserved. Connectivity Protocols 27

Copyright 2015 EMC Corporation. All rights reserved. Connectivity Protocols 28

Bit 47 is the Individual/Group bit. This bit is set to 1 for broadcast.

Bits 23-0 are assigned by the vendor.

Copyright 2015 EMC Corporation. All rights reserved. Connectivity Protocols 29

We have abbreviated the addresses in the frame header for simplicity.

Copyright 2015 EMC Corporation. All rights reserved. Connectivity Protocols 30

Copyright 2015 EMC Corporation. All rights reserved. Connectivity Protocols 31

Copyright 2015 EMC Corporation. All rights reserved. Connectivity Protocols 32

Copyright 2015 EMC Corporation. All rights reserved. Connectivity Protocols 33

Copyright 2015 EMC Corporation. All rights reserved. Connectivity Protocols 34

Copyright 2015 EMC Corporation. All rights reserved. Connectivity Protocols 35

Copyright 2015 EMC Corporation. All rights reserved. Connectivity Protocols 36

Copyright 2015 EMC Corporation. All rights reserved. Connectivity Protocols 37

Copyright 2015 EMC Corporation. All rights reserved. Connectivity Protocols 38

Copyright 2015 EMC Corporation. All rights reserved. Connectivity Protocols 39

Copyright 2015 EMC Corporation. All rights reserved. Connectivity Protocols 40

STP  Spanning Tree Protocol (IEEE 802.1d-1998 standard)

RSTP  Rapid Spanning Tree Protocol (IEEE 802.1w standard)

PVST+  Per-VLAN Spanning Tree (Cisco proprietary)

Rapid PVST+  Rapid Per-VLAN Spanning Tree (Cisco proprietary)

MSTP  Multiple Spanning Tree Protocol (IEEE 802.1s standard)

Copyright 2015 EMC Corporation. All rights reserved. Connectivity Protocols 41

Copyright 2015 EMC Corporation. All rights reserved. Connectivity Protocols 42

Copyright 2015 EMC Corporation. All rights reserved. Connectivity Protocols 43

Copyright 2015 EMC Corporation. All rights reserved. Connectivity Protocols 44

Copyright 2015 EMC Corporation. All rights reserved. Connectivity Protocols 45

Copyright 2015 EMC Corporation. All rights reserved. Connectivity Protocols 46

Copyright 2015 EMC Corporation. All rights reserved. Connectivity Protocols 47

Copyright 2015 EMC Corporation. All rights reserved. Connectivity Protocols 48

Copyright 2015 EMC Corporation. All rights reserved. Connectivity Protocols 49

Copyright 2015 EMC Corporation. All rights reserved. Connectivity Protocols 50