Sie sind auf Seite 1von 94

Storage

Fundamentals

March 2016

 Copyright 2016 Hewlett Packard Enterprise Development LP


Introduction

Confidential – For Training Purposes Only 2


Course overview

After completing this training, you should be able to:


– Explain hard drive types, interconnect technologies, and RAID levels.
– Explain Fibre Channel storage area networks (SANs) and components, and compare SANs to direct
attached storage (DAS) and network-attached storage (NAS).
– Describe the Fibre Channel architecture, characteristics, and operation, including naming and addressing.
– Describe the Fibre Channel Arbitrated Loop topology and its operation and benefits.
– Describe a switch topology with fabric operation and zoning concepts.
– Describe fiber optic technology and Fibre Channel cabling options and connectors.

Confidential – For Training Purposes Only 3


Defining the storage technology

Confidential – For Training Purposes Only 4


Hard drives

Entry drives Midline drives Enterprise drives

High capacity High reliability


General description Lowest unit cost
Lowest cost per gigabyte High performance
External storage
Boot drive Mission critical
Use environments Backups/archival
Non-critical strorage High I/O
Redundancy
Workload < 40% < 40% Unconstrained workloads
Reliability 2 X Entry drive reliability 3.5 X Entry drive reliability
SATA 3 Gb/s,
Interface SATA 3 Gb/s SAS 3 Gb/s and 6 GB/s
SAS 3 Gb/s and 6 GB/s
Single port SATA
Connectivity Single port Single and dual port
Dual port SAS
RPM 5,400 and 7,200 7,200 10,000 and 15,000

Confidential – For Training Purposes Only 5


Characteristics of drives

– Form factor
– Small form factor (SFF)—2.5-inch
– Large form factor (LFF)—3.5-inch

– Drive capacity
– Depends on number of platters the drive contains, the surface area of each platter, and the areal density
(the number of bits that can be stored per unit area)
– Expressed in gigabytes

– Disk drive performance


– Depends on the rotational speed of the platters, the seek performance, the mechanical latency, the read/write
bandwidth, the queuing strategies, and the interface technologies

– Reliability
– Measured in terms of Annual Failure Rates (AFRs)

Confidential – For Training Purposes Only 6


Drive interconnect technologies

The technology to connect one or more drives to a computer system has transitioned from parallel
bus data interfaces to serial interfaces
– Parallel interfaces:
– ATA—Advanced Technology Attachment
– IDE—Integrated Drive Electronics, also called PATA, Parallel Advanced Technology Attachment
– SCSI—Small Computer System Interface

– Serial interfaces:
– SATA—Serial ATA
– SAS—Serial Attached SCSI

Confidential – For Training Purposes Only 7


Serial ATA and Serial Attached SCSI drives

Serial ATA Serial Attached SCSI


– SATA uses a half-duplex serial connection and ATA uses – SAS uses a point-to-point, full-duplex serial connection
a command set and the SCSI command set
– Three generations of SATA drives: – Two generations of SAS drives:
– 1.5 Gb/s – First-generation SAS supported a link speed of
– Targeted at replacing ATA in the desktop and consumer markets 3 Gb/s

– 1.5 Gb/s with extensions – The current generation supports a link speed of up to 6 Gb/s
– Targeted for workstations and low-end servers
– This generation added native command queuing

– 3 Gb/s
– Targeted for workstations and low-end servers
– This generation increased the data transfer rate

Confidential – For Training Purposes Only 8


Improving performance and reliability with RAID

– Storing data on the single drive creates the risk of losing Raid level Description
data
RAID 0 Striping
– To achieve better performance and fault tolerance, it is
recommended to store data across multiple drives RAID 1 Mirroring
RAID 1+0 Striping and mirroring
– Disks can be combined to form an Redundant Array of
Independent Disks (RAID) RAID 5 Block striping with distributed parity
– RAID strategies vary RAID 6 Block striping with distributed parity
– How they achieve data reliability
– How many drives they require
– How efficient they are at data storage

Confidential – For Training Purposes Only 9


RAID 0

– Minimum of 2 disks
– Excellent performance (as blocks are striped)
– No redundancy (no mirror, no parity)
RAID 0
– Do not use this for any critical system

NOTICE: RAID 0 provides no data redundancy. A1 A2


A3 A4
A5 A6
A7 A8
Disk 0 Disk 1

Confidential – For Training Purposes Only 10


RAID 1

– Minimum of 2 disks
– Good performance (no striping, no parity)
– Excellent redundancy (blocks are mirrored)
RAID 1
– Provides 50% of usable disk space

NOTE: For more information about mirriring, go to: A1 A1


http://en.wikipedia.org/wiki/Disk_mirroring.
A2 A2
A3 A3
A4 A4
Disk 0 Disk 1

Confidential – For Training Purposes Only 11


RAID 1+0

– Minimum of 4 disks
– RAID 1+0 is also called ”stripe of mirrors” RAID 1+0
– Excellent redundancy (blocks are mirrored)
RAID 0
– Excellent performance (blocks are striped)
RAID 1 RAID 1
– This is the best option for any mission-critical
applications (especially databases)
– Provides 50% of usable drive space A1 A2
A1 A2
A3 A3 A4 A4
A5 A5 A6 A6
A7 A7 A8 A8
Disk 0 Disk 1 Disk 2 Disk 3

Confidential – For Training Purposes Only 12


RAID 5

– Minimum of 3 disks
– Good performance (blocks are striped)
– Good redundancy (distributed parity) RAID 5
– The most cost-effective option, providing both
performance and redundancy
– Use this for a database that is heavily read oriented A1 A2 A3 Ap
– Write operations will be slow B1 B2 Bp B3
– Provides 67% to 93% of usable drive space C1 Cp C2 C3
Dp D1 D2 D3
Disk 0 Disk 1 Disk 2 Disk 3

Confidential – For Training Purposes Only 13


RAID 6

– Just like RAID 5, this does block-level striping


– However, it uses dual parity
– It creates two parity blocks for each data block

– Can handle two disk failures RAID 6


– Requires a minimum of 4 drives
– This RAID configuration is complex to implement in a A1 A2 A3 Ap Aq
RAID controller because it has to calculate two parity
B1 B2 Bp Bq B3
data for each data block
C1 Cp Cq C2 C3
Dp Dq D1 D2 D3
Disk 0 Disk 1 Disk 2 Disk 3 Disk 4

Confidential – For Training Purposes Only 14


Storage arrays

– Multiple drives combined to increase overall storage


capacity, data availability, and performance
– Drives are combined to form RAID groups
– Available disk space is arranged in the form of logical
(virtual) drives
– Clients (hosts) access the available storage space using
available communication channels such as:
– iSCSI (SCSI over TCP/IP)
– FC (Fibre Channel)
– FCoE (Fibre Channel over Ethernet)

Confidential – For Training Purposes Only 15


Switches, Fibre Channel, iSCSI technologies

– The fabric for a SAN provides the connectivity between


the host servers and the storage devices
– The dominant architecture for SANs is based on Fibre
Channel (FC)
– Compared to SCSI devices, many more storage devices
can be connected over much larger distances with
higher data transfer rates
– In Fibre Channel topologies, the host server can be
connected to the storage directly, or by means of a hub
or a switch

Confidential – For Training Purposes Only 16


Introduction to DAS, NAS, and SAN

Confidential – For Training Purposes Only 17


DAS, NAS, and SAN

DAS NAS SAN

Simple Performance
implementation
Advantages Speed and security Scalability
Unrestricted distance
over the LAN Manageability

Distance restrictions
High network Greater initial
High network
Disadvantages overhead and limited investment support
overhead
scalability expertise
Limited scalability

Confidential – For Training Purposes Only 18


Direct attached storage

– The traditional method of locally attaching storage to servers through a dedicated SCSI communication
channel between the server and storage
– Storage for each server is managed separately and cannot be shared
– DAS supports disk drives, a RAID subsystem, or another storage device

HPE SERVERS DIRECT ATTACHED STORAGE

SCSI CABLES

Confidential – For Training Purposes Only 19


Network-attached storage

– NAS provides a file-level access to storage systems


Application NAS Database
– NAS devices are: Server Server
– Server-independent
– Used to off-load storage traffic to a single, dedicated storage
device

Clients Internal SCSI or Clients


SAN attached storage

Confidential – For Training Purposes Only 20


Fibre Channel storage area network

Dedicated network that provides access to


consolidated, block-level data storage
– Special switches are used to connect storage arrays with
servers and with each other
– Network communication uses the Fibre Channel
protocol, which was specially developed for the transport
of files
– This protocol is reliable, with speeds up to 16 Gbit/s

– FC SAN components allow for high levels of redundancy


and resiliency

Confidential – For Training Purposes Only 21


SAN considerations

When designing SAN solutions, consider the following:


– Scalability (number of FC ports and expansion capability)
– Storage capacity, efficiency, and cost
– Availability of the fabric, systems, and data
– Performance
– Remote replication of data

Confidential – For Training Purposes Only 22


Comparing SAN and NAS

SAN benefits NAS SAN


– Network speed
– Reliability
– Centralization
– Data protection
NAS benefits
– Interoperability
– Lower TCO
– Simplicity

Confidential – For Training Purposes Only 23


Comparing DAS, NAS, and SAN

DAS NAS SAN

Application Application Application


software software software

Network

File system File system File system

FC/GbE

Storage Storage Storage

Confidential – For Training Purposes Only 24


Choosing between SAN, NAS, and DAS

DAS NAS SAN

Applications Any File serving Storage for application servers


Server and Operating System General purpose Optimized General purpose
Storage Devices Internal or external dedicated External direct-attached External shared
Management Labor intensive Centralized Centralized
Small workgroup to enterprise
Data Centers Workgroup or departmental Workgroup or departmental
data centers
Performance Network traffic Increased network performance Higher bandwidth
Distance None Limited distance Greater distances
Greater speeds (up to 16
Speed Bottlenecks Improved bottlenecks
Gbit/s)
No Single Point of Failure
Availability Limited Limited
(NSPOF)
Cost Low cost Affordable High host, but great benefits

Confidential – For Training Purposes Only 25


Tiered storage

Policy-based Data Migration


Performance and cost

Tier 1

Tier 2 Online

Near-online Instant data access and


Tier 3 recovery
Tier 4 Reference information Faster recovery
High Performance, FC
Near line, Frequently accessed data based Disk Arrays
Searchable
File recovery Indexed online archive Mid-Range FC based Disk
Arrays
Backup devices, Tapes NAS or DAS based
and Tape Libraries solutions

Scalability and availability

Confidential – For Training Purposes Only 26


SAN components

Confidential – For Training Purposes Only 27


Identifying SAN components

– Host
– Servers
– HBAs

– Fabric
– Hubs or switches
– Routers
– SAN software
– Fibre Channel cables

– Storage
– Storage devices
– Backup devices

Confidential – For Training Purposes Only 28


Host component (initiator)

Consists of servers and components that enable


servers to connect to the SAN
– HBAs
– In-server components that perform digital-to-optical signal
conversion

– HBA drivers
– System software that enables the operating system of a
server to communicate with the HBA

– Multipath software
– A software component that enables fault-tolerance and
performance enhancements (MPIO)

Confidential – For Training Purposes Only 29


HBAs

Fibre Channel HBAs


– Can address more devices than the
SCSI or NIC counterparts
– Provide I/O connectivity to more
devices over longer distances than
SCSI
– Enable Fibre Channel frames to
relay over gateways

Mezzanine HBA
PCI HBA

Confidential – For Training Purposes Only 30


Disk array (target)

Disk array characteristics


– Multiple port connections
– Up to 99.999% uptime—about 5 minutes of downtime
per year!
– Battery-backed controller cache for protected “write-
back” caching
– Snapshot and cloning capabilities
– Remote, controller-based replication for data integrity
and disaster recovery

Confidential – For Training Purposes Only 31


Interconnect devices

Fibre Channel switches


– Two types:
– Fabric switches—Smaller fixed
configurations
– Directors—High port count in a
modular (slot-based) chassis with no
single point of failure

Fabric switch

SAN director switch

Confidential – For Training Purposes Only 32


SAN boot order

3 1 2

Confidential – For Training Purposes Only 33


Fibre Channel basics

Confidential – For Training Purposes Only 34


Open System Interconnection

What is OSI?
– A reference model is a framework for understanding relationships
– Open System Interconnection (OSI) is a reference model for how messages should be transmitted
between any two points in a telecommunications network
– The purpose of the OSI reference model is to guide vendors so the digital communication products they
create will interoperate

Confidential – For Training Purposes Only 35


OSI layers

Layer 7 Application

Layer 6 Presentation

Layer 5 Session

Layer 4 Transport

Layer 3 Network

Layer 2 Data-link

Layer 1 Physical

Confidential – For Training Purposes Only 36


OSI layers in the Fibre Channel stack

Layer Title Fibre Channel

7 Application
6 Presentation SCSI-3, IPI, HIPPI, IP
5 Session
4 Transport FC – 4 Protocol Interface ULP
3 Network FC – 3 Encryption Authentication
2 Data Link FC – 2 Framing Flow Control Class of Service
FC – 1 Encoding Link Control
1 Physical FC – 0 Physical

Confidential – For Training Purposes Only 37


World Wide Name
The definition
What is WWN?
– A World Wide Name is a 64-bit IEEE structured address
– Example: 21:11:00:02:AC:00:08:EB
– It is used to preserve the identity of a node if its FC – 2 (Data Link) or FC – 3 (Network) layer address is
changed
– The WWN is unique worldwide, and it is assigned for the life of a connection device
– A WWN consists of three sections:
– Section 1: Identifies the WWN as a standard format WWN
– The first 2 bytes are either hex 10:00 or 2x:xx (where the x's are vendor-specified)
– Section 2: TheOrganizationally Unique Identifier (OUI)
or “company_id” that identifies the vendor
21:00 00:e0:8b 00:e0:8b
– Section 3: A unique identifier created by the vendor
Section 1 Section 2 Section 3

Confidential – For Training Purposes Only 38


WWN Port Name and Node Name

Two types of WWNs


– World Wide Node Name (WWNN)—Assigned to the node (server or storage array)
– World Wide Port Name (WWPN)—Assigned to the port of the Fibre Channel device

1xWWNN
2xWWPN 1xWWNN

4xWWPN

Confidential – For Training Purposes Only 39


Fibre Channel WWN

A WWN can be used for:


– Zoning—To identify zone members
– LUN masking—To identify entities that are permitted or denied access to LUN resources within an array
A WWN is not used for:
– Frame delivery
– Inter-switch (fabric) traffic delivery

Confidential – For Training Purposes Only 40


Nodes, ports, and links

Device ports Link Switch ports


– N_Port—Node port – Connects ports together – U_Port—Universal port
– A device directly attached to a fabric – Can be a copper or a fiber optic – A port that is waiting to become a
different port type
– NL_Port—Node loop port cable
– F_Port—Fabric port
– A device connected to a hub
– A port that is attached to an N_Port

– FL_Port—Fabric loop port


– A switch connected to a hub

– E_Port—Expansion Port
– A port that is connected to another
switch using an inter-switch link

– G_Port—Generic Port
– A port that is waiting to become an
E_Port or an F_Port

Confidential – For Training Purposes Only 41


SAN topologies

Confidential – For Training Purposes Only 42


Fibre Channel topologies

Arbitrated loop (FC-AL) Switched fabric (FC-SW)


N_Port NL_Port
L_Port
F_Port FL_Port
L_Port L_Port FC 0
FC 1
HBA Fabric L_Port

F_Port FL_Port
L_Port L_Port
Host
L_Port L_Port N_Port

L_Port L_Port L_Port

L_Port

Point-to-point (FC-P2P)
N_Port N_Port

Confidential – For Training Purposes Only 43


Point-to-point topology

– FC-P2P is inexpensive
– Uses full bandwidth and has limited scalability
– Only connects two devices
– A separate P2P configuration must be created for each new storage device, requiring a new HBA for each
one

Node A Node A

Receiver Transmitter
Transmitter Receiver

Confidential – For Training Purposes Only 44


Arbitrated loop topology

– A serial, full-duplex data transfer architecture


– Each port (NL_Port) on the loop has a transmit (TX) and receive (RX) lines
– The TX line of the upstream device connects to the RX line of the downstream device
– Only one port at a time can transmit data—the bandwidth is divided among all devices on the loop
– Because of the loop arbitration, performance degrades when the number of devices in the loop exceeds
35
L_Port

L_Port L_Port

L_Port L_Port

L_Port

Confidential – For Training Purposes Only 45


Private arbitrated loop

– If there is no active FL_Port on the arbitrated loop, it is referred to as a private loop


– The private loop can accommodate up to 126 NL_Ports
– A private loop is not connected to a switch, so communication and bandwidth are limited to the ports in the
loop
NL_Port

L_Port
NL_Port NL_Port
L_Port L_Port

L_Port L_Port

NL_Port L_Port NL_Port

NL_Port

Confidential – For Training Purposes Only 46


Public arbitrated loop

– A public loop has at least one active FL_Port on the loop


– The public loop can accommodate up to 126 NL_Ports and one FL_Port
– The FL_Port extends the number of ports for communication and introduces the loop identifier, which is
common to all NL_Port addresses in the loop
Fibre Channel Switch

FL_Port FL_Port FL_Port FL_Port FL_Port FL_Port

NL_Port NL_Port

NL_Port NL_Port

NL_Port HUB HUB NL_Port

NL_Port Loop 1 Loop 2 NL_Port

NL_Port NL_Port

Confidential – For Training Purposes Only 47


Switched fabric topology

Switched Fabric (FC-SW)

N_Port NL_Port
F_Port FL_Port

Fabric L_Port

F_Port FL_Port
L_Port L_Port

N_Port

L_Port L_Port

L_Port

Confidential – For Training Purposes Only 48


Comparing topologies

Point-to-point Arbitrated loop Switched fabric


– Advantages: – Advantages: – Advantages:
– Full bandwidth for the link – Scalability – Multiple devices communicate at the
same time
– Good topology for disk drive I/O
– Loss of one component does not
interrupt the link
– Full bandwidth for each switch port
– Performance only minimally depends
on length

– Disadvantages: – Disadvantages: – Disadvantages:


– High cost for hardware – All ports share bandwidth – Higher initial cost compared to
Arbitrated Loop
– No scalability – Maximum of 126 ports per loop
– The failure of one port forces loop
initialization
– Performance depends on the loop
length and the number of NL_Ports

Confidential – For Training Purposes Only 49


Fibre Channel port types (1 of 2)

Name Description

N_Port A port on the node (storage device or host).

NL_Port A port on the node used in the FC-AL topology.

F_Port A fabric port on the switch that connects to the N_Port.

FL_Port A fabric loop port on the switch that connects to the FC-AL loop.

E_Port An expansion port; the connection between two Fibre Channel switches. When ports between two
switches form a link, that link is referred to as an inter-switch link (ISL).

B_Port A Bridge Port is a fabric inter-element port that is used to connect bridge devices with E_Ports on a
switch. The B_Port provides a subset of the E_Port functionality.

D_Port A diagnostic port, used for the purpose of running link-level diagnostics.

Confidential – For Training Purposes Only 50


Fibre Channel port types (2 of 2)

Name Description

EX_Port The connection between a Fibre Channel router and a Fibre Channel switch. On the side of the
switch, it looks like a normal E_Port, but on the side of the router it is an EX_Port.
TE_Port An extended inter-switch link (ISL) that is used for virtual SANs. Also known as a trunking E_Port.
Auto An auto-sensing port that can automatically become an E_, TE_, F_, or FL_Port as needed.
Fx_Port A generic port that can become an F_Port (when connected to a N_Port) or an FL_Port (when
connected to an NL_Port).
GL_Port A port on a switch that can operate as an E_Port, FL_Port, or F_Port. Found on QLogic switches.
G_Port A generic port; a port waiting to be used as an E_Port or F_Port. Found on Brocade, McData, and
QLogic switches.
L_Port A loose term used for any arbitrated loop port, NL_Port, or FL_Port. Also known as a loop port.
U_Port A loose term used for any arbitrated port or a port waiting to become another port type. Also known
as a universal port. Found only on Brocade switches.

Confidential – For Training Purposes Only 51


Establishing a new link between ports

– When a new link is established between ports, the switch effectively poses 3 questions to the newly
connected port:
– Loop initialization process (LIP)—Do you support loop functions?
– Fabric Login (FLOGI)—Do you support 24-bit addressing?
– All others—Send Link Service frames to establish an “E” port connection?

– All other connections will be ignored by the switch port

Confidential – For Training Purposes Only 52


Fibre Channel architecture

Confidential – For Training Purposes Only 53


Fibre Channel function levels

FC-4
Node level:
– Networks (802.2, IP, ATM)
– FC-4
FC-3
– FC-3
– Common services

FC-2

– Signaling, Framing Protocol, and flow control

FC-1 Port level:


– FC-2
– Encode and decode
– FC-1
FC-0
– FC-0
– Available at 1, 2, 4, 8, 10, 16, and 20 Gbit/s

Confidential – For Training Purposes Only 54


FC-0—Physical level

Defines the physical link in the Fibre Channel system


– Transceivers
– Connection
– Media type
Available data rates
– 133 Mbit/s
– 266 Mbit/s
– 531 Mbit/s
– 1062 Mbit/s

Confidential – For Training Purposes Only 55


Transceivers

Transceiver Data rate Distance

SFP 155 M/622 M/ 300 m/2 km/


1.25 G/ 10 km/15 km/
SFP+ - 16 Gigabit
2.5 G/3 G/ 20 km/40 km/
4.25 G 60 km/80 km/
100 km/120 km/150 km
Application:
– Switches SFP+ 6 G/8.5 G/10 G/ 220 m/300 m/
16 G 2 km/10 km/
– Disk controllers 20 km/40 km/
– FCIP/iSCSI bridges 60 km/80 km

XFP 10 G 220 m/300 m/


2 km/10 km/
20 km/40 km/
60 km/80 km/
120 km

Confidential – For Training Purposes Only 56


Fibre Channel connectors

– SFP, SFP+, and XFP transceivers are compatible with the Lucent Connector (LC) type of connectors
– Cables containing LC connectors on both sides are known as LC-LC cables

Confidential – For Training Purposes Only 57


Fibre Channel cabling

Confidential – For Training Purposes Only 58


Multimode fiber

– Multiple streams of light to travel different paths


– Most popular for networking
– Fibre Channel uses single wavelength
– Example: 850 nm

Confidential – For Training Purposes Only 59


Single-mode fiber

Highest bandwidth and lowest performance loss


– One stream of light travels a single path
– Long wave lasers
– Single-mode, step-index fiber

Confidential – For Training Purposes Only 60


Single-mode step-index fiber

Best for long-distance communication

Confidential – For Training Purposes Only 61


Fiber-optic class signal loss—Attenuation

Attenuation
– The reduction in power of the light signal as it is transmitted
– Caused by passive media components such as cables, cable splices, and connectors

Confidential – For Training Purposes Only 62


Fiber-optic class signal loss—Dispersion

Dispersion
– Spreading of the signal over time
– Two types of dispersion can affect an optical data link:
– Chromatic dispersion—Resulting from the different speeds of light rays
– Modal dispersion—Resulting from the different propagation modes in the fiber

Confidential – For Training Purposes Only 63


Cable bending and damage

Micro bending Macro bending


– Difficult to diagnose – Can reduce the effective data transport distance
– Causes bit transport errors – Causes signal degradation

Confidential – For Training Purposes Only 64


FC-1 coding layer

FC-1 8b/10b encode/decode


– FC-1 defines the transmission protocol including:
– Serial encoding and decoding rules
– Special characters
– Error control

– The information transmitted over a fiber is encoded 8 bits at a time into a 10-bit transmission character

Also used in:


– PCI Express

– IEEE 1394b

– Serial ATA

– SSA

– Gigabit Ethernet

– Infiniband

Confidential – For Training Purposes Only 65


FC-2 signaling protocol level

The transport mechanism of Fibre Channel Building blocks


– Framing rules – Ordered sets
– Payload – Frames
– Service classes and control mechanisms – Sequences
– Management of the data transfer sequence – Exchanges

Confidential – For Training Purposes Only 66


FC-3 common services

– The FC-3 layer covers functions that can span multiple N-ports
– FC-3 defines the common services necessary for the higher level capabilities
– FC-3 provides features such as:
– Port striping
– RAID
– Virtualization
– Compression
– Encryption
– Hunt groups
– Multicast

Confidential – For Training Purposes Only 67


FC-4 ULP mappings

– Each upper-level protocol supported by the Fibre Channel transport requires a mapping for its Information
Units to be presented to the lower levels for transport
– The FC-4 layer provides these mappings for:
– SCSI-3
– IP
– High-Performance Peripheral Interface (HIPPI)
– FC-AV—A high-bandwidth video link for video networks, up to 500m
– FC-VE—Fibre Channel Virtual Interface Architecture
– FC-AE—Fibre Channel Avionics Environment
– Ficon, IEEE 802.2 LLC, ATM, Link Encapsulation, SBCCS, IPI

– A Fibre Channel SAN is almost exclusively concerned with using the SCSI-3 mapping

Confidential – For Training Purposes Only 68


Introduction to iSCSI

Confidential – For Training Purposes Only 69


IP storage

Meeting storage challenges with IP-based network storage


– Increased utilization
– Reduced management cost
– Increased reliability
– Simplified backup and recovery

Confidential – For Training Purposes Only 70


IP storage protocols

iSCSI, FCIP, and iFCP transports

End Fabric
devices services

Internet
iSCSI iSCSI/IP
protocol

Fibre Fibre
FCIP
channel channel

Fibre Internet
iFCP
channel protocol

Confidential – For Training Purposes Only 71


Overview of the iSCSI protocol

What is iSCSI?
– iSCSI is a transport layer protocol that describes how SCSI packets should be transported over a TCP/IP
network
– iSCSI works on top of the TCP
– It allows the SCSI command to be sent end-to-end over LANs, WANs, or the Internet

Confidential – For Training Purposes Only 72


The iSCSI protocol

– The SCSI protocol has been mapped over various transports such as Parallel SCSI, Firewire, and Fibre
Channel
– These transports are I/O specific and have limited distance capabilities
– The iSCSI protocol uses TCP/IP, which can take advantage of existing Internet infrastructure and
management facilities and address distance limitations

Confidential – For Training Purposes Only 73


Comparing iSCSI and Fibre Channel

Fibre channel iSCSI

Designed for enterprise markets Designed for SMB markets


High bandwidth of 1 Gbps, 2 Gbps, 4 Gbps Low bandwidth of 1 GibabitEthernet, 10 GigabitEthernet
Low latency of 2 msec per port High latency; IP has msec latency
Smaller payload of up to 1500 bytes; up to 900 bytes
Large payload of up to 2112 bytes
using jumbo frames
Low overheads of 5.5% for 1 KB payload, 3% for 2 KB
Higher overheads of 8% for 1 KB payload
payload
Long distance; no theoretical limit over IP networks, but
Short distance of 10 km per link for single mode fiber
high latency
Low cost: use existing NiC and LAN; iSCSI HBAs are
High cost HBAs and switch ports
expensive

Confidential – For Training Purposes Only 74


iSCSI protocol stack

Initiator Target
SCSI SCSI
iSCSI iSCSI
TCP TCP
IP IP
IPSec IPSec
Link Link
IP network

Confidential – For Training Purposes Only 75


iSCSI encapsulation

Ethernet
IP TCP iSCSI Data CRC
Header

Delivery of iSCSI Protocol Data Unit


(PDU) for SCSI functionality (initiator,
target, data read/write, etc.)

Reliable data transport and delivery (TCP


Windows, ACKs, ordering, etc.) Also demux
within node (port numbers)

Provides IP „routing” capability so that packet can find


its way through the network

Provides physical network capability (Cat 5, MAC, etc.)


Confidential – For Training Purposes Only 76
iSCSI drivers and offload engines
Apps/file Apps/file Apps/file
systems systems systems
SCSI SCSI SCSI

iSCSI HBA
TOE cards
Other protocols Other protocols
NIC cards

Other protocols
iSCSI iSCSI iSCSI
TCP TCP TCP

IP IP IP

Network Network Network


hardware hardware hardware

Apps/file
systems
SCSI
Fabric adapter

Other protocols
iSCSI
TCP

IP Processed in
Processed in
the network
Network the server
card
hardware

Confidential – For Training Purposes Only 77


iSCSI names

– iSCSI names:
– Are used for identification
– Are used for authentication
– Enable iSCSI resources to be managed regardless of their location

– Each iSCSI initiator and target must have an iSCSI name


– The iSCSI name consists of two parts: a “type designation” followed by a unique name string
– The three type designators for iSCSI are:
– iqn. iSCSI qualified name (iqn.2003-02.com.hp:server3)
– eui. IEEE EUI-64 identifier in ASCII-encoded hexadecimal (eui.02004567A425678D)
– NAA. T11 Network Address Authority Format NASA 64 or 128 bit identifier (naa.52004567BA64782D)

Confidential – For Training Purposes Only 78


Internet Storage Name Service

An iSNS implementation provides four primary services:


– Name Registration and Storage Resource Discovery
– Discovery Domains and Login Control
– State Change Notification
– Bidirectional Mappings Between Fibre Channel and iSCSI Devices

Confidential – For Training Purposes Only 79


iSCSI target discovery

– Before establishing the iSCSI connection, the iSCSI initiator needs to find (discover) targets
to which it has access
– The four discovery methods are:
– iSCSI targets are configured on the initiator
– The initiator uses a configuration file containing the target information
– The iSCSI initiator queries the target
– The initiator issues a SendTargets message to request the list of targets
– The initiator uses the Service Location Protocol (SLP)
– It locates iSCSI targets or SNS without specifying the address
– The initiator queries a Storage Name Server (SNS)
– It locates iSCSI targets without specifying the address

Confidential – For Training Purposes Only 80


iSCSI operations
iSCSI login request to
initiate a session over TCP

iSCSI initiator iSCSI target

Persistent session carrying


the authentication and
exchange of certificates

NOTE: After the persistent state is initialized, iSCSI will use multiple parallel sessions to aggregate bandwidth and improve performance.
The iSCSI session terminates when its TCP session is closed.

Confidential – For Training Purposes Only 81


iSCSI security

Authentication

– iSCSI initiators and targets prove their identity to each other using the
Challenge Handshake Authentication Protocol (CHAP).

Logical network isolation


– This is the deployment architecture, to mitigate the authentication risk.
– It is usually provided through the VLAN capability of network equipment.
Physical network isolation

– This is used to prevent cabling mistakes.

Authorization

– iSCSI aims for storage consolidation. Authentication is used to prevent


unrelated initiators from accessing storage resources.

Confidentiality and integrity

– The IPsec protocol provides standards-based cryptographic protection


for the iSCSI traffic.

Confidential – For Training Purposes Only 82


iSCSI advantages and limitations

iSCSI advantages iSCSI limitations

A separate network for SAN is not required. You can use The IP network is currently a “best effort” network. The
existing IP networks and components. packages might drop or be delivered out of order because
of network congestion.
The iSCSI SAN can coexist with a Fibre Channel-based The server CPU might be burdened with TCP/IP SAN
SAN. traffic.
The iSCSI SAN does not have distance limitations. Running iSCSI on the same network as production might
lead to congestion.
You can use specialized HBAs or standard NICs. iSCSI operates on a clear text protocol, so the traffic must
be encrypted.
iSCSI is suitable for virtualized environments because it
supports software-based initiators.
It provides a means of direct backup to tape or disks,
even from certain virtual servers.

Confidential – For Training Purposes Only 83


Fibre Channel over Ethernet

Confidential – For Training Purposes Only 84


What is FCoE?

– Fibre Channel over Ethernet is a mapping of Fibre Channel over selected full-duplex IEEE 802.3 networks
– The goal is to provide I/O consolidation over Ethernet, reducing network complexity in the data center
– Customer benefits of a unified fabric:
– Fewer NICs, HBAs, and cables
– Lower capital expenditures and operating expenses

Confidential – For Training Purposes Only 85


FCoE I/O consolidation

Confidential – For Training Purposes Only 86


FCoE mapping

– FCoE maps the Fibre Channel commands and data directly into Ethernet frames to create FCoE
– Fibre Channel frames are encapsulated in Ethernet frames

– The mapping is 1:1, meaning there is no segmentation or compression of the Fibre Channel frames
FC – 4 FC – 4
FC Level
FC – 3 FC – 3
(Unchanged)

FC – 2 FC – 2

FC – 1 FCoE mapping
IEEE 802.3
MAC Layers
FC – 0 PHY

Ethernet FCoE FC
Header Header Header SCSI Commands/Data CRC

Confidential – For Training Purposes Only 87


FCoE lossless Ethernet infrastructure

– FCoE has to create a lossless Ethernet environment to ensure the reliability of


large-scale data transportation
– Two standards enable lossless Ethernet
– Data Center Bridging (DCB)
– Converged Enhanced Ethernet (CEE)

– In addition to DCB and CEE, the standard introduces three enhancements to


the Ethernet to make it lossless:
– Priority Flow Control (IEEE 802.1Qbb)
– Congestion Notification (IEEE 802.1Qau)
– Enhanced Transmission Selection (IEEE 802.1Qaz)

Confidential – For Training Purposes Only 88


Priority Flow Control

Priority Flow Control (IEEE 802.1Qbb)


– IEEE 802.1Qbb is an enhanced QoS service
– Traffic is classified in 8 lanes, each of which could be assigned a priority level
– Priority Flow Control issues a “Pause” command to manage and prioritize traffic when there is congestion
– The administrators can create lossless (virtual) lanes for FCoE traffic and lossy (virtual) lanes for normal
IP traffic

Confidential – For Training Purposes Only 89


Congestion Notification

Congestion Notification (IEEE 802.1Qau)


– Congestion is measured at the congestion point, but link rate limiting is taken at the point of origin
– Example: An aggregation switch can ask an edge switch to stop (or limit) its traffic from a particular port, if congestion
occurs

Confidential – For Training Purposes Only 90


Enhanced Transmission Selection

Enhanced Transmission Selection (IEEE 802.1Qaz)


– High-priority traffic such as FCoE is allocated with a minimum guaranteed bandwidth
– If the FCoE traffic does not fully utilize its reserved capacity, the extra bandwidth can be used by other
types of traffic, and this can be controlled dynamically

Confidential – For Training Purposes Only 91


FCoE components

Fiber Channel Fiber Channel


Network (Storage) Network
(Storage)
HBA
1
Converged Network 2 3
HBA
Adapter (CNA)
Converges Network Ethernet Network
Adapter (CNA) (LAN)
NIC
NIC Ethernet
Network (LAN)
FCoE Switch / Ethernet Switch
FCoE Switch / Ethernet
Supporting FCoE Switch
Supporting FCoE

Confidential – For Training Purposes Only 92


FCoE advantages and limitations

FCoE advantages FCoE limitations


FCoE reduces the two network adapters (HBA for The only Ethernet component that is currently compatible
storage connectivity and NIC for network connectivity) with FCoE is the cables.
and two individual cables to just one.
FCoE can carry traffic over the Ethernet medium. The cost of a Unified CNA (although the price is coming
down) might be more than the cost of the HBA and NIC
combined.
Having one network adapter instead of two results in FCoE is currently restricted to access networks only
some power savings for the server. (server-to-switch connections).
FCoE can be used in virtualized environments. Security on FCoE networks might have to be re-evaluated
because the network is now running over Ethernet, which
is more accessible than Fibre Channel.
Unlike iSCSI, FCoE is reliable. It can scale up to
thousands of servers.

Confidential – For Training Purposes Only 93


Thank you
Confidential – For Training Purposes Only 94