Storage Area Networks and Fibre Channel

Steven Wilson Preetham Gopalaswamy

Brocade Communications Systems, Inc.

Todays Topics
Storage Area Networks Fibre Channel Technology Fibre Channel Management Common Information Model (CIM)

The Server-to-Storage Bottleneck

1990s Technology Does Not Scale

What is a SAN?

Open Systems Model for Network Storage Enhanced Storage Management Flexibility to add or reconfigure storage as needed without downtime Independent Scaling of CPU and Storage capacity De-couples servers and storage so that either can be scaled separately Easy Migration Current applications run without software changes Incremental deployment allows flexible adoption

SAN Benefits
Dynamic Allocation of Resources (storage and applications) High Data/Application Availability Non-disruptive Maintenance
Continuous operations if server or storage has to be removed from cluster Add, delete storage on the fly

Cost Savings
Shared Storage

What is Fibre Channel

Open standard, ratified in 1993 Optimized for large block transfer with built-in reliability Distancefor disaster tolerant configurations Independent scaling of servers and storagecreating the virtual private data center/Virtual Private SANs Adopts legacy environments and applications Concurrency of networking and storage protocols on single NICreducing costs of ownership: SCSI, IP, VI, FICON, Etc. Single technology for server-storage area networking, clustering (server-server)

Fibre Channel Standards Activities

T11 SNIA IETF T10 DMTF FCIA
Fibre Channel transport,Topology, Generic Services, physical, media standards SAN Application,discovery,security, management Standards. Not yet an accredited standards body, but provides input to other standards bodies IP related standards and MIBs and Storage over IP efforts SCSI storage protocols for Fibre Channel and others Fibre Channel work group, Common Information Model (CIM) Fibre Channel technology road maps, interoperability specifications and plug-fests

Important T11 Standards

FC-FS Framing and Signaling, Replaces FC-PH, FC-PH-2, and FC-PH-3 FC-SW-2 and FC-SW-3 Switch Fabric Standard, Describes How Switches Communicate with One Another FC-GS-3 and FC-GS-4 Generic Services, Describes the WellKnown Server Architecture and Related Transports FC-BB-2 Backbone, Describes How Fibre Channel Frames are Transported Over WAN Connections FC-MI Methodologies for Interconnects, Interoperability Profile, FC HBA API FC-SP Security Protocols, Authentication, Authorization, Policy Management, Confidentiality

Fibre Channel - Hybrid Transport System -

Multiple Protocols On Common Fibre Channel Transport

Multiple Mapping Standards

Streams Transfer

FICON

VI CT

Single Transport Standard 10 GB

FC-0
Physical Variants

Optical (Laser, LED) Copper (Coax, Twisted Pair) Single Mode vs Multi-mode Fibre

FC-1
8B10B Encoding Running Disparity

Ensures Virtually An Equal Number of 1s and 0s

DC-Balanced

Facilitates Amplifier Design Lower Power Ensures Synchronization For Clocking Purposes
IBM Holds the Patent

Fibre Channel Information Transfer FC-2 LayerFraming and Protocol

Sequence 3 Sequence 2 Packets (Large Blocks) Sequence 1 Exchange (Protocol Operation)

Sequence

Buffer
S O F

Buffer
Data C R C E O F (server/storage/WS) Device 2

Header

FC Frame (Max. 2112 Bytes) (server/storage/WS)

Device 1

FC-4 Mappings
Maps Upper Level Protocols to Fibre Channel Examples are SCSI, IP, VI, FICON FC-CT is Mapped for Inband Management Use

Fibre Channel Ports and Nodes

N_Port, NL_Port, F_Port, FL_Port, E_Port, B_Port Each Nx_Port Has a Fabric Unique 24 Bit Address Each Nx_Port Has a Unique WWN Nx_Ports Must Login With One Another Prior to Data Transfer When a Fabric Exists, Ports Also Login to the Fabric A Node is a Collection of Ports Each Node Has a Unique WWN

Data Transport Services

- Meet Different Application Needs Class 1
Service characteristics

Class 2

Class 3

Dedicated, full bandwidth Fixed routing Guaranteed inorder frame delivery Confirmation of both delivery and nondelivery of frames End-to-end flow control

Multiplexed frame delivery at the frame boundary Adaptive routing No guaranteed inorder frame delivery Confirmation of both delivery and nondelivery of frames Both end-to-end and link-level flow control

Multiplexed, frame delivery at the frame boundary Datagram service No confirmation of delivery or non-delivery of frames Only link-level flow control

Communication mode

Connectionoriented Circuit-switched

Connectionless Frame-switched

Connectionless Frame-switched

Fibre Channel Topologies

Point-to-pointtwo devices connected together

Fibre Channel Topologies

Arbitrated loop

Up to 126 devices on a shared

media for small systems at reduced cost and reduced performance level

Fibre Channel Topologiesthe Fabric

Large connectivity on non-shared media, which allows concurrent communicating pairs Highest performance level High scalability Good fault isolation Embedded management and services

Fibre Channel Services

Login Server Fabric Controller Common Transport
Name Server Alias Server Time Server Management Server

Port Interfaces
NL_Port

F_Port

N_Port

Point-to-Point Remote Connection between Fibre Channel Systems Through WAN

Fibre Channel switch Fibre Channel switch Fibre Channel switch

WTU

ATM/ SONET/IP network

WTU

Fibre Channel switch

FC-BB-2 and FCIP Standards Remote Backup sRemote Mirroring sDisaster Recovery
s

WTU: Wan Tunneling Unit

Characteristics of FC Switches
Switches Connect to One Another Using E_Ports and Inter Switch Links (ISLs) Switches Route Frames Based on the 24 Bit DID

DID Consists of: 8 Bit Domain ID, 8 Bit Area ID, and 8 Bit Port ID Each Switch Has a Unique Domain ID (239 Max)

Switch to Switch Communication Uses Class F Which is Similar to Class 2 Switches Implement a Fabric Controller and other Well-Known Servers Switches Allow Inband and Out-Band Management

Functions of the Fabric

Switch Port Initialization Fabric Configuration FSPF Zoning Distributed Server RSCN

Zoning
Similar To VLANs in the Networking World Provides an Access Control Mechanism Allows End-Devices to Only Communicate With End-Devices in the Same Zone Two Types of Enforcement

Hard Soft
Affects the Discovery Process May Eventually be Applied to Resources Behind the N_Port (e.g. LUNs)

Zoning Structure
Z o n e
A Z o n in g O

S e t

t t r ib u t e s :

O M

n e o r o r e A t tri b u te s :

o n e ( s )

M O n e o Z o n e r M o r e A ttr ib u te s :

e m

b e r s

b je c t T y p e a m O e O b je c t s

h e x 0 1 Z o n e S e t N N u m b e r L is t o f Z o n e

Z o n in g O b je c t T y p e h e x 0 2 P ro t o c o l Z o n e N u m b e L is t o f

o f Z o n e

b j e cF t l sa g s

Z o n e M e m b e r T y p e 0 1 N - _P o _ t Na m e r 02 - D o m a i n _ I D a n d P o r t 0 3 N- _ P o r t _ I D N a m e 04 - Z o n e A l i a s N a m e r o f Z o n e M e m b e r s F la g s Z o n e M e m b e rs L e n g th Z o n e M e m d be e rt if ie r I n

Z o n A e l i a( s s )
A Z h Z N L

Z o n e

A l ia s

e m

b e r s

A t t r ib u t e s : ttr ib u te s : Z o n e A lia s M e m b e r T y p e o n in g O b je c t T y p e 0 1 N - _ o r_ t Na m e P e x 0 3 02 - D o m a i n _ I D a n d P o r t o n e A lia s N a m e u m b e r o f Z o n e A l i a s O Mn ee m o b r e0 r 3s N - _ P o r t _ I D F la g s M i s t o f Z o n e A l i a s M e m bo e r re s L e n hg t Z o n e A l ia s M I d e n bt ie f i re r e m

Fibre Channel Generic Services

Management Service

Simple Name Service

Discovery Service

Registration and discovery of Switches, Fabric ports and their attributes. Configuration Management Fabric Device Management Zone Management Integrated with Fabric and distributed
Node

Registry and directory service to discover nodes and their attributes (connected to Fabric) Integrated with Fabric and Distributed Operational in Nature
Node N_Port

Discovery of Physical topology Discovery of Logical association between devices Acquires topology information from Simple Name Service and Management service

Node

Fibre Channel Fabric (Distributed Services)

Node Node

Node N_Port

FC-GS-3 FC-GS-4 Standards

Node

N_Port Node

Brocade Software Stack

Management Applications Management Services Critical Services
Manageability Services and Applications

S A M S

Core Services Kernel Drivers OS Kernel

Fabric Services Group

Platform Group

Hardware

Traditional Fibre Channel Management

Common Transport (FC-CT)

Standardized Native Inband Management Proprietary In-band Management Out of Band Proprietary Management Standardized (Almost) Out of Band Management Proprietary Out of Band Management Proprietary Management Out of Band Management

IP over FC: WEB Based Management SNMP Telnet/CLI API Based Management

Post Modern Fibre Channel Management

Object Based Management CIM/WBEM (Web-based Enterprise Management) Combines Management Capabilities Exposed Through Other Interfaces CIM Capabilities Can be Provided By Agents or Integrated Directly Into SAN Products

Managing Brocade Fabrics

WebTools
SNMP API

Fabric Manager

Management Agent Brocade Data Model

Fabric

RPCd

SNMPd

HTTPd

Switch Unified Data Access Layer

CIM Common Information Model

It is a Data Model, not an implementation There are two parts to CIM
The CIM Specification CIM Schema

CIM Specification (currently 2.2) describes the language, naming, Meta Schema (a formal definition of the model) Formal definition of the CIM Schema is expressed in a Managed Object File (MOF)

Everything about CIM that you really did not want to ask

Everything about CIM that you really did not want to ask
Objects have inheritance
Abstract and Concrete

Objects have methods: Intrinsic and Extrinsic Objects can maintain backward compatibility and support deprecation Associations are objects which means they can have properties Events are Objects CIM Schema provides the actual model descriptions
Core Schema Common Schema (System, Network, Devices,) Extension Schema

CIM 2.7 has simplified the Fibre Channel model and added Zoning

Everything about CIM that you really did not want to ask
The methods supported by a class vary with what interfaces/interface methods a given provider supports for that class. Instance Provider deleteInstance createInstance getInstance setInstance enumerateInstanceNames enumerateInstances PropertyProvider setProperty getProperty Method Provider invokeMethod Query Provider invokeQuery

CIM-based Management Solution

Access Protocol and Data Format Customer Deliverables
Client I/F

Data Information Model

Agent

Higher Level Services

Object Manager
Provider I/F

Device Provider
Fabric Data Access Fabric Layer

Object Model

Peeking under the hood

To Provider
Host Fabric

Http Daemon

RPC Daemon

SNMP Agent

Name Server Zoning Routing/Pathing

Fabric Watch Perf. Monitoring

Mgmt. Server

Why move to a CIM-agent solution?

Looser linkage between application and Brocade library
Less integration headaches More flexible application interfaces (i.e. java and other compilation environments) Agent can support multiple applications simultaneously

Delivers a standards-compatible interface (e.g. CIM)

Questions

Thank You!