FIBRE CHANNEL

MODULE-2

FIBRE CHANNEL

• Fibre Channel, or FC, is a gigabit-speed network technology primarily used for storage networking to perform serial data transfer.

• Fibre channel includes the advantages of channel as well as network

i.e., the speed of channel and scalability of network.

• Fibre Channel provides a standard data transport frame into which multiple protocol types can be encapsulated.

WHAT IS FABRIC?

• Fabric is a well designed, intelligent and self configurable network that follows FC standards.

• It is called self configurable because the ports of fabric are intelligent and software specific, they configure themselves according to the devices connected to them.

• Fabric comprises of switches, routers and gateway devices which make the fabric an intelligent network.

• In a single fabric 1 to 239 switches can be kept i.e., 1 to 239 domains as each switch has its own domain ID.

• Fabric can also include hubs and tape libraries.

HOW IS FABRIC SELF-CONFIGURABLE?

• To interface fabric to real world port should be software specific, depending on the device the ports are self configurable.

Ex: if loop devices like jbod, hub, tape library are connected to fabric i.e., switch, the port configures it self to loop port ,it displays FL (Fabric Loop) near switch port and NL near the port of Jbod.

• If a host is connected to the fabric it configure itself as fabric port and it shows F port near the switch and N port near the host.

• If a port is connected to storage sub systems it configure it self as fabric port ,it displays F port near switch port and N port near the storage sub system.

• If a switch is connected to fabric, it configures it self to E port or expansion port near the ports of both the switches.

Host

File I/O

FABRICFABRIC

N

F

JBOD

EX

EE

E

F

F

NL

NL

FL

FL

FC Router

Hub

FC switch

Storage

Ports of fabric

Fibre Channel Topologies

FIBRE CHANNEL TOPOLOGIES

There are three types of fibre channel topologies.

1. Point to Point

2. FC-AL (Fibre Channel Arbitrated Loop)

3. Switched Fabric

Point to Point Topology

• One to One communication (not in use now a days)

• Every point has two nodes -Transmitter -Receiver

• No address required

• Always linked across transmitter to receiver

• No multi sharing

Point to Point Topology

Transmitter

Receiver

Transmitter

Receiver

A B

FC-AL (Fibre Channel –Arbitrated Loop)

• All devices are connected serially in a loop or ring, similar to token ring networking.

• Every device has Transmitter and Receiver

• Only two devices can communicate at a time, rest are blocked. This is called blocking technology

• The failure of one device causes a break in the ring.

• Fibre Channel hubs exist to connect multiple devices together and may bypass failed ports.

Transmitter

Tra

nsm

itter

Re

ceiv

er

Receiver

TransmitterReceiver

Re

ceive

r

Tra

nsm

itte

rFC-AL Topology

D

C

BA

FC-AL Hub & Addressing

• In FC-AL topology the multiple devices are connected through hub, the hub is a FC-AL device which has transmitter and receiver looped internally, if any port is blocked, the loop opens and bypasses the port for uninterrupted communication.

• THE FC-AL follows 8 bit pattern of Addressing called AL-PA 2 = 256 (00 – ff)

• Only 127 are supported as the rest are affected by ‘+’ and ‘-’ volts

• Address 00 has the highest priority and reserved for FL port and the rest are 126 ports, to which devices can be connected.

• Bandwidth is shared and arbitrated among the devices based on time slices.

• As the devices increase the bandwidth reduces.

Switched-Fabric

• All devices or loops of devices are connected to Fibre Channel switches

• Basic topology followed is star topology, that enables any –to –any communication.

• Band width is aggregated and utilized efficiently.

• The switches manage the state of the fabric, providing optimized interconnections • Multiple pairs of ports may communicate simultaneously.

• Failure of a port is isolated and should not affect operation of other ports.

• Switched Fabric is widely in use, and covers 85% of the industry

FC-Switch

Switched Fabric

Switched Fabric Addressing

• 1 – 239 switches can be placed in a fabric

• Every switch is identified by a domain number, and up to 239 domains are available in a fabric.

• The addressing pattern followed in a switch is 24 bit addressing called PID (Port Identifier) 224 =16 million

• Up to 16 million devices can be connected to the fabric

• Practically 16000 devices are connected with in a fabric.

Differences between FC-Topologies

Attribute Point to Point Arbitrated Loop Switched Fabric

Max Ports 2 127 ~16777216 (224)

Address N/A 8 bit AL-PA 24 bit PID

Side effect of port failure

N/A Loop fails

(until port bypassed)N/A

Mixing different

link rates N/A

No Yes

Frame delivery In order Not guaranteed Not guaranteed

Access to medium Dedicated Arbitrated Dedicated

Fibre Channel Address

Fibre Channel -Address

• In a fabric 239 switches can be connected ,every switch connects several number of devices to the fabric.

• Address pattern followed in Switched fabric is 24 bit address, which can accommodate addresses to16 million devices .

• The addressing method in Fibre channel almost resembles TCP/IP addressing.• In TCP/IP two types of Addresses are available

- Static Address: Media Access Control address (MAC) Address

MAC address is an unique 48 bit physical address

assigned to the device by the manufacturers as directed

by IEEE.

- Dynamic Address: IP Address (Internet Protocol Address)

IP address is 32 bit address bound to the MAC address

of the device, it changes according to the requirements

of network connectivity.

Cont’d

• As there are MAC ID and IP address for TCP/IP networks,Fibre channel network has WWN (World Wide Name) an unique identifier as MAC Address and PID (Port Identifier) as IP address which can be altered depending upon its connectivity.

• WWN (world wide name) is a 64 bit hexa - decimal unique identifier assigned to the device by its manufacturer, and it is static.

• PID (Port Identifier) is a 24 bit identifier assigned to the device connected to the fabric, and it can be changed when the devices connected to fabric are changed to another switch or domain

• WWN is mapped to PID ,to establish connectivity.

WWN (World Wide Name) WWN is a 64 bit address, It is classified into two types.

1)WWNN : World Wide Node Name

2)WWPN : World wide Port Name

WWNN WWPN

WWN

World Wide Node Name (WWNN)

• WWNN is an 64 bit unique identifier assigned to a device.

• WWNN is divided into 8 octets

• Each octet is represented in Hexa – decimal pattern 00 – ff

Ex: 00:ff:de:12:48:ce:12:00

WWNN is classified as below

5 0 0 0 a b f e 1 c 2 0 3 0 0 1: : : : : : :

NAA

Future Purpose

IEEE

vendor

WWNN Cont’d

• First field in the WWNN represents the device ,as specified by NAA (Name Assigned Authority)

NAA specifications to identify devices

• Next three fields 2,3& 4 are reserved for future purpose• Next three octets are reserved for IEEE• Last three octets are reserved for vendor

DEVICES I DENTIFIER

Storage sub system 5

Switch 1

HBA 2

World Wide Port Name(WWPN)

• WWPN is a 64-bit unique identifier assigned to a port on device.

• WWPN is divided into 8 octets • Each octet is represented in Hexa – decimal pattern 00 – ff Ex: 20:04:de:12:48:ce:12:00 WWPN is classified as below

2 0 0 4 a b f e 1 c 2 0 3 0 0 1: : : : : : :

NAA

Port Address

EX 4th port

IEEE

vendor

WWPN Cont’d

• First field in the WWPN represents the device ,as specified by NAA (Name Assigned Authority)

NAA specifications to identify devices

• Next three fields 2,3& 4 are reserved for addressing ports.• Next three octets are reserved for IEEE• Last three octets are reserved for vendor• On a devices there can be many WWPNS ,but only one WWNN is available• To enforce restrictions on port ,WWPN is used

DEVICES I DENTIFIER

Storage sub system 5

Switch 2

HBA 1

WWNN &WWPNS

• There are two devices shown once FC-Switch and HBA card.

• Switch has 16 ports and HBA has one port

• HBA is assigned with 1 WWNN and 1WWPN

• Switch is assigned with 1 WWNN and 16 WWPN s.

PID (Port Identifier)

• PID is a 24 bit dynamic address assigned to the port of the device connected to the fabric.

• Dynamic address is divided into three octets

- XX – Domain ID

- YY – Port Number

- ZZ – AL-PA

• In fabric AL-PA is always ’00’.

X X Y Y Z Z

0 04 0 0 0 0 4 0 1 0 0

0 1 2 3 4 5 6 7

Domain ID 04

N

F

N

F

NL

FL 0 4 0030

0 1 2 3 4 5 6 7

0 10 0 0 0

JBOD

50 000 0

FC-Switch

FC-AL Hub

Port Identifier (PID)

• There are two types of PID’s

• If a device is attached to fabric through FC-Switch, the device is assigned with Public

Id

EX: (AL-PA octet shows 00)

• If a device is connected to fabric through FC-AL hub, the device is identified by Private Id

EX: (Domain ID & Port Number shows 00 but AL-PA shows 04)

Port Identifier

Public Id Private Id

0 04 0 0 0

0 40 0 0 0

Translative

• Host attached to the fc-switch in a fabric is identified by Public Id.

e.g.

• Jbod connected to the fabric through FC-Al hub is identified by Private Id.

e.g.

• To establish connectivity between host and Jbod, Translative address should be used.

-Translative-Public address translated to Private address.

Example:

0 04 0 4 0

0 40 0 0 0

0 04 0 4 0

0 40 0 0 0

0 44 0 4 0 Translative Address

Modes of PID• In Brocade switches PID is in two modes

PID

Core PID Native PID

x zx y y z zx y1x z

0 04 0 4 0 0 04 1 4 0Example

Cont’d• In core PID both the characters in 2 octet shows “yy” i.e., the octet supports

up to 256 ports (00 – ff), The core PID mode is suitable for latest switches with more number of ports.

• In Native PID out of two character in second octet first character shows 1 and next character shows y i.e,this mode supports only 16 ports (10 – 1f),The native mode is suitable for IInd generation and older switches ,in which the maximum ports are 16.

• To reduce the loop over head ,the first field in second octet of Native PID mode is hard coated as 1.

• Once the switch in a fabric is configured in Native PID mode ,the entire switches in the fabric should be configured to Native mode. So with the Core PID mode

Well Known address in switch level for well known Services

In brocade point of View the well known addresses for well known fabric services are as follows

- Flogi

- Plogi

- Prlogi

- SCN

- RSCN

Well Known address in switch level for well known Services

Login Server Login Server

Name Server

Fabric controller

Prlogi

Plogi

Flogi Flogi

SCN

RSCN

Well Known Services

• Flogi (Fabric login): Once the host is connected it doesn’t communicate with switch unless login is authenticated and configured, this process runs as follows

- Host sends packet to login server in Switch - Login server checks the parameters and addresses - Acknowledgement sent to the Host - Host is assigned with 24 bit address (PID) The above process is called ‘Flogi’.

• Plogi (Port login): To access one host from another host in the fabric, the process runs as follows

- Let us take an example ,we have two hosts (A&B) connected to two different switches in a fabric.

- A & B updates their respective information to name server - Once updated A & B can identify and communicate with each other - The Plogi is accomplished

Cont’d

• Prlogi (Process Logi): To access the SCSI disks in the fabric

• SCN (State Change Notification) : Any internal changes occurs are updated to fabric controller, through State Change Notification (SCN).

• RSCN (Registered State Change Notification): The changes are updated and notified to the registered nodes, through registered state change notification (RSCN).

FC-LAYERS

FC - LAYERS

FC-0

FC-1

FC-2

FC-3

FC-4

PHYSICAL LAYER

UPPER LAYER PROTOCOL MAPPING

FRAMING & FLOW CONTROL

COMMON SERVICES

8/10 BIT ENCODING & DECODING

FC – 0 (PHYSICAL LAYER)

• FC-0:- A physical layer that represents speeds & feeds

• SPEEDS : fibre channel supports the high speed data transfer, the speeds available in the industry are 1Gbps, 2Gbps, 4 Gbps, 8Gbps.

• FEEDS : to support the high speed data transfers feeds are required, feeds are none other than cables.

• Cables are of two types, they are multi mode and single mode

• The components of cable are

1. cover

2. clad

3. core

• These are the components of optical fibre cables in which data transfers in form of light rays.

COMPONENTS OF CABLE

• Cover : The upper layer of the cable is called cover

• Clad : clad is a thin glass rod

• Core : an aperture made at the center of the glass rod creating a tunnel

to transmit data in form of light rays

COVER

CORE

CLAD

MULTI MODE CABLE

Multi mode cable - supports multiple light incidence as the size of core is bigger

- cover of the cable is orange

- supports shorter distances up to 250 meters

- there are two sizes in core 50 µ and 62.5 µ

- if size of the core is 62.5 µ that supports 200 meters of cable length

- if size of the core is 50 µ that supports 250 meters of cable length

- size of the clad is 125 µ

- size of the cable is presented as follows : CORE/CLAD = 62.5/125 µ

CONT’D

If the core size increases the angle of light incidence is high and increase in number of reflections, Increase in reflections of light ray results in absorption of light, consequently wave length reduces and distance traveled by light ray decreases.

LIGHT RAY

CORE

SINGLE MODE

• Supports single light incidence as the size of the core is small

• Colour of the cover is yellow

• The size of the core is 9 µ

• The size of the clad is 125 µ

• The size of the cable is represented as follows 9/125 µ

• In both the modes i.e., multi mode and single mode the size of the clad is same but there is remarkable difference between the sizes of core.

• Single mode cable supports up to 5 kms

CONT’D

• As the size of core is smaller, the angle of light incidence is low and number of reflections are less. so the wavelength is constant and less attenuations as the light is less absorbed , consequently the light ray travels more distance.

LIGHT RAYCORE

CONNECTORS

• Cables are connected to connectors

• there are two types of connectors for optical fibre cables in san environment

- SC – Siemens Connectors

- LC – Lucent Connectors

• If copper cable is used the connector is DB-9

Siemens Connector Lucent Connectors

Adapters• Siemens connectors are connected to the ports through SC Adaptors - Siemens adaptors are called GBIC (Gigabit interface card) - GBICs occupy 1 inch space on switch - Maximum speed supported by GBIC 1 Gbps - Siemens connectors are out-dated• Lucent connectors are connected to ports through LC Adaptors - Lucent adaptor is called SFP (Small Form Pluggable)

- SFPs occupy 1/2 inch space on switch - 1Gbps – 8 Gbps speeds can be supported

GBIC SFP .

Couplers

• To extend the cable distance couplers are required

- LC to LC - SFP to SFP SC coupler

- LC to SC - SFP to GBIC

- SC to SC - GBIC to GBIC

LC coupler

MIA –Media Interface adapter converts electronic signals to Optic signals and converts DB-9 to Siemens Connector.

Media Interface Adapter

FC-1 (8/10 bit encoding and decoding)

• FC-1 8/10 bit encoding and decoding

• The prime function of this layer is error detection and correction at data transfer level.

• This concept is follows a thumb rule:

LHS = RHS

• Communication is done in bits, for every 8 bits another 2 bits are added and 10 bits are generated called character

• The 2 bits serves as parity bits for each 4 bits in a character.

PPData Bit

Parity Bit

FC-1 CONT’D• The 10 bits information is transmitted through fibre channel. where there

can be disparity.

• There are three types of disparities

1) Positive(+) disparity

2) Negative(-) disparity

3) Neutral(=) disparity

• In a communication stream every 10 bits comprise of “Zeros(0)” and “Ones(1)”,

• out of these 10 bits

- Number of ‘1’ s are more than 5 bits it is Positive(+) disparity

- Number of ‘0’ s are more than 5 bits it is Negative(-) disparity

- Number of ‘0’ s and ‘1’ s are equal then it is Neutral disparity

FC-1 CONT’D• Across the fibre channel neutral disparity should be maintained, as the

(+)Positive disparity is afflicted by (-)Negative volts and (-)Negative disparity is afflicted by (+)Positive volts.

• To monitor the flow of characters and control the flow of (+)Positive and (-)Negative disparity across the communication channel a couple of protocols are developed they are:

- PSM (Port State Machine) for switch

- LPSM (Loop Port State Machine) for hub

• PSM & LPSM follows K28-5 algorithm to monitor and manage the disparity levels across the link.

FC-1 CONT’D

• 10 bits information generated including the parity is called character in fibre channel communication.

• Character are punctuated by primitives generated by PSM & LPSM in the link.

• In fibre channel communication the size of each frame is 2112 bytes,and each frame is set with delimiters called (SOF) start of frame and (EOF) end of frame.

• These delimiters are assigned by PSM and LPSM

Frame

2112 bytesS

O

F

E

O

F

FC-1 Transaction Word

• In fibre channel the communication is done in Transaction words, which comprises of 4 characters i.e., 40 bytes.

• To send data across the communication channel data as well as control information are required. Transaction word includes both.

• Transaction word consists of Data word and Ordered set - Data word is the information to be sent across the link. - Ordered set is the control information maintained to transfer the data. -Any packet that starts with ‘ k ’ is called control packet -Any packet that starts with ‘ D ’ is called data packet• Ordered set contains: - Delimiters - Primitive Signals - Primitive Sequences

Transaction Word

Transaction Word (40 bits)

Ordered set Data word

DelimitersPrimitive Signals Primitives

Start of Frame

End of Frame

Fill Words

Non Fill words

Sequence

Controlled byPSM & LPSM

Transaction Word cont’d

• Delimiters: Start of frame and End of frame

• Primitive Signal Events:

Fill words: If there is no data in the link and the port is operational, PSM/LPSM generates a packet called ‘idle’ ‘idle’ ‘idle’ ‘idle’.If the data is arbitrated ‘Arb’ ‘Arb’ ‘Arb’ packet is generated

Non Fill words: Signals the events like (R_RDY,VC_RDY,OPN,CLS etc,)

• Primitives

Sequence Signals: No signal packets Non operational ‘NOS’, if the ‘NOS’ signal is sent three times then it notifies that the link is down. and also include signals like (OLS,LRR,LIP etc.) any signal need to be transmitted thrice to know the appropriate status of port.

• PSM & LPSM controls the signals over the links ,PSM in fabric and LPSM in FC-AL networks

FC-2 Framing and Flow control

• FC-2 Framing and flow control defines the structure and organization of the information being delivered and the methods to control and manage information delivery.

• Framing and flow control includes:

- Exchange and sequence management

- Frame structure

- Flow control

- Class of service

Exchange sequence and frame management

SCSIRead

SCSIWrite

SCSIUpdate

FC-4 Mapping Upper Layer Protocol

1 Read 2.Write 3.Update

1/21/1 1/3 1/4 2/1 2/2 2/3 2/4 3/43/33/23/1

FC-1

FC-0

Single Exchange

Multiple Sequence

Frames

Port

FC-2

Exchange & sequence management`

• Fibre Channel, there are almost no limits on the size of transfers between applications.

• All commands coming from Upper layer protocols are mapped into a logical constructs called “Information Units”.

• Exchange management is the mechanism that two fibre channel ports use to identify and assign an exchange ID number for a set of related information units.

• If the entire stream of data fits in a single frame (2112 bytes) a single exchange id is created and a sequence number is assigned else data is put in a sequence of frames.

Exchange & sequence management CONT’D

• Within the exchange ID sequence management is used to number the sequence segments in the stream of data.

• Sequence numbers associated with the exchange ID will be used at the recipient to re-order the sequence segments, to re-assemble as a contiguous stream of data.

• Frame Structure has a start-of-frame delimiter ordered set and ends with an end-of-frame delimiter set.

• In other protocols, this is commonly known as fragmentation and re-assembly.

Frame structure

• Frame has a header and pay load • Header contain control info and addressing info associated with frame

• Payload contains information to be transported by the frame.

• There are many different payload formats, based on the protocol.

• Total size of payload amounts to 2112 bytes.

• Total size of the frame is 2148 bytes, as control information occupies 36 bytes apart from payload.

• The routing control INFO defines the content of the frame or identify the function of the frame.

PayloadHeader

SOF

EOF

CSC

4 24 2112 4 4

FRAME

SOF: Start Of Frame

Header: The control Information of frames.

CSC (Class Specific Control): The control on class of services used in frame transfer.

EOF: End Of Frame

Flow Control

• Flow control is the process to deliver a frame.

• Frame is sent thru the encoder (8b/10b), to the serializer (sfp/gbic) and transmitted to the receiver port where it is deserialized, decoded and stored in a receive buffer.

• The receiving port sends to the transmitting port a credit to send another frame and decrements a credit from the credit value established during the login session (buffer to buffer credit).

• Buffer credits regulate the flow of frames into and out of the fabric.

• End-to-end credit is established between the pairs and is used to manage the flow of frames between a specific pair of N_ports

Class of services

• Flow control is managed using class of fibre channel services.

• There are six class of fibre channel services

1) Class 1 – connection oriented: port-to-port, with ack (acknowledge), guaranteed bandwidth, in-order delivery (IOD) A circuit-switching COS

One One

2)Class 2 – connectionless: end-to-end (EE_Credit) / ack and buffer to buffer / receiver ready (BB_Credit/r_rdy)flow control

3)Class 3 – connectionless: no ack, BB_Credit/r_rdy flow control only - Errors are handled at higher level

4)Class 4 – connection oriented with ack: Creates Virtual Circuits with dedicated fractional bandwidths. Bandwidth and latency handled by a Quality of Service parameter that is IOD guaranteed.

5) Class 6 – behaves like Class 1 with one many capabilities.

6) Class F – connectionless: switch-to-switch, with ack and BB_Credit ready.

COS cont’d

• Out of six class of services cos 1,4 & 6 are used for telecommunications\

• COS 2,3 & f is used by Brocade switches

• COS 2 and F use R_RDY and ACK (end to end or EE_Credit) flow control, each ACK received increments EE_Credit value

• COS 3 uses R_RDY (buffer to buffer or BB_Credit) flow control, each R_RDY received increments BB_Credit value

• COS F – connectionless: switch-to-switch, with ack and BB_Credit ready.

COS -2• Class 2 Service

• Each frame routed separately by Fabric

• If multiple routes supported, frames might be delivered out of order

• Confirmation of delivered frames

• Connectionless service with no turnaround delay to establish connection

• End-to-End (EE) delivery confirmation with ACK

• Buffer-to-Buffer (BB) Link level flow control

• Notification of frame delivery failure

End –to –End credit/Ack

FC-Switch FC-Switch

STORAGE

2 2

Buffer Buffer

Ack

ENDEND

12

Credit Value

Once the frame reaches the destination,Acknowledgement(Ack) is sent from destination to source, as the acknowledgement is received the credit value is incremented from ‘1’ to ‘2’ and the next frame is transmitted, suppose the next frame was failed to deliver ,no acknowledgement is sent and the credit value is decreased at the transmission end.

COS-3/ Buffer to Buffer credit/Receiver ready

Class 3 Service

• Each frame routed separately by Fabric

• If multiple routes supported, frames might be delivered out of order

• Unconfirmed delivery of frames

• Datagram service

• Connectionless service with no turnaround delay to establish connection

• Buffer-to-Buffer (BB) Link level flow control

• ULP recovery from frame delivery failures

Buffer to Buffer credit/Receiver ready

• Frames are moved from one buffer to another using Receiver Ready (R_Rdy) primitive signals

• Frame flow is always from the source buffer to the destination buffer

• Multiple intermediate buffers may be involved

• COS 3 uses R_RDY (buffer to buffer or BB_Credit) flow control, each R_RDY received increments BB_Credit value

.

• Flow control is dependent on class of service (COS), but most use BB_Credits

Buffer to Buffer credit/Receiver ready

STORAGE

frame3

frame2frame1

frame4

frame4

frame5

Buffer

Buffer

Receiver Ready

Frame transfer between two buffers with different capacities in a fabric, takes place .once the buffer at the destination is emptied and R_Ready signal is raised, once the signal is raised, the buffer at the source, releases frames to the buffer on the destination. Frame flow is controlled by the receiver as a back-pressure. This is called “Receiver pack pressure mechanism”.

FC-3: Common Services

• This layer defines advanced features such as striping (to transmit one data unit across multiple links)

• multicast (to transmit a single transmission to multiple destinations) and hunt group (mapping multiple ports to a single node).

• while the vital layer of communication FC-2 level concerns itself with the definition of functions with a single port. The FC-3 level deals with functions that span multiple ports. So this layer is theoretic but not in use.

FC-4 Upper Level Protocol (ULP) Mapping

• Maps the upper layer protocols and collects information units to be transmitted.

• The information units are forwarded next layers for further process.

• The information Unit can be

- Audio/Visual

- Channels

- Networks

• Brocade supports SCSI and IP

SCSI IP ATM SONNET HPPI

Thank you