EoS

Yaakov (J) Stein

Chief Scientist
RAD Data Communications

Course Outline
1) Introduction
2) Background - Ethernet
3) Background HDLC
4) Background - PPP
5) Background - SONET/SDH
6) VCAT
7) LCAS
8) POS (PPP over SONET/SDH RFC 1619/2615)
9) LAPS
10) GFP
11) Alternatives
Y(J)S EoS Slide 2

Introduction

Y(J)S EoS Slide 3

Motivation
Assume that you are a traditional operator

You have an extensive SONET/SDH network

This network has cost you Millions-Billions to build
This network is highly reliable
Your staff is well trained to maintain it
You may have not yet reached Return On Investment
It supports the service that brings the most revenue voice
It supports the service with the highest margin leased lines

But suddenly customers are asking for something new

Ethernet handoff

And new competitors are willing to supply it!

Y(J)S EoS Slide 4

Option 1: install new infrastructure

You may choose to build a new IP/MPLS based network (BT 21CN approach)
Yes this means significant investment, but this is definitely the future!
But SONET/SDH has comparative advantages:
Reliable optical transport
Well known technology and protocols
Ubiquitous with present operators
Many supported data rates (from 1 Mbps to many Gbps)
Low overhead
Strong OAM (MPLS isnt there yet )
So if you replace the existing network
How will you handle the service that brings your main income voice ?
You may lose your existing leased line customers
You will need to solve the timing distribution problem
And if you keep your existing network
You need to maintain two completely different networks !
This sounds problematic !
Y(J)S EoS Slide 5

Option 2: leased lines

Ethernet
Switch

I
W
F

A
D
M

SONET
RING

A
D
M

I
W
F

Ethernet
Switch

You can try to convince these customers to use leased lines

The customer converts traffic into T1/E1 (e.g. by using frame relay)

You can supply this service now

The major expense is for the customer (who needs FRAD, CSU/DSU, etc.)
Leased lines are profitable

But this only worked before the new competitors appeared

You will probably lose these customers !

Y(J)S EoS Slide 6

Option 3: ATM
Ethernet
Switch

A
T
M

A
D
M

SONET
RING

A
D
M

A
T
M

Ethernet
Switch

You can offer ATM service

The customer converts traffic into ATM (AAL5)
You can supply this service now
ATM is a well-known technology
ATM is a reliable and high-quality service
ATM maps efficiently onto SONET/SDH
You may even be able to perform the conversion at your POP
(but Ethernet is notoriously hard to transport over distances)

But ATM has its disadvantages

ATM has high overhead but you can only charge for user BW
ATM is an additional network

you will have to train and pay new staff

maintain another operations center
ATM usually carries IP, not native Ethernet traffic
Y(J)S EoS Slide 7

Option 4: EoS
Ethernet
Switch

I
W
F

SONET
RING

I
W
F

Ethernet
Switch

A new choice is Ethernet over SONET/SDH (EoS)

The customers Ethernet traffic is transported directly by SONET/SDH
You build on your existing network
You transport native Ethernet
neednt route at network edges
maintain all Ethernet features
New SONET/SDH features make EoS highly efficient
But EoS and related protocols are new technologies
You may need to upgrade existing equipment
Market hasnt yet stabilized on one technology
So you will probably need to take this course !
Y(J)S EoS Slide 8

Worlds Apart
SONET/SDH is presently the most prevalent transport infrastructure
Ethernet is by far the most popular user data interface
So we need efficient methods for carrying Ethernet over SONET
But Ethernet
comes in bursty frames (packets)
uses basic rates of 10, 100, 1000 Mbps
While SONET/SDH
is constant bit rate
is designed for various rates such as 1.6, 2.176, 6.784 Mbps
So the job isnt easy !
Y(J)S EoS Slide 9

Standards we will encounter

IEEE 802.3 Ethernet
ISO 3309

HDLC

RFC1661

PPP (ex 1548)

RFC1662

PPP in HDLC framing (ex 1549)

RFC2615

PoS (ex 1619)

G.707

SDH (especially the new section 11 VCAT)

G.709

OTN

G.7041

GFP

G.7042

LCAS for SDH

G.7043

VCAT for PDH

X.85

IP over SDH using LAPS

X.86

Ethernet over SDH using LAPS

Y(J)S EoS Slide 10

Background
Ethernet

Y(J)S EoS Slide 11

Ethernet frame
For our purposes, Ethernet is any layer 2 protocol
using 1 of the following frame formats :
64 1518 B
DA (6B)

SA (6B)

T/L (2B)

data (0-1500B)

pad (0-46)

FCS (4B)

68 1522 B
DA(6B)

SA(6B)

VT(2B)

VLAN(2B)

T/L(2B)

data (0-1500B)

pad(0-46)

FCS(4B)

Y(J)S EoS Slide 12

Ethernet frame size

Minimum frame is 64 bytes

Maximum payload was 1500 bytes

and maximum frame was 1522 bytes

802.3as lengthened maximum frame to 2000 bytes

Various physical layer modulations and framing

Rates : 10 Mbps, 100 Mbps, 1 Gbps, 10 Gbps,

Y(J)S EoS Slide 13

Background
HDLC

Y(J)S EoS Slide 14

Packet to bit stream

The first problem in converting Ethernet to TDM:
Ethernet consists of frames carrying packets
TDM is a continuous bit stream
We can convert a sequence of packets into a bit stream
by using an idle code
packet 1

packet 2

packet 3

packet 4

packet 1

packet 2

packet 3

packet 4

For example, we can use a sequence of 1s as idle indication

111111111111111111111110 packet 1 0111111111111111111110 packet 2 011111111111111111111110 01111110 packet 3 01111111111111111

The appearance of a 0 bit indicates that data follows

Y(J)S EoS Slide 15

Packet to bit stream (cont.)

How does the receiver know when to return to idle?
We use a specific flag (HDLC uses hex 7E = 01111110)
We can use the flag as the idle code as well
01111110 01111110 01111110 packet 1 01111110 01111110 01111110 packet 2 01111110 01111110 01111110 01111110 packet 3 01111110

Some implementations allow zero sharing

0111111011111101111110 packet 1 011111101111110 01111110 packet 2 011111101111110 1111110 1111110 packet 3 011111101111110

But the flag must not appear in valid data!

If we have access to the physical layer we can mark there (violations)
Otherwise (we only access bits) we must disallow the idle code
by replacing it with something else

Y(J)S EoS Slide 16

HDLC flags
ISO developed High level Data Link C based on IBMs SDLC
HDLC inputs packets of bytes
HDLC uses hex 7E as its idle code (flag) 01111110
So an idle HDLC stream repeats 7E
01111110 01111110 01111110 packet 1 01111110 01111110 01111110 packet 2 01111110 01111110 01111110 01111110 packet 3 01111110

Alternatively, 1s can be sent as idle, flags as delineators

11111111111111111 01111110 packet 1 01111110 111111111101111110 packet 2 01111110 11111111111111111101111110 packet 3 01111110

There are two methods of disallowing flags

bit stuffing (zero insertion)

byte (octet) stuffing

Y(J)S EoS Slide 17

Bit stuffing / zero insertion

ECMA-40
Whenever the encoder sees 5 successive 1s it appends a 0
thus there are never 6 successive 1s in the data
When the decoder sees 5 successive 1s :
If the next bit is a 0 it is deleted
If the next bit is a 1 then this is the closing flag
Notes:
bit stream length is no longer necessarily divisible by 8
bit stream length is not a priori predictable
worst case expansion is 20%
encoding/decoding is easy in HW, hard in SW

Y(J)S EoS Slide 18

RFC1549

Byte (octet) stuffing

Whenever the encoder sees hex 7E

It replaces it with 7D 5E
Whenever the encoder sees hex 7D
It replaces it with 7D 5D
Optionally other codes (e.g. some under hex 20) can be escaped
Second byte is original with 6th bit complemented (xor with hex 20)
e.g. ^Q = hex 11 7D 31 ^S = hex 13 7D 33
When the receiver sees 7D xx
It replaces it with the original byte (complementing 6th bit)
Notes:
bit stream remains byte oriented
length expansion is typically about 1%, but can range from 0 to 100% !
(there is also a consistent overhead algorithm but not in use)

encoding/decoding is easy in SW
Y(J)S EoS Slide 19

HDLC framing
HDLC frame is bounded by flags, and has a particular structure
flag (8)

address (0/8/16) ctrl (8/16)

data

FCS (16/32) flag (8)

Many variants (SDLC, ISO, LAPB, LAPD, LAPF, LAPS, SS7, PPP-HDLC, Cisco-HDLC, etc)
Address:
There may be no address (e.g. SS7 HDLC)
SDLC always had 8 bit addresses
ISO 3309 HDLC has structured multibyte address
SAPI

C/R EA

EA

Service Access Point Identifier (MSB of SAPI =1 may indicate broadcast/multicast)

EA=1 means 8 bit, EA=0 means extended address
C/R=1 for commands, C/R=0 for responses
The single byte hex FF is recognized as the broadcast address
Y(J)S EoS Slide 20

HDLC control
HDLC networks can be configured:
Balanced all stations have equal responsibility
Unbalanced primary and one or more secondary stations
and HDLC can operate :
Best effort (datagram)
uses Un-numbered (U) frames
Reliable (Asynchronous Balanced Mode)
uses frames with sequence numbers in control field
Information (I) frames (data + acknowledgement)
Supervisory (S) frames (only acknowledgement)
The various frame types are indicated by the control field
which varies widely between different protocols

Y(J)S EoS Slide 21

HDLC FCS
HDLC uses a Frame Check Sequence to detect errors
The FCS is implemented as a shift-register
CRC-16 X16 + X12 + X5 + 1
CRC-32 X32 + X26 + X23 + X22 + X16 + X12 + X11 + X10 + X8 + X7 + X5 + X4 + X2 + X + 1
Some HDLC-based protocols require 32 bit FCS
others allow 16 bit but recommend 32 bit FCS

Y(J)S EoS Slide 22

Background
PPP

Y(J)S EoS Slide 23

Point to Point Protocol (RFC 1661)

PPP is a method for transporting datagrams between 2 peers
over full-duplex, point-to-point data links
for example: short lines, leased lines, dial-up modems
PPP may be used to connect hosts to routers, and routers to routers
PPP is made up of 3 components:

encapsulation method for (multiprotocol) datagrams

Link Control Protocol for establishing, configuring,

and testing data-link connections

Network Control Protocols for establishing

and configuring different network-layer protocols

PPP is a suite containing many protocols

ML-PPP, PPPoE, BAP, BCP, IPCP,
Y(J)S EoS Slide 24

Basic PPP encapsulation (RFC 1661)

protocol (8/16)

information

padding

Encapsulation enables demuxing of different network-layer protocols

Only 1 field needs to be examined for protocol determination
Protocol field obeys ISO 3309 rules:
protocol value must be odd (for EA=1)
if 16-bit, then the LSB of first byte must be zero (for EA=0)
PPP protocol values managed by IANA
(http://www.iana.org/assignments/ppp-numbers)
Padding may be used (e.g. to cause header to fall on 32-bit boundary)
Y(J)S EoS Slide 25

PPP using HDLC framing (RFC 1662)

flag

address

ctrl

protocol

7E

FF

03

(8/16b)

information

padding

FCS

flag

(optional)

(16/32b)

7E

When using PPP over synchronous links

we use HDLC-like framing
1 byte Broadcast address is used by default (users may define alternative address)
Synchronous Link may be bit-oriented or byte-oriented
Basic PPP encapsulation is extended by 8 bytes
Bit stuffing or byte stuffing allowed
Escape mechanism
allows transparent transfer of control data (e.g. ^S/^Q)
enables removal of spurious control data (inserted by intermediate boxes)

Y(J)S EoS Slide 26

RFC1662 vs. X.85

ITU-T X.85 defines IP over SDH using LAPS (will study later)
Its encapsulation is similar to RFC1662 (but cant co-exist with it)
Instead of the protocol ID it has a SAPI = 21 for IPv4 =57 for IPv6
The FCS MUST be 32 bits and no padding is used
No special escaping is defined
PPP frame
1662

X.85

flag

address

ctrl

protocol

7E

FF

03

(8/16b)

flag

address

ctrl

SAPI

7E

04

03

(16b)

information

padding

FCS

flag

(optional)

(16/32b)

7E

FCS

flag

(32b)

7E

IP Packet

Y(J)S EoS Slide 27

Background
SONET/SDH

Note:
For more information see SONET/SDH course.

Y(J)S EoS Slide 28

SONET architecture
ADM

regenerator

ADM

Path

Line

Section

Line

Path

Termination

Termination

Termination

Termination

Termination

path
line
section

line
section

line
section

section

SONET (SDH) has at 3 layers:

path end-to-end data connection, muxes tributary signals path section

there are STS paths + Virtual Tributary (VT) paths

line protected multiplexed SONET payload

section physical link between adjacent elements

multiplex section
regenerator section

Each layer has its own overhead to support needed functionality

SDH terminology
Y(J)S EoS Slide 29

SONET STS-1 frame

9 rows

90 columns

Synchronous Transfer Signals are bit-signals (OC are optical)

Each STS-1 frame is 90 columns * 9 rows = 810 bytes
There are 8000 STS-1 frames per second
so each byte represents 64 kbps (each column is 576 kbps)
Thus the basic STS-1 rate is 51.840 Mbps
Y(J)S EoS Slide 30

SDH STM-1 frame

9 rows

270 columns

Synchronous Transport Modules are the bit-signals for SDH

Each STM-1 frame is 270 columns * 9 rows = 2430 bytes
There are 8000 STM-1 frames per second
Thus the basic STM-1 rate is 155.520 Mbps
3 times the STS-1 rate!
Y(J)S EoS Slide 31

SONET/SDH rates
SONET

SDH

STS-1

columns

rate

90

51.84M

STS-3

STM-1

270

155.52M

STS-12

STM-4

1080

622.080M

STS-48

STM-16

4320

2488.32M

STS-192

STM-64

17280

9953.28M

STS-N has 90N columns

STM-M corresponds to STS-N with N = 3M

SDH rates increase by factors of 4 each time

STS/STM signals can carry PDH tributaries, for example:

STS-1 can carry 1 T3 or 28 T1s or 1 E3 or 21 E1s

STM-1 can carry 3 E3s or 63 E1s or 3 T3s or 84 T1s

Y(J)S EoS Slide 32

SONET/SDH tributaries
SONET

SDH

STS-1

T1

T3

E1

E3

28

21

E4

STS-3

STM-1

84

63

STS-12

STM-4

336

12

252

12

STS-48

STM-16

1344

48

1008

48

16

STS-192

STM-64

5376

192

4032

192 64

E3 and T3 are carried as Higher Order Paths (HOPs)

E1 and T1 are carried as Lower Order Paths (LOPs)

Y(J)S EoS Slide 33

STS-1 frame structure

9 rows

6 rows

3 rows

90 columns

Synchronous Payload Envelope

section + line
overhead

Section overhead is 3 rows * 3 columns = 9 bytes = 576 kbps

framing, performance monitoring, management
Line overhead is 6 rows * 3 columns = 18 bytes = 1152 kbps
protection switching, line maintenance, mux/concat, SPE pointer
SPE is 9 rows * 87 columns = 783 bytes = 50.112 Mbps
Similarly, STM-1 has 9 (different) columns of section+line overhead !
Y(J)S EoS Slide 34

STM-1 frame structure

270 columns

Transport
Overhead
TOH

Similarly, STM-1 has 9 (different) columns of transport overhead !

RS overhead is 3 rows * 9 columns
Pointer overhead is 1 row * 9 columns
MS overhead is 5 rows * 9 columns
SPE is 9 rows * 87 columns
Y(J)S EoS Slide 35

Scrambling

SONET/SDH receivers recover clock based on incoming signal

Insufficient number of 0-1 transitions causes degradation of clock performance
In order to guarantee sufficient transitions, SONET/SDH employ a scrambler
All data except first row of section overhead is scrambled
Scrambler is 7 bit self-synchronizing X7 + X6 + 1
Scrambler is initialized with ones
A short scrambler is sufficient for voice data
but NOT for data which may contain long stretches of zeros
When sending data an additional payload scrambler is used
modern standards use 43 bit X43 + 1
run continuously on ATM payload bytes (suspended for 5 bytes of cell tax)
run continuously on HDLC payloads

Xn

Yn = Xn + Yn-43

Z-43
Y(J)S EoS Slide 36

HOP SPE structure

2 bytes in the line overhead point to the STS path overhead POH
pointer (floating) allows frequency/phase compensation
(after re-arranging) POH is one column of 9 rows (9 bytes = 576 kbps)
Y(J)S EoS Slide 37

Path overhead
J1
B3
C2

POH is responsible for

path performance monitoring
status (including of mapped payloads)
trace

G1

C2
(hex)

Payload type

00

unequipped

01

nonspecific

02

LOP (TUG)

04

E3/T3

F2

2 bytes are of particular interest to us:

12

E4

H4

C2 is the signal label

indicates path payload type

13

ATM

16

PoS RFC 1662

H4 is the multiframe indication

used by VCAT/LCAS (discussed later)

18

LAPS X.85

1A

10G Ethernet

1B

GFP

CF

PoS - RFC1619

F3
K3
N1

POH

Y(J)S EoS Slide 38

STS-1 HOP
1

30

59

87

1 column of SPE is POH

2 more (fixed stuffing) columns are reserved
We are left with
84 columns = 756 bytes = 48.384 Mbps for payload
This is enough for a E3 (34.368M) or a T3 (44.736M)

Y(J)S EoS Slide 39

LOP
1

30

59

87

VTG
1 2 3 4 5 6 7

To carry lower rate payloads, divide 84 available columns

into 7 * 12 interleaved columns, i.e. 7 Virtual Tributary (VT) groups
VT group is 12 columns of 9 rows, i.e. 108 bytes or 6.912 Mbps
VT group is composed of VT(s)

There are different types of VT in order to carry different types of payload

all VTs in VT group must be of the same type

but different VT groups in same SPE can have different VT types

A VT can have 3, 4, 6 or 12 columns

Y(J)S EoS Slide 40

SONET/SDH : VT/VC types

VT/STS

LOP

HOP

VC

column
rate

payload

VT 1.5

VC-11

1.728 DS1

(1.544)

4 per group

VT 2

VC-12

2.304 E1

(2.048)

3 per group

3.456 DS1C (3.152)

2 per group

6.912 DS2

1 per group

VT 3
VT 6

VC-2

12

STS-1

VC-3

48.384 E3

(34.368)

STS-1

VC-3

48.384 DS3

(44.736)

STS-3c

VC-4

149.760 E4

(6.312)

(139.264)

standard PDH rates map efficiently into SONET/SDH !

Y(J)S EoS Slide 41

Payload capacity
VT1.5/VC-11 has 3 columns = 27 bytes = 1.728 Mbps
but 2 bytes are used for overhead
so actually only 25 bytes = 1.6 Mbps are available
Similarly
VT2/VC-12 has 4 columns = 36 bytes = 2.304 Mbps
but 2 bytes are used for overhead
So actually only 34 bytes = 2.176 Mbps are available

Y(J)S EoS Slide 42

VCAT
Virtual Concatenation

Y(J)S EoS Slide 43

Concatenation
Payloads that dont fit into standard VT/VC sizes can be accommodated
by concatenating of several VTs / VCs
For example, 10 Mbps doesnt fit into any VT or VC
so w/o concatenation we need to put it into an STS-1 (48.384 Mbps)
the remaining 38.384 Mbps can not be used
We would like to be able to divide the 10 Mbps among
7 VT1.5/VC-11 s = 7 * 1.600 = 11.20 Mbps or
5 VT2/VC-12 s = 5 * 2.176 = 10.88 Mbps

Y(J)S EoS Slide 44

Concatenation
There are 2 ways to concatenate X VTs or VCs:

Contiguous Concatenation (G.707 11.1)

HOP STS-Nc (SONET) or VC-4-Nc (SDH)
or LOP 1-7 VC-2-Nc into a VC-3
since has to fit into SONET/SDH payload
only STS-Nc : N=3
* 4n or VC-4-Nc : N=4n
components transported together and in-phase
requires support at intermediate network elements

Virtual Concatenation (VCAT G.707 11.2)

HOP STS-1-Xv or STS-Nc-Xv (SONET) or VC-3/4-Xv (SDH)
or LOP VT-1.5/2/3/6-Xv (SONET) or VC-11/12/2-Xv (SDH)
HOP: X 256 LOP: X 64 (limitation due to bits in header)
payload split over multiple STSs / STMs
fragments may follow different routes
requires support only at path terminations
requires buffering and differential delay alignment
Y(J)S EoS Slide 45

Contiguous Concatenation: STS-3c

270 columns
9 rows

258 columns of SPE

9 columns of
section and
line overhead

3 columns of
path overhead

258 columns * 0.576 = 148.608 Mbps

STS-3

270 columns
9 rows

STS-3c
0.576 = 149.760 Mbps

260 columns of SPE

9 columns of
section and
line overhead

1 column of
path overhead

260 columns *

Y(J)S EoS Slide 46

STS-N vs. STS-Nc

Although both have raw rates of 155.520 Mbps
STS-3c has 2 more columns (1.152Mbps) available
More generally, For STS-Nc gains (N-1) columns
e.g. STS-12c gains 11 columns = 6.336Mbps vis a vis STS-12
STS-48c gains 47 columns = 27.072 Mbps
STS-192c gains 191 columns = 110.016 Mbps !

However, an STS-Nc signal is not as easily separable

when we want to add/drop component signals

Y(J)S EoS Slide 47

Virtual Concatenation

H4

VCAT is an inverse multiplexing mechanism (round-robin)

VCAT members may travel along different routes in SONET/SDH network
Intermediate network elements dont need to know about VCAT
(unlike contiguous concatenation that is handled by all intermediate nodes)
Y(J)S EoS Slide 48

SDH virtually concatenated VCs

VC
VC-11-Xv

VC-12-Xv

VC-2-Xv

Capacity (Mbps)

if all members in one VC

1.600, 3.200,
1.600X

in VC-3 X 28 C 44.800

2.176, 4.352,
2.176X

in VC-3 X 21 C 45.696

6.784, 13.568, ,
6.784X

in VC-3 X 7

in VC-4 X 64 C 102.400

in VC-4 X 63 C 137.088
C 47.448

in VC-4 X 21 C 142.464

So we have many permissible rates

1.600, 2.176, 3.200, 4.352, 4.800, 6.400, 6.528, 6.784, 8.000,

Y(J)S EoS Slide 49

SONET virtually concatenated VTs

VT

Capacity (Mbps)

VT1.5-Xv 1.600, 3.200, 1.600X

If all members in one STS

in STS-1

X 28 C 44.800

in STS-3c X 64 C 102.400
VT2-Xv

2.176, 4.352, 2.176X

in STS-1

X 21 C 45.696

in STS-3c X 63 C 137.088
VT3-Xv

3.328, 6.656, 3.328X

in STS-1

X 14 C 46.592

in STS-3c X 42 C 139.776
VT6-Xv

6.784, 13.568, 6.784X

in STS-1

X 7 C 47.448

in STS-3c X 21 C 142.464
So we have many permissible rates
1.600, 2.176, 3.200, 3.328, 4.352, 4.800, 6.400, 6.528, 6.656, 6.784,
Y(J)S EoS Slide 50

Efficiency comparison
rate

w/o VCAT

efficiency

with VCAT

efficiency

10

STS-1

21%

VT2-5v

92%

VC-12-5v
100

STS-3c

67%

VC-4
1000

STS-48c
VC-4-16c

STS-1-2v

100%

VC-3-2v
42%

STS-3c-7v

95%

VC-4-7v

Using VCAT increases efficiency to close to 100% !

Y(J)S EoS Slide 51

PDH VCAT

VCAT
overhead
octet

1st
frame
of
4 E1s
TS0

Recently ITU-T G.7043 expanded VCAT to E1,T1,E3,T3

Enables bonding of up to 16 PDH signals to support higher rates
Only bonding of like PDH signals allowed (e.g. cant mix E1s and T1s)
Multiframe is always per G.704/G.832 (e.g. T1 ESF 24 frames, E1 16 frames)
1 byte per multiframe is VCAT overhead (SQ, MFI, MST, CRC)
Supports LCAS (to be discussed next)
each E1

time
Y(J)S EoS Slide 52

VCAT
overhead
octet

PDH VCAT overhead octet

frames
of an
E1

TS0

There is one VCAT overhead octet per multiframe, so net rate is

T1: (24*24-1=) 575 data bytes per 3 ms. multiframe = 191.666 kB/s
E1: (16*30-1=) 495 data bytes per 2 ms multiframe = 247.5 kB/s
T3 and E3 can also be used
We will show the overhead octet format later
(when using LCAS, the overhead octet is called VLI)

Y(J)S EoS Slide 53

Delay compensation
802.1ad Ethernet link aggregation cheats
each identifiable flow is restricted to one link
doesnt work if single high-BW flow
VCAT is completely general
works even with a single flow
VCG members may travel over completely separate paths
so the VCAT mechanism must compensate for differential delay
Requirement for over second compensation
Must compensate to the bit level
but since frames have Frame Alignment Signal
the VCAT mechanism only needs to identify individual frames

Y(J)S EoS Slide 54

VCAT buffering

Since VCAT components may take different paths

At egress the members
are no longer in the proper temporal relationship
VCAT path termination function buffers members
and outputs in proper order (relying on POH sequencing)
(up to 512 ms of differential delay can be tolerated)

VCAT defines a multiframe to enable delay compensation

length of multiframe determines delay that can be accommodated
H4 byte in members POH contains :
sequence indicator (identifies component) (number of bits limits X)
MFI multiframe indicator (multiframe sequencing to find differential delay)
Y(J)S EoS Slide 55

Multiframes and superframes

Here is how we compensate for 512 ms of differential delay
512 ms corresponds to a superframe is 4096 TDM frames

(4096*0.125m=512m)

For HOS SDH VCAT and PDH VCAT (H4 byte or PDH VCAT overhead)
The basic multiframe is 16 frames
So we need 256 multiframes in a superframe (256*16=4096)
The MultiFrame Indicator is divided into two parts:

MFI1 (4 bits) appears once per frame

and counts from 0 to 15 to sequence the multiframe

MFI2 (8bits) appears once per multiframe

and counts from 0 to 255

For LOS SDH (bit 2 of K4 byte)

a 32 bit frame is built and a 5-bit MFI is dedicated
32 multiframes of 16 ms give the needed 512 ms
Y(J)S EoS Slide 56

LCAS
Link Capacity Adjustment Scheme

Y(J)S EoS Slide 57

LCAS
LCAS is defined in G.7042 (also numbered Y.1305)
LCAS extends VCAT by allowing dynamic BW changes
LCAS is a protocol for dynamic adding/removing of VCAT members
hitless BW modification
similar to Link Aggregation Control Protocol for Ethernet links
LCAS is not a control plane or management protocol
it doesnt allocate the members
still need control protocols to perform actual allocation
LCAS is a handshake protocol
it enables the path ends to negotiate the additional / deletion
it guarantees that there will be no loss of data during change
it can determine that a proposed member is ill suited
it allows automatic removal of faulty member
Y(J)S EoS Slide 58

LCAS how does it work?

LCAS is unidirectional (for symmetric BW need to perform twice)
LCAS functions can be initiated by source or sink
J1
B3
C2
G1
F2
H4
F3
K3
N1

POH

LCAS assumes that all VCG members are error-free

LCAS messages are CRC protected
LCAS messages are sent in advance
sink processes messages after differential compensation
message describes link state at time of next message
receiver can switch to new configuration in time
LCAS messages are in the upper nibble of
H4 byte for HOS SONET/SDH
K4 byte for LOS SONET/SDH
VCAT overhead octet for PDH VCAT and LCAS Information
LCAS messages employ redundancy
messages from source to sink are member specific
messages from sink to source are replicated
Y(J)S EoS Slide 59

LCAS control messages

LCAS adds fields to the basic VCAT ones
Fields in messages from source to sink:
MFI
MultiFrame Indicator
SQ
SeQuence indicator (member ID inside VCAT group)
CTRL ConTRoL (IDLE, being ADDed, NORMal, End of Sequence, Do Not Use)
GID
Group Identification (identifies VCAT group)
Fields in messages from sink to source (identical in all members):
MST
Member Status (1 bit for each VCG member)
RS-Ack ReSequence Acknowledgement
Fields in both directions
CRC
Cyclic Redundancy Code
The precise format depends on the VCAT type (H4, K4, PDH)
Note: for H4 format SQ is 8 bits, so up to 256 VCG members
for PDH SQ is only 4 bits, so up to 16 VCG members
Y(J)S EoS Slide 60

H4 format

reserved fields

MFI2 bits 1-4

MFI2 bits 5-8

CTRL

GID

CRC-8 bits 1-4

CRC-8 bits 5-8

MST bits

more MST bits

RS-ACK

16 frame multiframe

reserved fields

MFI1

Y(J)S EoS Slide 61

H4 format some comments

CRC-8 (when using K4 it is CRC-3)
covers the previous 14 frames (not synced on multiframe)
polynomial x8 + x2 + x + 1
MST

each VCG member carries the status of all members

so we need 256 bits of member status
this is done by muxing MST bits
there are MST bits per multiframe
and 32 multiframes in an MST multiframe
no special sequencing, just MFI2 multiframe mod 32

GID
single bit - cycles through 215-1 LFSR sequence
Y(J)S EoS Slide 62

VLI format

reserved fields

MFI2 bits 1-4

MFI2 bits 5-8

CTRL

GID

CRC-8 bits 1-4

CRC-8 bits 5-8

MST bits

more MST bits

RS-ACK

16 frame multiframe

reserved fields

MFI1

Y(J)S EoS Slide 63

LCAS adding a member (1)

When more/less BW is needed, we need to add/remove VCAT members
Adding/removing VCAT members first requires provisioning (management)
LCAS handles member sequence numbers assignment
LCAS ensures service is not disrupted
Example: to add a 4th member to group 1
GID=g SQ=1 CTRL=NORM

Initial state:

GID=g SQ=2 CTRL=NORM

GID=g SQ=3 CTRL=EOS

Step 1: NMS provisions new member

source sends CTRL=IDLE for new member
sink sends MST=FAIL for new member

GID=g SQ=1 CTRL=NORM

GID=g SQ=2 CTRL=NORM
GID=g SQ=3 CTRL=EOS
GID=g SQ=FF CTRL=IDLE
Y(J)S EoS Slide 64

LCAS adding a member (2)

Step 2: source sends CTRL=ADD and SQ
sink sends MST=OK for new member

if it has been provisioned

if receiving new member OK

if it is able to compensate for delay

otherwise it will send MST=FAIL
and source reports this to NMS

GID=g SQ=1 CTRL=NORM

GID=g SQ=2 CTRL=NORM
GID=g SQ=3 CTRL=EOS
GID=g SQ=4 CTRL=ADD

Step 3: source sends CTRL=EOS for new member

new member starts to carry traffic
sink sends RS-ACK

GID=g SQ=1 CTRL=NORM

GID=g SQ=2 CTRL=NORM
GID=g SQ=3 CTRL=NORM
GID=g SQ=4 CTRL=EOS

Note 1: several new members may be added at once

Note 2: removing a member is similar
Source puts CTRL=IDLE for member to be removed and stops using it
All member sequence numbers must be adjusted
Y(J)S EoS Slide 65

LCAS service preservation

To preserve service integrity if sink detects a failure of a VCAT member
LCAS can temporarily remove member (if service can tolerate BW reduction)
GID=g SQ=1 CTRL=NORM

Example: Initial state

GID=g SQ=2 CTRL=NORM

GID=g SQ=3 CTRL=NORM
GID=g SQ=4 CTRL=EOS

Step 1: sink sends MST=FAIL for member 2

source sends CTRL=DNU (special treatment if EoS)
and ceases to use member 2
Note: if EoS fails, renumber to ensure EoS is active

GID=g SQ=1 CTRL=NORM

GID=g SQ=2 CTRL=DNU
GID=g SQ=3 CTRL=NORM
GID=g SQ=4 CTRL=EOS

Step 2: sink sends MST=OK indicating defect is cleared

source returns CTRL to NORM
and starts using the member again
Note: if NMS decides to permanently remove the member, proceed as in previous slide
Y(J)S EoS Slide 66

PoS
Packet over SONET

Y(J)S EoS Slide 67

Packet over SONET

Currently defined in RFC2615 (PPP over SONET) obsoletes RFC1619
SONET/SDH path can provide a point-to-point byte-oriented
full-duplex synchronous link
PPP is ideal for data transport over such a link
PoS uses PPP in HDLC framing to provide a byte-oriented interface
to the SONET/SDH infrastructure
SONET/SDH POH signal label (C2)
indicates PoS as C2=16 (C2=CF if no scrambler)

Y(J)S EoS Slide 68

PoS architecture
IP
PPP
HDLC
SONET/SDH

PoS is based on PPP in HDLC framing

Since SONET/SDH is byte oriented, byte stuffing is employed
A special scrambler is used to protect SONET/SDH timing
PoS operates on IP packets
If IP is delivered over Ethernet
the Ethernet is terminated (frame removed)
Ethernet must be reconstituted at the far end
require routers at edges of SONET/SDH network
Y(J)S EoS Slide 69

What happened to the Ethernet ?

Ethernet

IP

Ethernet

The conventional model:

Ethernet is a LAN technology

last 100m
10s of hosts

IP is a WAN technology
data transported in native IP
different L2 technologies for last segment

But modern Ethernet wants to be more

Y(J)S EoS Slide 70

PoS Details
IP packet is encapsulated in PPP
default MTU is 1500 bytes
up to 64,000 bytes allowed if negotiated by PPP
FCS is generated and appended
PPP in HDLC framing with byte stuffing
43 bit scrambler is run over the SPE
byte stream is placed octet-aligned in SPE
(e.g. 149.760 Mbps of STM-1)
HDLC frames may cross SPE boundaries
Y(J)S EoS Slide 71

RFC2615 vs. RFC1619

RFC1619 did not have the 43 bit scrambler
Malicious users could generate packets
containing frame alignment pattern
deceiving framer into mis-syncing
with low transition density
degrading clock performance
containing SONET/SDH reset scrambler pattern
causing errors
So RFC2615 added the scrambler
scrambler does not reset during use
hard to guess proper internal state

Y(J)S EoS Slide 72

POS problems
PoS is BW efficient
but POS has its disadvantages

BW must be predetermined

HDLC BW expansion and nondeterminacy

BW allocation is tightly constrained by SONET/SDH capacities

e.g. GbE requires a full OC-48 pipe

POS requires removing the Ethernet headers

So lose RPR, VLAN, 802.1p, multicasting, etc

POS requires IP routers

Y(J)S EoS Slide 73

LAPS
Link Access Protocol over SDH
X.85 and X.86

Y(J)S EoS Slide 74

LAPS
In 2001 ITU-T introduced protocols for transporting packets over SDH

X.85 IP over SDH using LAPS

X.86 Ethernet over LAPS

Built on series of ITU LAPx HDLC-based protocols

Use ISO HDLC format
Implement connectionless byte-oriented protocols over SDH
X.85 is very close to (but not quite) IETF PoS

Y(J)S EoS Slide 75

X.85 vs. X.86

X.85

X.86

IP

IP

IP

LLC

LAPS

LLC

MAC

SDH

MAC

IP

IP

IP

LLC

LLC

LLC

MAC

MAC

MAC

LAPS
SDH

X.85 transports IP packets

if delivered over Ethernet, the Ethernet is terminated
X.86 transports Ethernet
can transport all sorts of Ethernet traffic not only IP packets
Y(J)S EoS Slide 76

X.85
flag

address

ctrl

SAPI

7E

(16b)

03

(16b)

IP Packet

FCS

flag

(32b)

7E

IP over SDH using LAPS

address = 04 (or FF for compatibility with PoS)
SAPI = 21 for IPv4 =57 for IPv6 (changed to be like PoS)
Scrambler always used
Can use LOP VCs, HOP VCs or STMs

Y(J)S EoS Slide 77

MAC

X.86

reconciliation
MII/GMII

LAPS
rate adaptation

SDH
Similar to X.85 (IP over SDH using LAPS)
but transports the entire Ethernet frame
Provides a virtual MII/GMII interface
Transparent to all Ethernet features (VLAN, P bits, RPR, etc.)
Rate adaptation by adding hex DD (after byte stuffing 7D DD)
Ammendment specifies use of Ethernet PAUSE frames for rate limiting

flag

address

ctrl

SAPI

Ethernet frame

FCS

flag

7E

(16b)

03

FE01

DA SA T/L INFO PAD FCS

(32b)

7E
Y(J)S EoS Slide 78

LAPS drawbacks
Only IP or Ethernet payloads
Single bit errors (e.g. in flags) may cause misalignment
Not very efficient
HDLC BW expansion
HDLC BW nondeterminacy

Y(J)S EoS Slide 79

GFP
Generic Framing Procedure

Y(J)S EoS Slide 80

GFP architecture
Defined in ITU-T G.7041 (also numbered Y.1303)
originally developed in T1X1 to fix ATM limitations
(like ATM) uses HEC protected frames instead of HDLC
GFP generically encapsulates client (e.g. IP, Ethernet)
onto transport network (e.g. SONET/SDH, OTN)

Ethernet

IP

HDLC

other

GFP client specific part

GFP common part
PDH

SDH

OTN

other

Client may be PDU-oriented (Ethernet MAC, IP)

or block-oriented (GbE, fiber channel)
GFP frames
are octet aligned
contain at most 65,535 bytes
consist of a header + payload area
Any idle time between GFP frames is filled with GFP idle frames
Y(J)S EoS Slide 81

GFP frame structure

Every GFP frame has a 4-byte core header
2 byte Payload Length Indicator
PLI = 01,2,3 are for control frames

2 byte core Header Error Control

X16 + X12 + X5 + 1

core
header

so idle frames are B6AB31E0 (Barker-like codes)

Non-idle GFP frames

have 4 bytes in payload area
the payload has its own header
2 payload modes : GFP-F and GFP-T
optionally protect payload with CRC-32
payload is scrambled like PoS

cHEC (2B)
payload header
(4-64B)

entire core header is XORed with B6AB31E0

Idle GFP frames
have PLI=0
have no payload area

PLI (2B)

payload
area

payload
optional payload
FCS (4B)

Y(J)S EoS Slide 82

GFP payload header

GFP payload header has
type (2B)
PTI (3b) PFI EXI (4b)
type HEC (CRC-16)
UPI (8b)
extension header (0-60B)
either null or linear extension (payload type muxing)
extension HEC (CRC-16)
type consists of
Payload Type Identifier (3b)

PTI=000 for client data

PTI=100 for client management (OAM dLOS, dLOF)

Payload FCS Indicator (1b)

PFI=1 means there is a payload FCS

Extension Header ID (4b)
User Payload Identifier (8b)

values for Ethernet, IP, PPP, FC, RPR, MPLS, etc.

type (2B)
tHEC (2B)
extension header
(0-58B)
eHEC (2B)

Y(J)S EoS Slide 83

GFP modes
GFP-F - frame mapped GFP
Good for PDU-based protocols (Ethernet, IP, MPLS)
or HDLC-based ones (PPP)
Client PDU is placed in GFP payload field
GFP-T transparent GFP
Good for protocols that exploit physical layer capabilities
In particular
8B/10B line code
used in fiber channel, GbE, FICON, ESCON, DVB, etc
Were we to use GFP-F would lose control info, GFP-T is transparent to these codes
Also, GFP-T neednt wait for entire PDU to be received (adding delay!)

Y(J)S EoS Slide 84

GFP-T
Main application Storage Area Networks (SAN)
SANs use 8B/10B line code and are very delay sensitive
8B/10B line code maps each of the 256 values of the 8-bit input
into 1 or 2 different 10 bit words
Maintains a running 0-1 balance and when encoding an input with 2 possibilities, it
chooses the one that improves the balance
spare 10b symbols are used as control codes (e.g. start/end of frame)
Were we to use GFP-F would lose control info, GFP-T is transparent to these codes
Also, GFP-T neednt wait for entire PDU to be received (adding delay!)
GFP-T maps 8B/10B line code into 64B/65B block code

Y(J)S EoS Slide 85

GFP-F
Client packet/frame without un-needed overhead (e.g. flags, preamble, etc)
is placed in GFP payload field
Interface is at link layer
More BW efficient than GFP-T since idle periods are filtered out
preambles, frame-start, etc are also not transported
GFP-F must know the client protocol in order to detect frames
Can mux different client protocols on a frame to frame basis
If the client protocol has a good FCS, dont need to use GFPs FCS
GFP-F is used for EoS
Either IP in PPP or native Ethernet can be used

Y(J)S EoS Slide 86

GFP advantages
Supports multiple protocols (not just Ethernet and IP)
For Ethernet, GFP can transparently transport entire frame
Robust single bit errors do not cause loss of alignment
Constant predictable overhead
Good efficiency (similar to LAPS best case)
GFP-T for SAN support
Can run over OTN (G.709) as well as SONET

Y(J)S EoS Slide 87

Alternatives

Y(J)S EoS Slide 88

There are yet other ways

Ethernet in the first mile (EFM)

WAN-PHY (10GBASE-W)

Ethernet over wavelengths (EoW) or OTN (G.709)

Ethernet over Resilient Packet Rings (RPR)

Ethernet pseudowires (PWs)

Y(J)S EoS Slide 89

Ethernet in the First Mile

IEEE 802.3ah task force produced the EFM definition
Optical technologies

point to point optical fiber @ 100Mbps 10 km

Dual fiber duplex 100Base-LX10
Single fiber simplex 100Base-BX10

point to point optical fiber @ 1Gbps 10 km

Dual fiber duplex 1000Base-LX10
Single fiber simplex 1000Base-BX10

point to multipoint optical fiber @ 1Gbps 10/20 km (EPON )

Single fiber simplex 1000Base-PX10/20

Copper technologies

point to point copper @ 10 Mbps 750 m (short reach PHY)

VDSL 10PASS-TS

point to point copper @ 2 Mbps 2.7 km (long reach PHY)

SHDSL.bis 2Base-TL
up to 45 Mbps by bonding

OAM
Y(J)S EoS Slide 90

WAN-PHY (10 GbE in STM-64)

10GBASE-W 802.3-2005 Clause 50

G.707 Annex F

There is a special case where Ethernet and SDH bit-rates are close
STM-64 is 9953.28Mbps
GbE 10GBASE-R (64B/66B coding) can be directly mapped
into a STM-64 (with contiguous concatenation) without need for GFP
MAC creates "stretched InterPacket Gap" to compensate for rate being < 10G
This is the fastest connection commonly used for Internet traffic
Complication: SDH clock accuracy is 4.6 ppm, GbE accuracy is 20 ppm

64*(270-9) = 16704
columns
J1

63 columns of fixed stuff

Y(J)S EoS Slide 91

Ethernet over Wavelengths

Rather than muxing Ethernet flows using SONET mechanisms
We can allocate a separate wavelength (lambda) per flow
Wavelength Division Multiplexing (WDM)
For example, each wavelength may support OC-48 (2.5 Gbps)

Up to 8 channels is called coarse CWDM

More than 8 wavelengths (20 Gbps) is called dense DWDM
Present DWDM technology allows about 80 channels
Higher densities expected soon

DWDMs tight channel spacing requires expensive cooled laser sources

Y(J)S EoS Slide 92

Ethernet PWs
Customer
Edge

Pseudowire (PW): mechanism that emulates essential

attributes of a native service while transporting over a PSN

(CE)
Customer
Edge
(CE)
Customer
Edge

Customer
Edge

MPLS network
Provider
Edge

Provider
Edge

(CE)

(PE)

(PE)

Customer
Edge

Ethernet
Ethernet

PseudoWires (PWs)

(CE)
MPLS
label
stack

PW
label

PWE
control
word

(CE)
Ethernet frame
(with or w/o FCS)
Y(J)S EoS Slide 93