High Speed WAN: Traditional and...

High Speed WAN: Traditional and emerging

technologiesEmerson Moura, Distinguished Systems Engineer

CCIE# 15356

BRKOPT-2117

• Introduction

• Traditional High Speed WAN Technologies

• Ethernet in the WAN

• High-Speed Optics

• Evaluation of Key Attributes for WAN

• Summary

Agenda

Introduction

• Business requirements are driving higher speeds in the WAN.

• Higher speeds can lead to higher cost and more complexity, pushing for technology innovations and new service paradigms.

• There are many options to migrate to higher speeds in the WAN, each one has it’s pros and cons.

• Goal is to understand and minimize the impact of the tradeoffs.

Challenges in the WAN

time

Bandwidth Requirements

Budget

Expected

$/Bandwidth

Tackling the bandwidth challenge

• New higher-speed interfaces

• Advances in optical / photonics

• Leverage economies of scale

• Packet switching based services

• Cost reductions

• Changes in network architectures

• Technology Convergence

• WAN Optimization

40G, 100G, 200G and Beyond

Silicon Photonics, Modulations

Ethernet

IP/MPLS, MEF Services

Moore’s Law, new technologies

CDNs, SDNs, Distributed DCs

Ethernet and Transport

TCP Optimization, Acceleration

Finding the right balance = Engineering

Faster

Cheaper

Simpler

Trade-Offs

Efficient

Reliable

Manageable

Constraints Requirements

Expected Technical Attributes of WAN Services

Low Overhead/Efficiency

Robustness/OAM/Protection

Interoperability/Transparency

Flexibility/Simplicity

Scalability/High Speeds

WAN Transport Technology MapNote: Partial, Simplified View. Only References to Fiber Optics Interfaces Were Included.

802.3ae WWDM

802.3ae WAN-PHY

802.3ae LAN-PHY

G.707

GR-253-CORE

802.1ah

802.1Q / 802.1ad

802.3

G.957802.3ae

802.3aq

RFC 1662

RFC 2615

Native Ethernet Ethernet + G.709 POS

G.709

Non-IP IP

• Introduction





• Summary

Agenda

Traditional High-Speed* WAN Technologies

• POS – Packet over SONET/SDH, commonly found in Service Providers and in large Enterprises.

• Classic Ethernet – 802.3ae LAN/WAN, widely used in Enterprise switches and routers without carrier grade protocols.

* Transmission rates above DS3/E3

Packet over SONET/SDH

Packet Over SONET/SDH (PoS)• PPP/HDLC over SONET/SDH frames.

Very efficient with low overhead.

• Standards :

• RFC 1662 PPP in HDLC-like Framing

• RFC 2615 PPP over SONET/SDH

• G.707/GR-253-CORE Framing format, hierarchy, etc

• Scales from 155Mbps up to 40Gbps.

• Available in Channelized interface options with very good granularity (down to DS0).

• Excellent OAM and protection - industry benchmark.

• Most efficient technology for applications with small packet sizes (eg. VoIP).

SONET SDH Bit Rate

(Mbps)

OC-3 STM-1 155.52

OC-12 STM-4 622.08

OC-48 STM-16 2,488.32

OC-192 STM-64 9,953.28

OC-768 STM-256 39,813.12

SONET/SDH Hierarchy

SONET/SDH Limit – 40Gbps

Pros and Cons of PoS Technology

• Fault Management

• Performance Management

• Fast, reliable protection

• Overhead Efficiency

• Full interoperability with legacy transport

• Cost

• Single class of service

• Fixed rates with rigid hierarchy

• Port density

• Stopped at 40Gbps

Pros Cons

Calculating PoS Data Rate for OC-192/STM-64

Frame Structure: 270 Columns x 9 Rows

• 9 Columns Fixed Overhead per Frame, 1 Column Path-Overhead per Frame

• 64 Concatenated Frames in OC-192/STM-64

Line Rate: (270 x 9)Bytes x 8 Bit/Byte x 8kfps x 64 = 9.953280 Gbps

Payload: ((270-9-1) x 9)Bytes x 8 Bit/Byte x 8kfps x 64 = 9.58464 Gbps

Frame Period = 125us (8kfps)S

TS

-19

2/V

C-4

-64

c

Efficient HDLC encapsulation

PoS Efficiency for IP Traffic at OC-192/STM-64

IP Packet

Size (Bytes)

Packet

%

Byte

%

IP Traffic

(Mbit/s)

40 58.7 5.9 555.9

44 2.0 0.2 20.9

576 23.66 34.5 3,227.7

1500 15.67 59.5 5,567,1

Total (Mbps) 9,371.5

Line Rate Efficiency 9,371.5 / 9,953.3 = 94.2%

Payload Efficiency 9,371.5 / 9,584.6 = 97.8%

Maximum Theoretical Values – IMIX Traffic Mix (IP)

POS encapsulation is very efficient (low overhead)

PoS OAM Architecture

• Several OAM Layers are provided: Client, Section, Line, Path. (end-to-end visibility)

• Comprehensive set of features: PM, FM, Signaling, Tracing, Protection triggering.

• OAM information is carried and processed in every SONET/SDH frame (near real-time).

• Seamless OAM integration between layers.

SONET/SDH

Network

Section Section Section

Line

Path

OC-N/STM-N

Client OAM (PDH/SONET/SDH)

OC-N/STM-N

SONET/SDH Frame SONET/SDH Frame

Classic Ethernet

Ethernet Frames

S

O

F

DA SAType/

Length

802.2 Header

and PayloadFCSPreamble

7 1 6 6 2 46-1500 4

IFG

12Bytes >

Field >

802.3 Frame

S

O

F

Type/

Length

802.2 Header


7 1 6 4 2 46-1500 4

IFG

12

802.1Q Frame

TAGDA SA

6

Worst case Overhead = 38 Bytes

Worst case Overhead = 42 Bytes

Bytes >

Field >

Service Frame*

Packet

Service Frame*

Packet

* MEF 10.3 Definition of Service Frame

Classic Ethernet Pros and Cons

• Cost

• Flexible and granular data rates

• Multiple Classes of Services

• Multiple connectivity options

• Speeds up to 100Gbps

• Ubiquitous technology

• No end-to-end PM

• No end-to-end FM

• Poor protection mechanisms

• Loops

• Potential interoperability issues with legacy transport

• Large overhead

Pros Cons

Example: 10 Gigabit Ethernet Layer Diagram*

LX4 SR LR SW LW EWER

* Fiber options shown only. Does not include 10GBASE-CX4, 10GBASE-T

Media Access Control (MAC)

Full Duplex

10 Gigabit Media Independent Interface (XGMII) or10 Gigabit Attachment Unit Interface (XAUI)

WWDM

PMD

1310 nm

Serial

PMD

850 nm

Serial

PMD

1310 nm

Serial

PMD

1550 nm

Serial

PMD

850 nm

Serial

PMD

1310 nm

Serial

PMD

1550 nm

WWDM

LAN PHY

(8B/10B)

Serial

LAN PHY

(64B/66B)

Serial

WAN PHY

(64B/66B + WIS)

9.95 Gbps10.31 Gbps12.5 GbpsInterface Rate

9.29 Gbps10 Gbps10 GbpsData Rate

Calculating 802.3ae WIS (WAN-PHY) Data Rate

As previously described, OC-192/STM-64 Payload = 9.58464 Gbps

MAC-layer data rate (include line coding): Data-Rate x 64/66 = 9.294196 Gbps

Frame Period = 125us (8kfps)

ST

S-1

92

/VC

-4-6

4c

Classic Ethernet Efficiency for IP Traffic at 10GE

LAN-PHY WAN-PHY

Packet Size

(Bytes)

Packet

%

Byte % IP Traffic (Mbit/s)

802.3 Frames

IP Traffic (Mbit/s)

802.1Q Frames

IP Traffic (Mbit/s)

802.3 Frames

IP Traffic (Mbit/s)

802.1Q Frames

40 58.7 5.9 541.1 536.2 502.9 498.3

44 2.0 0.2 20.2 20.1 18.9 18.7

576 23.66 34.5 3,142.4 3,113.7 2,920.7 2,893.9

1500 15.67 59.5 5,419.9 5,370.4 5,037.4 4,991.3

Total (Mbps) 9,123.8 9,040.4 8,479.8 8,402.3

Line Rate Efficiency 9,123.8 / 10,312.5 = 88.5% 9,030.4 / 10,312.5 = 87.6% 8,479.8 / 9,953.3 = 85.2% 8,402.3 / 9,953.3 = 84.4%

Data Rate Efficiency 9,123.8 / 10,000.0 = 91.2% 9,030.4 / 10,000.0 = 90.4% 8,479.8 / 9,584.6 = 88.5% 8,402.3 / 9,584.6 = 87.6%

Maximum Theoretical Values – IMIX Traffic Mix (IP)

Ethernet has a high encapsulation tax for small packets

• Uses SONET/SDH frame with minimum overhead functions

• Requires VC-4-64c (64 contiguous VC-4s) orSTS-192c (192 contiguous STS-1s)

• Clock tolerance differs from ITU-T *:

• 802.3ae: +/- 20ppm

• ITU-T: +/- 4.6 ppm

• Result: generates higher pointer justifications in SONET/SDH networks.

• There is no WAN-PHY or equivalent for 40G or 100G.

802.3ae WIS (WAN-PHY) Framing Considerations

* Some component providers claim better specifications than IEEE.

Better Applicable to xWDM Transport

802.3ae WIS OAM Overhead

A1 A1 A1 A2 A2 A2 J0 Z0 Z0 J1 ….

B1 E1 F1 B3 ….

D1 D2 D3 C2 ….

H1 H1 H1 H2 H2 H2 H3 H3 H3 G1 ….

B2 K1 K2 F2 ….

D4 D5 D6 H4 ….

D7 D8 D9 F3/Z3 ….

D1

0

D1

1

D1

2K3/Z4 ….

S1 M1 N1/Z5 ….

Lin

e O

ve

rhe

ad

Mu

ltip

lex S

ectio

n

Se

ctio

n O

ve

rhe

ad

Re

ge

ne

ratio

n S

ectio

n

Pointers

Path Overhead Fixed Stuff (63 Columns)1 2 3 4 5 6 7 8 9

1

2

3

4

5

6

7

8

9

Undefined Overhead Bytes –

Set to Zero

SONET/SDH Defined Bytes Not Used

by 10GE WAN-PHY– Set to Zero

XX Fixed Value Bytes

XX Calculated Value Bytes

XX Provisioned Value Bytes

Reference

OAM in Native Ethernet Networks

• Up to 10GE, traditional Ethernet provides simplest OAM possible – CRC/FCS.

• Lacked tools for PM, FM, Signaling, Tracing, Protection, etc.

• Physical and logical failures have to be handled by upper layer protocols timers (BFD, IGPs, STP, etc).

Ethernet

Network

FCS FCS FCSFCS FCS

Ethernet

Upper Layers Fault Detection Mechanisms (ex. BFD, IGP Timers, etc.)

Ethernet

Ethernet and Transport OAM Interoperability

• OAM is provided only at the Transport layer.

• No OAM Interworking between layers. Typical solution – Failure detected at transport layer triggers client port squelching.

• Network failures or degradation requires upper layer protocols for detection and convergence.

• Dribbling conditions can’t be easily detected.

SONET/SDH

Network

Section Section Section

Line

Path

Ethernet

Upper Layers Fault Detection Mechanisms (ex. BFD, IGP Timers, etc.)

Ethernet

FCS FCS

40GE and 100GE

• IEEE 802.3ba added 40Gb/s and 100Gb/s Ethernet interfaces.

• Not just higher speeds – brings different interface architectures.

• Standard ratified June 17, 2010.

• Aligned with ITU Study Group 15, Next Generation Optical and Transport Networks (OTN):

• ODU4 defined and completed, compatible with 100GE

802.3ba Interfaces

Interface PHY Layer Link Distance

40GBASE-CR4 4 Lanes of Shielded

Copper Bal. Cabling

7m/23ft

40GBASE-SR4 4 Lanes of Multimode

Fiber

100m/330ft

40GBASE-LR4 4 WDM Lanes on

Singlemode Fiber

10km/6mi

100GBASE-CR10 10 Lanes of Shielded

Copper Bal. Cabling

7m/23ft

100GBASE-SR10 10 Lanes of Multimode

Fiber

100m/330ft

100GBASE-LR4 4 WDM Lanes on

Singlemode Fiber

10km/6mi

100GBASE-ER4 4 WDM Lanes on

Singlemode Fiber

40km/25mi

Source: 802.3ba – Table 80-1

IEEE Interfaces

Reach up to

40km/25 Miles

Requires OTN/DWDM

for Extended Reaches

802.3ba 100GE Specifications

Source:

IEEE 802.3ba

Table 88-8

WDM

Not compatible

with ITU-T G.694

Reference

OTN

Network

OTN BIP

Ethernet PCS BIP

40GE/100GE 40GE/100GE

1.00E-09

1.00E-08

1.00E-07

1.00E-06

1.00E-05

1.00E-04

1.00E-03

1.00E-02

1.00E-01

1.00E+00

1.00E-09 1.00E-08 1.00E-07 1.00E-06 1.00E-05 1.00E-04 1.00E-03 1.00E-02 1.00E-01 1.00E+00

Actual BER

Indic

ate

d B

ER

BIP-8

Invalid

Sync

Theory Measured

PCS BIP

MONPCS BIP

MON

PHY Level Error Monitoring for Ethernet

• BIP-8 monitoring incorporated into 802.3ba standard (40GE/100GE)

• Improved error monitoring, without the need for OTN encapsulation

• Transparent transport over the OTN network

• Proactive monitoring with user configurable SF/SD BER thresholds, etc

OTN

TxP

OTN

TxP

DP-QPSK

40G/100G DWDM Network

OTU440G/100G Router/Switch

???

Native Ethernet OTN “Short Reach” Client

• Transport maps Ethernet to OTN

• Lowest cost .. but perceived lack of OAM

• IEEE 802.3ba added BIP error check

• SONET/OTN like error monitoring

SF/SD BER thresholds, etc

• Router maps Ethernet to OTN

• Good OAM.. but OTN requires serial optics

• ITU added support for parallel optics:

OTL4.4, OTL4.10 OTL3.4, etc

• Can run OTN clients (both 40G and 100G)

over ‘Ethernet’ CFP optics.

TxP

40G/100G Client Interface Options

DP-QPSK

OTN/SONET OAM Ethernet Phy OAM Comments

LOS LOS Both based on monitoring optical power level.

LOF/LOMF PCS_Block_LossPCS_Block_Loss provides a robust, deterministic link failure

alarm similar to LOF in SONET/OTN.

AIS LF (Local Fault)

LF is inserted at the physical layer to indicate to the

downstream MAC of an upstream failure. Same function as AIS

in SONET/OTN.

BDI RF (Remote Fault)

When a local MAC receives a LF condition, it transmits a RF

condition to the remote MAC. Same function as RDI/BDI in

SONET/OTN

Error Monitoring:

OTU, ODU BIP etc

64/66 Sync Header ErrorsMonitors errors in 66bit block sync headers. Can be used to

determine bit error ratio of the link

PCS Lane BIP-8 monitor *Same error checking mechanism as SONET/OTN. Added by

802.3ba to improve error monitoring for 40GE/100GE .

OTU-SF-BER PCS_high_BERIndicates PCS is detecting BER> e-4. Same as OTU-SF-BER,

except fixed and not user programmable.

OTU-SD-BER PCS_SD_BER *

Could implement a user configurable BER alarm, based on

monitoring PCS errors. Could also use to trigger LF/RF and

take interface down.

TCA-BER TCA_PCS_BER *Performance monitoring TCA (threshold crossing alert). Could

implement same thing using PCS error monitoring.

* Added in IEEE 802.3ba for 40GE/100GE

OTN/SONET versus Ethernet PHY OAM Reference

• Introduction





• Summary

Agenda

Introducing Metro Ethernet Forum CE 2.0 Services

CE 2.0 Services

Point to Point

E-Line

EPLine

EVPLine

Multipoint

E-LAN

EPLAN

EVPLAN

E-Tree

EPTree

EVPTree

Wholesale

E-Access

EPAccess

EVPAccess

Reference: BRKSPG-2720 SessionBRKOPT-2117

Three Approaches to Ethernet in the WAN

• Use Ethernet as client and traditional TDM technologies as transport;

• Make Ethernet more carrier grade and use it as both client and transport;

• Use Ethernet as client and another carrier grade packet switching technology as transport (eg. MPLS);

A fourth approach….

• The combination of two or all approaches above!

Ethernet over Traditional Transport Networks

Ethernet

SONET/SDH

VC/STS Containers

OTN

Optical

xWDM

Wavelength

ODU Containers

Transparent (2R)

Sub-wavelength

ODU Containers

Electrical

OTN Switches

ODU containers

Mapping Ethernet to TDM with GFP

Preamble

IPG

SOF Delimiter

DA

SA

Length/Type

Data

Data

FCS

GFP Payload

tHEC

Type

GFP Ext. Header

cHEC

PLI

7

1

6

6

2

4

2

2

2

2

0-60

Byte

s

Byte

s

Ethernet MAC Frame GFP Frame

12

GFP: Generic Framing Procedure

Ethernet over SONET/SDH Containers

VC-N(1)/STS-N(1)

VC-N-mc / STS-N-mc

VC-N(m)/STS-N(m)

VC-N(1)/STS-N(1)

VC-N(3)/STS-N(3)

VC-N(2)/STS-N(2)

VC-N(m)/STS-N(m)

Contiguous Concatenation

Virtual Concatenation (VCAT)

VC-N-mv / STS-N-mv

Link Capacity Adjustment Scheme (LCAS)

Allows in-service scale up or down the size of the virtual container.

m times containers of same size that must be contiguous.

m times containers of same size that can be anywhere in the SONET/SDH structure.

unavailable

Ethernet over SONET/SDH Pros and Cons

• Well-known, simple service;

• Sub 50ms Protection;

• Deterministic, no shared bandwidth*;

• Huge infrastructure deployed and usually available;

• Native data scrambling;

• Additional overhead and latency;

• Rigid bandwidth (n x container size);

• No OAM or signaling integration;

• Point-to-point services only **;

• No statistical multiplexing **;

• No service multiplexing **;

• Usually limited bandwidth available.

Pros Cons

* Packet networks could also achieve this.

** Refers to native support on SONET/SDH technology.

Optical Transport Network (OTN)

• Standards :• G.709 Hierarchy and frame

structures

• G.872 Architecture

• G.798 Management functions etc

• Defines G.709 as a framing technology and hierarchy that is very similar to SONET/SDH.

• G.709 started as a wrapper around WDM client signals to improve reach and manageability

• Evolved to a complex multiplexing hierarchy that enables a service layer

Payload

Frame Payload (OPU)

ODU0 1,238,954 kbit/s

OTU1 2,488,320 kbit/s

OTU2 9,995,276 kbit/s

OTU3 40,150,519 kbit/s

OTU4 104,355,975 kbit/s

G.709 Hierarchy

Optical Transport Network (OTN) Model

Optical Amplifiers

MUX/DEMUXMUX/DEMUX

Optical

Channels

OCh

OMS

OTSOTS OTS

Optical Channel (Och)

Optical Multiplex Section (OMSn)*

Optical Transmission Section (OTSn)*

*Under Definition.

Reference

OTN Options

• Optical OTN:

• DWDM Transport

• All Optical Network

• Lambda or Sub-lambda services

• Cross-connect or switching at the Lambda/Wavelength Level

• Electrical OTN:

• SONET/SDH Evolution

• Switching of ODU-k containers

• Circuit services

• Can use Optical OTN as transport

ITU-T G.709 Mapping

OTUk

OPUk

ODUk

Client Payload

Client Payload

Client Payload

Client Payload

OH

OH

OTUk FEC

GFP-T GFP-F SONET/SDH ATM

Ether

IP

ESCON/FC

0 = 1.25G (ODU Only)

1 = 2.5G

2 = 10G

3 = 40G

4 = 100G

k Indicates the Order:

G.Sup43

OH OH

OH OH

Reference

Lo

g (

BE

R)

4 5 6 7 8 9 10 11 12 13 14 15–15

–14

–13

–12

–11

–10

–9

–8

–7

–6

–5

–4

–3

–2

–1

0

S/N (dB)

Uncoded

No FEC

Raw Channel BER=1.5e-3

EFEC=8.4 dB

FEC=6.2 dB

Forward Error Correction (FEC)Compensates for Optical Impairments

• FEC extends reach and design flexibility, at “silicon cost”

• G.709 standard improves OSNR tolerance by 6.2 dB (at 10–15 BER)

• Offers intrinsic performance monitoring (error statistics)

• Higher gains (8.4dB) possible by enhanced FEC (with same G.709 overhead)

FEC/EFEC Extends Reach and Offers 10–15 BER

Ethernet over G.709 Encapsulation

• G.709 was developed to support SONET/SDH networks, not Ethernet.

• Ethernet line rates and G.709 payloads were not directly compatible.

• Solution to the GE interoperability problem: ODU0 (1.25Gbps payload)

• Different solutions exist to the 10GE interoperability problem (next slides).

• OTN provides full compatibility with 100GE (ODU4).

Ethernet Line Rate Original OTN Payload (ODU-N)

GE: 1.25 Gbps OPU1 = 2.488 Gbps

10GE LAN-PHY: 10.325 Gbps OPU2 = 9.953 Gbps

The Challenge to Transport 10GE LAN-PHY over G.709

Mapping Description Advantages Disadvantages

GFP-F into OPU2 Mapping Ethernet Frames Using

GFP-F into OPU2

• Defined in G.7041 • Not transparent to overhead

information such as preamble or IPG

• Does not support full Ethernet rate

Overclocked OPU2 Mapping of 64B/66B Encoded

Data into OPU2 with Increased

Clock Rate

• Bit level Ethernet transparency • Requires increasing the G.709 bitrate

• Two different mappings and rates

currently proposed

Rate Adaptive Mapping into

OPU2

Mapping of 64B/66B Encoded

Data into OPU2 with Idle

Removal and Optional Flow

Control

• Bit level Ethernet transparency

• Maintains G.709 bit rate

• Does not support full Ethernet rate

• Requires client to reduce rate by

inserting idles or support of 802.3x flow

control

GFP-F Mapping into Extended

OPU2

Extended OPU2 Payload Using

GFP-F

• Information level Ethernet

transparency – preserves

preamble and ordered sets


• No bit level transparency

• No G.709 Compliant (OPU OH used

as Payload)

Reduced Rate into OPU2 Lower Ethernet Rate so that

64B/66B Encoded Data Can Be

Mapped into a Standard Rate

OPU2


• Bit level Ethernet transparency

• Requires defining a new Ethernet

interface with a lower bit rate

OTN ODUFlex Split 64B/66B Encoded Data

into 5 OPU1k VC

• Transparency transport of 10GE

LAN @10.3 G

• 2 lambdas requested to transport a

single 10GE LAN

ReferenceReference

Transporting 40GE over OTN

• IEEE has made 40GE more friendly to OTN.

• 40GE LAN – same 64B/6BB coding as 10GE LAN.

• Result – 41.25 Gbps serial signal.

• ODU3 Payload (OPU3) rate: 40.15Gbps• Solution:

• Transcode 64B/66B into a more efficient 1024B/1027B block code.

• Resulting stream = 40.11Gbit/s – fits into OPU3.

• Map resulting stream in the OPU3 using ITU-T GMP.

100GE over OTN

• ITU-T has optimized the OTU4 rate for 100GE.

• OTU-4 rate is 111.809974 Gbit/s.

• Beneficial for 100GE – full compatibility.

• Challenges for OTN:

• breaks the 4x increase. 100G is not a clear integer of lower ODUk signals (ex. 4x ODU3 = 160Gbps).

• What’s next? OTU-5 (400Gbps vs 1Tbps)

Additional OTN Containers and Capabilities

• ODU-0 – smaller 1.25Gbps container.

• ODU-3e – similar to ODU-2e, for 40GE and 4x10GE.

• ODU-4 – compatible with 100GE.

• ODU-Flex – custom ODU rates (nx ODU-0/1.25Gbps).

• GMP – Generic Mapping Procedure.

Ethernet over OTN Pros and Cons

• Similar to SONET/SDH*; plus

• More bandwidth available compared to SONET/SDH.

• Similar to SONET/SDH*; plus

• May not be transparent to 10GE or 40GE (bit level transparency).

• Big containers only, not suitable for lower speed services.

Pros Cons

* See slide 45.

Making Ethernet Carrier Grade

New standards complement Ethernet to address carrier grade, high-performance MAN/WAN environments.

OAM SynchronizationTransport

EncapsulationScale

802.1ad802.1ah

Y.1731802.1ag802.3ah

1588v2SyncE

OTNG.709

*Not covered in this session.

Resiliency *

G.8031

G.8032

Ethernet Frame FormatsAddressing Service Provider Scale Needs

S

O

F

Type/

Length

802.2 Header


7 1 6 4 2 46-1500 4

IFG

12QinQ or 802.1ad Frame (PB)

C

TAGDA SA

6

S

TAG

4

Overhead = 46 Bytes

Bytes >

S

O

F

Type/

Length

802.2 Header


7 1 6 4 2 46-1500 4

IFG

12

802.1ah Frame (PBB)

C

TAGDA SA

6

S

TAG

4

Overhead = 68 Bytes

ITAG

4

B

DA

B

SA

B

TAG

466

TPID

2

Field >

End to End Service

Performance Management

End to End Service

Fault Management

Point to Point

Link Fault Management

Ethernet OAM

Source: Metro Ethernet Forum (www.metroethernetforum.org)

Key Capabilities:

http://www.metroethernetforum.org/

Ethernet OAM in Action

Link

OAMLink

OAM

Link

OAM

Link

OAM

Link

OAM

Ethernet Ethernet

Customer Domain

Carrier Ethernet

Network

SP Domain

E-LMIE-LMI

Down

MEP

Up

MEPUp

MEP

Down

MEP

MIP MIP

Acronyms: MEP - Maintenance End Point MIP - Maintenance Intermediate Point E-LMI - Ethernet Local Management Interface

• Y.1731: Fault Management (FM) and Performance Management (PM)

• 802.3ah - Link OAM: Link level monitoring and signaling.

• E-LMI: PE-CE communication of service status and attributes.

Ethernet and Transport OAM Comparison

SONET/SDH Ethernet Notes

Device Configuration Minimal Heavy SONET/SDH works

“out-of-the-box”.

Ethernet requires configuring

MEPs/Probes per service

How OAM Information

Is Sent

In every frame Sent on dedicated

frames

Specific fames for Fault Management

and Performance Management

OAM Frame

Frequency

125us 1 ms to 1 minute Ethernet FM/PM Frame processing

requires CPU or specialized hardware

OAM Overhead ~4% ~5% 10ms/2500 MEPs/256B

messages/10Gbps interface =

512Mbit/s

CFM Connectivity Check Messages

Only

Adding Transport Encapsulation to Ethernet with G.709

• Available in new generation of Routers, Carrier Ethernet Switches and Packet Transport equipment.

• Suitable to DWDM applications leveraging FEC, colored optics and dark fiber.

• Potentially PoS performance at price points closer to native Ethernet.

Packet

Processing

Ethernet

MAC

Layer

G.709

Framer

Optical

I/F

(Grey/

Colored)

G.709 Enabled Ethernet Interface Architecture

Achieving 10GE LAN PHY Line Rate over G.709

Standard G.709

Overclocking

10GBASE-R Bit Rate Coded with 64B/66B: 10,3125 Gbit/s

G.709 (ODU2) Payload: 9,995 Gbit/s

Solution: Overclocked G.709 Frame (ODU2e)

IP/MPLS Services for Ethernet

• IP/MPLS provides the most comprehensive services to Ethernet.

• VPWS: Pseudowires for point to point services;

• VPLS: full mesh of Pseudowires to provide multipoint services;

• EVPN as the latest addition, leverages BGP control plane.

• Latest standards further enhance carrier grade features of IP/MPLS:

• TI-LFA: Topology Independent Loop-free Alternate, provides sub-50ms convergence;

• SR: Segment Routing, removes protocols for simpler deployment;

• RFC 3107: Hierarchical LSPs to scale MPLS domain to 100’s of thousands of nodes.

• BGP-PIC: Prefix Independent Convergence, predictable convergence times.

• Introduction





• Summary

Agenda

Needs for 40GE & 100GE optical modules

Tradeoff

Triangle

Reach

Density

Cost

Seems impossible to optimize all

3 vertices simultaneously…

Core

Distribution

Access

Servers

& Clients

Choice of optimized vertices will

vary by end user & segment of the

network

40/100G Client and Transport Reference

• Within a building/campus (< 10km)

• Single I/F / Fiber (Grey optics)

• POS, OTN & Ethernet

• Cost is King

Transport Network (OTN/DWDM)

TxP

Regen

Client Optics Line Optics

• Across city/state/country (100s or 1000s km)

• Multiple channels / Fiber (DWDM)

• OTN

• Performance is King

OA

Mux/Demux

ROADM

OA OA OA

40G/100G DWDM Optics

• Expect 40G/100G Deployments to:•Target 10G distances—1500 - 2000 Km reach

•Plug-and-play over existing 10Gig networks

•Must be spectrally efficient- 50GHz Grid

•Power / density / cost / performance trade off

• As Bit Rate increases the above becomes more challenging

• Reach & bandwidthSimplify deployment of 100Gig into 10Gig Systems

Choosing the Best Modulation FormatFrom 2.5Gb/s to 100Gb/s:

2.5G 10G 40G 100G > 100G

10G NRZASK

Variant:

Chirp

Electronic equalizer

MLSE

40G ODB40Gbs, ASK + PSK

40G DPSK40Gb PSK

40G A-DPSK40Gb PSK

A=Adaptive

40G RZ-QPSK20Gbaud, Q-PSK, RZ

40G CP-QPSK10Gbaud, pol mux,

Coherent Rx

100G CP-QPSK25Gbaud, pol mux,

Coherent Rx

--- ? ---

Most likely

Multilevel Tx

Polarization mux

Coherent Rx

e.g.

28 Gbaud 16QAM

--- ? ---

2.5G NRZASK

Only ONE modulation format is considered in the industry for 100G “lessons learned” from 40G

OIF activity to standardize the interface, components, …

Flex Mod

200G PM-16QAM

28 Gbaud/s

Nyquist shaped

100G PM-QPSK

50G PM-BPSK

Nyquist shaping and software configurable modulation

Coherent Optical Transmission Benefits

• Can avoid Dispersion Compensation Units (DCUs)

• No need to have precise Fibre Characterization

• Simpler Network Design

High Chromatic Dispersion (CD) Robustness

• High Bit Rate Wavelengths deployable on all Fibre types

• No need for “fancy” PMD Compensator devices

• No need to have precise Fibre Characterization

High Polarization Mode Dispersion (PMD) Robustness

• More capacity at greater distances w/o OEO Regeneration

• Possibility to launch lower per-channel Power

• Higher tolerance to Channels Interferences

Low Optical Signal-to-Noise Ratio (OSNR) Needed

How to increase the port density at high speeds?

• Solution must address power and footprint requirements;

• CMOS photonics;

• Next-generation transceivers:

• CPAK;

• CFP2, CFP4;

• CXP2;

• QSFP28;

• xXP;

• Systems on a chip – integrate more functions in less ASICs.

• Introduction





• Summary

Agenda

High-Level Comparison - End-User View

Cost Scalability/

Speed

Efficiency

L3/L1

(IMIX)

Operat.

Simplicity

E2E

OAM

Compat./

Interop. with

Transport

Ethernet

(LAN-PHY)

Ethernet

(WAN-PHY)

Ethernet

(w/ Ethernet OAM)

Eth. + G709

(Integrated)

POS

Best Worst

L3 Throughput Overview: Comparing 10Gbps Options

Note: Ethernet interfaces don’t take into account EOAM frames which can be considered additional overhead.

0

1,000,000

2,000,000

3,000,000

4,000,000

5,000,000

6,000,000

7,000,000

8,000,000

9,000,000

10,000,000

40 64 128 256 512 1024 1518 9618

L3

Data

Rate

(kb

it/s

)

Packet Size (bytes)

L3 Data Rate for Different 10Gbps Transport Technologies by packet sizes

PPP/HDLC Cisco HDLC Ethernet over ODU2 with GFP

Ethernet LAN-PHY Ethernet WAN-PHY Ethernet LAN-PHY 802.1Q

Ethernet WAN-PHY 802.1Q Ethernet LAN-PHY 802.1ad Ethernet LAN-PHY 802.1ah

Reference

L3 Throughput Overview (IMIX):Comparing Types of 10Gbps Interconnections

70.0%

75.0%

80.0%

85.0%

90.0%

95.0%

100.0%

7,000.00

7,500.00

8,000.00

8,500.00

9,000.00

9,500.00

10,000.00

PoS (PPP/HDLC)

PoS (Cisco HDLC)

LAN-PHY LAN-PHY with 802.1Q

WAN-PHY WAN-PHY with 802.1Q

Effi

cie

ncy

%

L3 D

ata

Rat

e -M

bit

/s

L3 Data Rate for Different 10Gbps Transport Technologies for IMIX Traffic

Total L3 Throughput (Mbit/s) Efficiency - L3 Data Rate vs Link Payload Capacity

Note: Ethernet interfaces don’t take into account EOAM frames which can be considered additional overhead.

Reference

• Introduction





• Summary

Agenda

Summary

• Business requirements are driving higher speeds in the WAN.

• Higher speeds can lead to higher cost and more complexity, pushing for technology innovations and new service paradigms.

• There are many options to migrate to higher speeds in the WAN, each one has it’s pros and cons. Goal is to understand and minimize the impact of the tradeoffs.

• Optical layer has evolved to support higher speeds (40Gbps and above) by adopting advanced modulations techniques or multiple lanes.

• There is a much more clear separation between client side optics and line side optics at speeds of 40Gbps and above.

Complete Your Online Session Evaluation

Don’t forget: Cisco Live sessions will be available for viewing on-demand after the event at CiscoLive.com/Online

• Give us your feedback to be entered into a Daily Survey Drawing. A daily winner will receive a $750 Amazon gift card.

• Complete your session surveys though the Cisco Live mobile app or your computer on Cisco Live Connect.

Thank you

WAN Branch Speed Mix

Source: Aryaka – State of Global Enterprise WAN Report - 2014

AcronymsAPC Automatic Power Control

BFD Bidirectional Forwarding Detection Protocol (IETF RFC 5880)

CAPEX Capital Expenditure

CRC Cyclic Redundancy Check

(D)WDM (Dense )Wave Division Multiplexing

E-LMI Ethernet Link Management Interface (Metro Ethernet Forum standard)

FCS Frame Check Sequence

FM Fault Management

GACH Generic Associated Channel (ref. MPLS-TP)

GE Gigabit Ethernet

GFP Generic Framing Procedure

GMP Generic Mapping Procedure

GMPLS Generalized Multiprotocol Label Switching

HDLC High-level Data Link Control Protocol

IA Implementation Agreement (related to MEF)

IGP Interior Gateway Protocol

IMIX Internet Mix (Mix of different IP packet sizes in typical Internet traffic)

ITU International Telecommunications Union

LAN Local Area Network

LSP Label Switched Path

AcronymsMAC Media Access Control

MAN Metropolitan Area Network

MEF Metro Ethernet Forum (www.metroethernetforum.org)

MP Multipoint

MPLS Multiprotocol Label Switching

MPLS-TP Multiprotocol Label Switching – Transport Profile

NNI Network-to-Network Interface

OAM Operations, Administration and Maintenance

OPEX Operational Expeditures

OTN Optical Transport Network

P2MP Point-to-Multipoint

P2P Point-to-point

PHY Physical Layer

PM Performance Management

PMD Physical Medium Dependant (related to IEEE 802.3 Ethernet Interfaces)

POS Packet over SONET/SDH

PPP Point-to-point Protocol

PW Pseudowire

SOAM Service OAM

AcronymsSONET/SDH Synchronous Optical Network Transport/Synchronous Digital Hierarchy

STP Spanning Tree Protocol

UNI User-to-Network Interface

VC Virtual Container

WAN Wide Area Network

High Speed WAN: Traditional and...

Documents

Transcript of High Speed WAN: Traditional and...