Core Network Evolution and IPoDWDM: 40G 100G and Beyond...(SONET, OTN, Ethernet, ESCON) Ethernet...

1 © 2010 Cisco and/or its affiliates. All rights reserved.

Core Network Evolution and IPoDWDM: 40G 100G and

Beyond Stefan Kollar Consulting Systems Engineer CCIE #10668 [email protected]

© 2010 Cisco and/or its affiliates. All rights reserved. 2

 OTN – Technical Foundation

 OTN – Properties and Impact on IP Layer

 MPLS-TP – Technical Foundation

 MPLS-TP Forwarding and OAM

 MPLS-TP Deployments

 40G/100G Design Considerations

 40G/100G Deployment Considerations in 10G Optical Networks

 100G - and beyond


Global IP Traffic Growth IP traffic will increase fivefold from 2008 – 2013

Source: Cisco Visual Networking Index—Forecast, 2008-2013

Other Services SONET/SDH Data Centre Private Line

IP packets will Dominate

IP Routed Services

L2 Packet Services

87% Consumer Traffic Video

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID 4

OTN – Technical Foundation


•  OTN defined a fixed “hierarchy” of payloads: from OTU1 (2.5G) to OTU3 (40G). Now ODU0 (1G) and OTU4 (100G) are being added

•  OTN started as a pure wrapper around WDM client signals to improve reach and manageability

•  Recently it has developed into a complex multiplexing structure that enables a service layer as well as TDM bandwidth mgmt

Frame Payload (OPU) ODU0 (coming) 1,238,954 kbit/s

OTU1 2,488,320 kbit/s OTU2 9,995,276 kbit/s OTU3 40,150,519 kbit/s

OTU4 (coming) 104,355,975 kbit/s


The OTN Multiplexing Hierarchy •  OTN Hierarchy

ODU0: ~1.22Gbps : GE ODU1: ~ 2.7Gbps : STM-16 ODU2 : ~10.7Gbps : STM-64 ODU3 : ~43Gbps : STM-256 ODU4: ~112Gbps : 100GE

ODU2e:~11.1Gbps : 10GE LAN PHY ODU3e:~40Gbps : 10GE onto 40G

•  ODU-FLEX Similar concept to VCATs in SONET/SDH but uses OTN containers Increments are variable (10Gbps, 1Gbps) but only one increment size per “link” 1.25Gbps increments is the most commonly discussed ODU-FLEX service .

GMP stands for the Generic Mapping Procedure that is currently under definition in Q11

Client

OD

U-F

LEX

GFP + idles

GFP

GFP

GFP

TS

TS

TS

TS

GM

P

Client

OD

U-FLEX GFP + idles

GFP

GFP

GFP

TS

TS

TS

TS

GM

P

OD

U-F

LEX

TS

TS

TS

TS

GM

P

TS

TS

TS

TS

GM

P

HO ODU HO ODU HO ODU HO ODU

Classifier

ODU-FLEX Switch ODU-FLEX Switch


Supported Ethernet Service Types per G.8011.x

OADM / Mesh DWDM node

G.709 formatted signal, OTUm

Multi-degree ROADM Cross Connecting Lambdas Dropping full lambdas

OTN HO Electrical Cross Connect Grooming and aggregation

Sub-lambda HO interfaces (SONET, OTN, Ethernet, ESCON)

Ethernet Switching Fabric

Ethernet interfaces (e.g. OC-3/STM-1)

Supported Ethernet Service Types per G.8011.x

  Point-to-Point Ethernet Private Line (EPL) Type 1

Ethernet Private Line (EPL) Type 2

Ethernet Private Line (EPL) Type 2 – timing transparent

Ethernet Virtual Private Line (EVPL) Type 1, 2, 3

  Point-to-Multipoint Ethernet Private Tree (EPT)

Ethernet Private LAN (EPLAN)

Ethernet Virtual Private Tree (EVPT), Type 1, 2, 3

Ethernet Virtual Private LAN (EVPLAN), Type 1, 2, 3


OTN – Properties and Impact on IP Layer


Encapsulation and OAM&P over optical spans (G.709 transponders)

Point to Point Multiplexing (G.709 muxponders)

OTN Grooming and Switching (OEO Cross Connects)

OTN OEO

OTN OEO

OTN OEO

OTN OEO

OTN OEO

OTN OEO

OTN OEO

OTN OEO

OTN OOO

OTN OOO

OTN OOO


Individual IFs

•  Works •  Cost

Channelized POS/OTN

•  No chOTN •  Cost

Ethernet w/ VLANs

•  Shaping •  Metrics


Primary Optical Path

Back Optical Path

Characteristics Optical level over-provisioning : 100% IP level over-provisioning : 0% Overall over-provisioning : 100% Complexity : High – need hold timers Complete IP level protection: No

Span Failure IP working : Yes

Primary Optical Path

Back Optical Path

Patch Cord, transponder or router card failure IP working : No

Primary Path failure


The Contrasting Topologies

L3 Routing Nodes

OTN ODU-FLEX switch

Internet

Internet

Business Data Centre


Internet

Internet



Video and Internet cache

Video and Internet cache

IP/TV Head End

IP/TV Head End

Hierarchical Solution Physical and Logical Topology the same

Bypass Solution

End User

End User


What’s the Impact at the Network Level ? Links and Over Provisioning

1Gbps Initial Demand 1Gbps increments

1Gbps Initial Demand 10Gbps increments

Network Wide Implications – 100 Node network

Worse over-provisioning : Links * b/w increment Network wide link upgrades = Links * Link upgrades

Worse case over-provisioning =200 * 10Gbps = 2000Gbps

Number network wide physical link upgrades = 200 * 0 = 0

Network Wide Implications – 100 Node network

Worse over-provisioning : Links * b/w increment Network wide link upgrades = Links * Link upgrades

Worse case over-provisioning =5000 * 1Gbps = 5000Gbps

Number network wide logical link upgrades = 5000 * 7 = 35000

Note : This does not take into account physical link upgrades

Link Level Over provisioning Link Level Over Provisioning

Network Efficiency and provisioning needs to account for total number of links, traffic growth, provisioning efficiency and upgrade frequency.

OTN – Technical Foundation


Client to OEO with SR optics

Client with SR Optics

Client with Integrated Optics

Photonics Bypass

Transponder Short Reach

Photonic Optical Approach

  Major IP demands are very large – often the driving force behind DWDM upgrades Full interfaces directly to the photonic layer of NG/ROADM Intelligent DWDM system Cut-through / bypass at the photonics layer

  Eliminates expensive high bandwidth opti-electrical components More pronounced with 40Gbps and 100Gbps

Transport OTN OEO Approach

Optical and in particular IPoDWDM is the most cost effective high b/w inter router connection


MPLS-TP – Technical Foundation


•  Evolution of SONET/SDH transport networks to packet switching driven by

Growth in packet-based services (L2/L3 VPN, IPTV, VoIP, etc) Desire for bandwidth/QoS flexibility

•  New packet transport networks need to retain same operational model

•  An MPLS transport profile being defined at IETF (in collaboration with ITU-T)

© 2010 Cisco and/or its affiliates. All rights reserved. 18 18

18

1: [RFC 5317]: Joint Working Team (JWT) Report on MPLS Architectural Considerations for a Transport Profile, Feb. 2009.

Definition of MPLS “Transport Profile” (MPLS-TP) protocols, based on ITU-T requirements

Note: IETF decided to support single MPLS-TP OAM solution. IETF Chair stated at IETF 79 (11/2010) and IETF 80 (3/2011)

Derive packet transport requirements

Integration of IETF MPLS-TP definition into transport network recommendations

IETF and ITU-T agreed to work together and bring transport requirements into the IETF and extend IETF MPLS forwarding, OAM, survivability, network management, and control plane protocols to meet those requirements through the IETF Standards Process.[RFC5317]1

ITU-T withdrawal of T-MPLS draft G.8114 in Jan. 2008.


•  MPLS-TP is proper subset of MPLS proper

•  So far, no „MPLS-TP only“ functionality standardized

MPLS Solution Space • ECMP • MP2Pt • LDP, IP

MPLS-TP Solution Space • PHP default disabled

MPLS-TP Only Solution Space


•  Connection-oriented packet switching model

•  No modifications to MPLS data plane

•  Interoperates/interworks with existing MPLS and pseudowire control and data planes

•  No LSP merging or PHP

•  LSPs may be point to point (unidirectional, co-routed bidirectional or associated bidirectional)

•  LSPs may be point to multipoint (unidirectional)

•  Networks created and maintained using static provisioning or a dynamic control plane: LDP for PWs and RSVP-TE (GMPLS) for LSPs

•  In-band OAM (fate sharing)

•  Protection options: 1:1, 1+1 and 1:N

•  Network operation similar to existing transport networks

See RFC 5654


MPLS-TP Forwarding Plane


•  Static

•  Bidirectional

•  Co-routed (same forward and reverse paths)

•  In-band Generic Associated Channel (G-ACh)

•  Ultimate hop popping (no explicit/implicit null)

•  No ECMP

•  Contained within a tunnel

MPLS-TP LSP

G-ACh MPLS-TP Tunnel


MPLS-TP

•  No IP routing required in control and forwarding planes

•  Node will still source/terminate IP packets (e.g. SNMP, NTP)

•  Link numbers required on each MPLS-TP interface

•  Two interface configuration models IP-enabled (uses ARP) IP-less (no ARP)

•  IP-enabled requires interface configuration for

Local IPv4 address Remote IPv4 (next-hop) address

•  IP-less requires configuration of Destination MAC address

•  Same OAM messages for IP-enabled and IP-less interface configurations


MPLS-TP Tunnel

Protect LSP

G-ACh G-ACh

Working LSP

•  MPLS-TP tunnels abstracted as Tunnel-tp interface

•  Tunnel holds a working LSP and a protected LSP

Working Protect (optional)

•  Tunnel may be configured with a bandwidth allocation

•  Tunnel operationally UP if at least one LSP operationally UP (and not locked out)

•  LSP operationally UP if OAM (Continuity Check) session operationally UP

•  LSP requires static configuration of LSP label imposition (output label and output link)

•  LSP requires static configuration of LSP label disposition (input label)

•  LSP must be co-routed (no embedded check)


•  Same OAM functions for LSPs, pseudowires and sections

•  In-band OAM packets (fate sharing)

•  OAM functions can operate on an MPLS-TP network without a control plane

•  Extensible framework with current standardization focus on fault and performance management

•  Independent of underlying technology

•  Independent of PW emulated service


•  OAM capabilities extended using a generic associated channel (G-ACh) based on RFC 5085 (VCCV)

•  A G-ACh Label (GAL) acts as exception mechanism to identify maintenance packets

•  GAL not required for pseudowires (first nibble as exception mechanism)

•  G-ACh used to implement FCAPS (OAM, automatic protection switching (APS), signaling communication channel, management communication channel, etc)

ACH OAM

Payload

GAL Label

Associated Channel Header Generic Associated Channel Label (GAL)

PW Associated Channel Header (ACH)

ACH OAM

Payload

Label PW Label

0 0 0 1 Version

RFC 5586

RFC 5085

IETF

LSP

G-ACh

PW G-ACh

Reserved Channel Type


•  Relies on a disjoint working and a disjoint protect path between two nodes

•  Provides 1:1 protection (only one active LSP) in revertive mode

•  Functionally similar to path protection in IP/MPLS

•  Protection switching can be triggered by Detected defect condition (AIS/LDI, LKR) Administrative action (lockout) Far end request (lockout) Server layer defect indication (LOS) Revertive timer (wait-to-restore)

PE1 PE2

P2

P1

Working LSP (Up, Active)

Protect LSP (Up, Standby)

PE1 PE2

P2

P1

Working LSP (Down, Standby)

Protect LSP (Up, Active)

Working LSP (Up, Active)

Protect LSP (Up, Standby)

Working LSP (Down, Standby)

Protect LSP (Up, Active)

Before Failure

During Failure


Function Description Tool

Continuity Check Checks ability to receive traffic BFD

Connectivity Verification Verifies that a packet reaches expected node

BFD (proactive)

LSP Ping (on-demand)

Diagnostic Tests General diagnostic tests (e.g. looping traffic) New

Route Tracing Discovery of intermediate and end points LSP Ping

Lock Instruct Instruct remote MEPs to lock path (only test/OAM traffic allowed) New

Lock Reporting Report a server-layer lock to a client-layer MEP New

Alarm Reporting Report a server-layer fault to a client-layer MEP New

Remote Defect Indication Report fault to remote MEP BFD

Client Failure Indication Client failure notification between MEPs PW Status

Packet Loss Measurement Ratio of packets not received to packets sent New

Packet Delay Measurement One-way / two-way delay (first bit sent to last bit received) New

IETF


MPLS-TP deployements


Flexible Edge and Services Architecture

Multiservice Core!Aggregation! Edge! Core!Static MPLS-TP Access

IP/MPLS Access (L2 and CE only)

Ethernet Access

IP/MPLS + Dynamic MPLS-TP IP/MPLS IP/MPLS

Pseudo-Wire Switching MPLS-TPIP/MPLS L3 IP Edge and Service Placement

  Flexible IP Edge and Service Placement – Agg, Edge or Core

  IP/MPLS in Core / Edge / Aggregation

  Use MPLS-TP toolbox to enhance dynamic IP/MPLS domain

  Variety of Access options – static MPLS-TP, Ethernet, IP/MPLS

  Common protocols and control plane aggregation to aggregation

Circuit Emulation + Ethernet


Centralised Edge and Transport Architecture

Multiservice Core!Aggregation! Edge! Core!MPLS-TP Access

IP/MPLS Access (L2 and CE only)

Ethernet Access

IP/MPLS (L2 and CE only) IP/MPLS

  Centralised IP Edge and Service placement : Edge, Core

  IP/MPLS in Core / Edge / Aggregation

  Variety of Access options – static MPLS-TP, Ethernet, IP/MPLS

  Common protocols and control plane aggregation to aggregation

  Ethernet Access / IP/MPLS L2 Aggregation : Widely deployed model

IP/MPLS

L3 IP + Services Placement



Centralised Edge and Transport Architecture

Aggregation! Edge! Core!

Ethernet Access Static MPLS-TP IP/MPLS IP/MPLS

Static MPLS-TP Access

L3 IP + Services Placement

  Centralised IP Edge and Service placement - Edge, Core

  IP/MPLS in Core / Edge

  Static MPLS-TP in Aggregation

  Variety of Access options : Ethernet / Static MPLS-TP

  Common forwarding protocols aggregation to aggregation

  End to end transport operations using pseudo-wire switching



40G/100G Design Considerations


Solutions for Implementing 100G DWDM

•  All lambdas upgraded to 100Gbps •  Sub-100G services provided by OTN OEO

Advantages All lambdas on a fibre are 100G

Disadvantages 100TXP investment upfront Need an additional OTN OEO All 10G TXPs are obsolete

10G and 100G DWDM Coexistence

  10G and 100G lambdas co-exist on same fibre   Packet uses 100G, everything else 10G

Advantages Only high demand clients upgraded to 100G Protects existing 10G DWDM investment Lowest cost per bit (100G TXPs>10 x10G TXPs)

Disadvantages Need a guard band between 10G and 100G frequencies Not appealing in ULH environments

100G lambda

10G lambda

100G lambdas

OTN OTN

10G SR

100G SR

OTN Multiplexing


•  40 Gig and above rates must meet minimum requirements: Target 10 Gig distances—1500 Km reach Not simply a Greenfield technology, but plug and play over existing 10Gig networks Must be as open as possible, operate over third party DWDM networks Must operate over both 100GHz as well as 50GHz spacings Power and footprint must be reasonable, can not redesign Router/transport shelf due to blade

•  To achieve must leverage/control: 1. Optical Impairments 2. Modulations schemes


•  Attenuation Loss of signal strength Limits transmission distance

•  Chromatic Dispersion (CD) Distortion of pulses Limits transmission distance Proportional to bit rate

•  Optical Signal to Noise Ratio (OSNR)

Effect of noise in transmission Caused by amplifier Limits number of amplifier


•  Polarization Mode Dispersion (PMD)

Caused by nonlinearity of fiber geometry Effective for higher bit rates (10G)

•  Four Wave Mixing (FWM) Effects in multichannel systems Effects for higher bit rates

•  Self/Cross Phase Modulation (SPM, XPM)

Effected by high channel power Effected by neighbor channels

Spreaded Pulse as It Leaves the Fiber


•  Cisco expects 100G Deployments to: Target 10G distances—1500 - 2000 Km reach Plug-and-play over existing 10Gig networks Must be spectrally efficient- 50GHz Grid Power / density / cost / performance trade off

•  As Bit Rate increases the above becomes more challenging

Simplify deployment of 100Gig into 10Gig Systems


Transmitter Decrease Speed – reduce $$ Increase Modulation - Increase spectral efficiency Increase Optical efficiency – Increase spectral efficiency

Receiver Move from Direct Detection to Coherent detection Compensate for Optical impairments in Electrical Domain(DSP) – reduce $$

Forward Error Correction (FEC) Move to Higher coding gain FECs – Increase reach

100Gig was our first challenge – overcame with PM-QPSK and new FECs


•  Need to go slower Optical impairments are directly related to signaling rates

•  Need to increase modulation efficiency Signaling speed decreases & Information Rate increases NRZ to ODB to (D)PSK to (D)QPSK

•  Need to increase optical efficiency Split signal over two polarizations (PM – Mod Scheme)

1 bit/symbol

NRZ

0 1

1 bit/symbol

PSK

1 -1

2 bits/symbol

QPSK

00

10 11

01


•  RX Laser behaves as Local Oscillator to provide a Polarization reference

•  90° Hybrid:

•  Converts Phase modulation in Amplitude modulation •  Signal Processor:

•  Recovers Polarization •  Compensates CD and PMD electronically


•  Utilizing Cisco’s advanced Optics and FEC

•  Distances up to 2000Km and beyond

•  Operates over existing Infrastructure at both 50 / 100GHz

100G PM-QPSK – At and / or exceeds 10Gig System Performance

PM-QPSK allows 100G operation over existing and greenfield networks at 10Gig distances


40G/100G Deployment Considerations in 10G Optical

Networks


3rd PARTY DWDM SYSTEM MUST SUPPORT ALIEN WAVELENGTHS!

Alien/foreign wavelength is any 3rd party ITU wavelength operating over an existing DWDM infrastructure.

G698.2 – Standard for “Alien/Foreign waves” defines:

properties for signal sources and sinks properties for DWDM links for “black links” (i.e. alien wavelengths)


Design Considerations •  40G receiver differs from 10G

Noise and Impairment Limits

40G IPoDWDM Transponder

(DPSK+) 10G Transponder

Launch Powers 0 dBm 0 dBm

Rx Windows 5 to –18 dBm 0 to –23 dBm

OSNR (.1nm) ~ 14.5 dB ~ 15 dB

CD +/- 750ps/nm +/- 2000ps/nm

PMD 2.5ps 10ps


•  One of the biggest challengers in Poland (~350k broadband users)

•  Implemented XR12000 core 1.5 year ago (one of the first XR12000 production networks in the world)

•  BB explosion has made their traffic grow quicker than expected => need to upgrade main nodes

•  Customer A expected big CapEx/OpEx savings


•  IPoDWDM CapEx/OpEx reduction Eliminates Transponder Shelf 4:1 Capacity Savings

•  Conducted successful tests on 40G IPoDWDM on Warsaw -> Poznań link (614 km) link designed for 10G optical


100G

Where are we today?


•  IEEE 802.3ba: 40Gb/s and 100Gb/s Ethernet Task Force 40G and 100G Ethernet Physical interfaces for Backplane, Copper, Fiber PMDs

•  IEEE 802.3bg: 40Gb/s SMF Ethernet Task Force 40G Serial PMD optimized for carrier applications

•  ITU Study Group 15: Optical and Transport Networks OTU4 frame format Single mapping for 40GE/100GE into OTU3/OTU4 OTL protocol enabling OTU3/4 over multi-lane (low cost) optics

•  OIF: 100G Long-distance DWDM Transmission Industry consolidation around a single 100G DWDM solution

Ratified

Ratified

Ratified

Ratified


•  Customer demands are driving the need for 100Gig and beyond Video – HD / 3D Video Conferencing – HD / 3D Gaming – HD / 3D

•  Packet will dominate 28% of population connected 14% of population broadband

•  100Gig is deploying NOW Content Providers Tier One SPs and MSOs

•  Mass deployments 2nd Half 2012 early 2013

Question is not “When 100Gig?” but rather “What is after 100Gig?”


•  Higher data rates 200Gig, 400Gig, 1T?

•  Need to investigate other modulation techniques PM-16QAM, PM-64QAM, …. or CO-OFDM ?

•  Need deeper look at FEC Advanced FEC What other algorithms are there

•  Need of intelligent DWDM layer Flex spectrum Control plane Advanced operations, troubleshooting and protection mechanisms

•  Must a channel really fit into 50GHz spacing? Or should it be gridless?


  Information distributed over a few Sub-Carriers spaced as closely as possible forming a 1,000Gbps Super-Channel

  Each Sub-Carrier transporting a lower Bit Rate, compatible with current ADCs and DSPs

-200 -150 -100 -50 0 50 100 150 200 f [GHz]

|Sch

(f)| 2

10x 100Gbit/s Sub-Carriers close-to-Baud-rate spaced

Super-Channel #1 Super-Channel #2 Super-Channel #3


•  Sub-Carrier spacing: 1.2 times the Baud Rate

•  Different approaches for 1,000Gb/s: CP-QPSK: 10 Sub-Carriers at 111 Gbit/s each

back-to-back sensitivity 12 dB CP-8QAM: 8 Sub-Carriers at 138.75 Gbit/s each

back-to-back sensitivity 16.1 dB CP-16QAQM: 5 Sub-Carriers at 222 Gbit/s each

back-to-back sensitivity 19.1 dB •  System Configuration:

Span of 90km each (ITU-T G.652) Span Insertion Loss: 25dB

CP-16QAM

CP-8QAM

CP-QPSK

1.04 1.08

1.12 1.20

1.44 1.8 1

1.04 1.08 1.12 1.2

1.44 1.8

1.04

1.08 1.12

1.2 1.44

1.8

with subcarrier spacings

Thank you.

Core Network Evolution and IPoDWDM: 40G 100G and Beyond...(SONET, OTN, Ethernet, ESCON) Ethernet...

Documents

Transcript of Core Network Evolution and IPoDWDM: 40G 100G and Beyond...(SONET, OTN, Ethernet, ESCON) Ethernet...