High Speed Optical Networks:

An Evolution of Dependency

November 2, 2001

Todd Sands, Ph.D

WEDnet Project

www.wednet.on.ca

University of Windsor

Latency

• The result of an event in time that slows the transport or processing of information

• E.g. Machine (processing) latency in microsecs (n =1.2)

• E.g. Network latency in millisecs (x < 130 ms)• Optical transport max. = 300,000 km/sec• Physical parameters of the transport media • Convergence of voice, image and data in the path• Switched cells and packet network behaviours• Potential of WDM optically switched and SONET

architectures

OSI Reference Model – Networking 101

• Application• Presentation• Session• Transport• Network• Data-link• Physical

When two computers communicate on a network, the software at each layer on one computer assumes it is communicating with the same layer on the other computer.

e.g. For communication at the transport layers, that layer on the first computer has no regard for how the communication actually passes through the lower layers of the first computer, across the physical media, and then up through the lower layers of the second computer.

Do we know the effects of latency!

Suspect that the answer is yes! We see it every day! No. of processors, power requirements, processing capability, storage

capacity, and the needs of research that use most facilities can be intensive. HPCS resources supplied and funded through a needs-based process, but

this can also be because of research What about a GRID? Is it on the same path? Are we mindful of details, such as latency…with respect to one of the most

fundamental parts of the GRID… THE NETWORK Do we know how computing resources connect to the outside world?…

Maybe… Do we have any control over the “extranet”?

Primary Network InterfaceTo Machine Resources

These switches provideEthernet to ATM SONET WAN interfacesfor TCP/IP traffic

PACKETS VS. CELLS VS. FRAMES

• Frames – used for larger data amounts over high-speed, low error rate links– 2,000 – 10,000 characters in size– Data corrections not link by link– Therefore link by link error checking impacts network latency greatly

• Packets – used for smaller data amounts across lower speed, high error rate links– 128 – 256 (bytes) characters in size– Lower chances of error in each packet, small amounts re-transmitted– Prioritization through tagging of packets leads to QoS

• Cells – very small amounts of data with sometimes no error checking– Highly reliable optical networks sometimes with no error checking– Up to 48 - 53 (bytes) characters in size– Small size allows for load balancing of traffic on network– No payload in cells, no transmission - full payload, then transmission– Uses ATM Adaptation Layers – AAL’s 1-5 for shaping the network

Optical Carrier Designations

• OC-1/STS-1 51.84 Mbps

• OC-3 155.52 Mbps

• OC-12 622.08 Mbps

• OC-48 2,488.32 Mbps

• OC-192 9,953.28 Mbps

• OC-768 39,813.12 Mbps

SONET• digital hierarchy based on Optical Carriers

(OC’s)

• maximum t-speed of 39.81312 Gbps

• defines a base rate of 51.84 Mbps = STS-1s

• OC’s are multiples of the t-speed

• defines Synchronous Transport Signals – STS’s and STS-3c = OC 3 = 155 Mbps

Overheads

• SONET carries 8,000 frames per second, 810 characters in size (36 characters of overhead and 774 characters of payload

• Section Overhead includes:– STS channel performance monitoring

– Data channels for management such as channel monitoring, channel administration, maintenance functions and channel provisioning

– Performs functions necessary for repeaters, add drop multiplexers (ADMs), termination gear, and digital access and cross connect systems (DACS)

• Line Overhead includes:– STS-1c performance monitoring

– Data channel management, payload pointers, protection switching information, line alarm signals, and far-end failure to receive indicators

• In addition to these overheads there are also Path overheads

Optical Wave Division• WDM multiplies (up to 32 more times) the capacity of

existing fibre spans – cross (wide)-band, narrow band or dense band transmission options

• DWDM Red waves 1550, 1552, 1555 & 1557 nm

• DWDM Blue waves 1529, 1530, 1532& 1533 nm

• Now can support 100 wavelengths with each wavelength supporting a channel rate of up to 10 Gbps

Local Area Access Architectures

ATM Network – OC12-OC48

GbE

Access Routers

1MM

System Processors and Interfaces 100 Mb- 1Gb

Router OC-12ATM

GbE

1MM

1-Meg or xDSL Modem Services in Communities

PVCs – on carriers network

Alternate CarrierMANs also Interface

1000 Mb GbE

Grid Access Node – GigaPoP?

Off Ramps -WDM

Central CO for Access Nodes

All PVCs (SVCs or PVPs) usually terminate on 1 or more Centralized Access Routers

Most carrier PVCs are UBR with access at minimum OC48 speeds 2.4 Gb/sec

Backbone may be optically switched with P.O.S on wavelengths using TCP/IP as the

main transport protocol but getting direct access to it is the key!

Direct access will also minimize latency and the synergistic effects of latency

What does a 5 minute average measurementshow us with MRTG?

PC 1MM

EthernetSwitch

(Catalyst)Network

(ATM)LAC

(SMS-1000)1MMDBIC

Network Protocol Stack Models (WAN with IP)

SONET/SDH1MMQAM

ATMATM

SARSAR

AAL5AAL5

LLC/SNAP(1483)

Ethernet

1MMQAM

SONET/SDH

ATMATM

SONET/SDH

ATMATM

SARSAR

AAL5AAL5

LLC/SNAP(1483)

Ethernet

PPPOEPPPOE

SONET/SDH

ATMATM

SARSAR

AAL5AAL5

LLC/SNAP(1483)

IP

UDP

L2TP

Ethernet Ethernet

10BaseT

Ethernet

PPPOEPPPOE

PPPPPP

IP

10BaseT

Ethernet

SONET/SDH

ATMATM

SARSAR

AAL5AAL5

LLC/SNAP(1483)

IP

UDP

L2TP

PPPPPP

IP

LNS

Making a CallTelevision

Video

V-Room

LE25

WEDnet uses WUC asa carrier such as Bellor METROnet with core gearLS1010 and 7200 series forATM and IP routing

TelevisionVideo

V-Room

LE25

IBM 8274 9 slot

LE 25 SMF

P-Tel Video

FVC VGATE

Universityof

Windsor

SharedH.261 ISDN

25 Mb ATM

OC3

Dial - up

FVC V-room

WRH WesternCampus

HDGH

LE25

Making a CallTelevision

Video

V-Room

LE25

WEDnet uses WUCas a carrier suchas a Bell or METROnetwith core gear LS1010 and 7200 series for ATM andIP routing

TelevisionVideo

V-Room

LE25

IBM 8274 9 slot

LE 25 SMF

P-Tel Video

FVC VGATE

Universityof

Windsor

SharedH.261 ISDN

25 Mb ATM

OC3

Dial - up

FVC V-room

WRH Western

Campus

HDGH

LE25

Making a CallTelevision

Video

V-Room

LE25

WEDnet uses WUC asa carrier such as Bellor METROnet with core gear LS1010 and 7200 series for ATM andIP routing

TelevisionVideo

V-Room

LE25

IBM 8274 9 slot

LE 25 SMF

P-Tel Video

FVC VGATE

Universityof

Windsor

SharedH.261 ISDN

25 Mb ATM

OC3

Dial - up

FVC V-room

WRHWestern

Campus

HDGH

LE25

Codec NegotiationTelevision

Video

V-Room

LE25

WEDnet uses WUC asa carrier such as a Bellor METROnet with core gearLS1010 and 7200 series for ATM and IP routing

TelevisionVideo

V-Room

LE25

IBM 8274 9 slot

LE 25 SMF

P-Tel Video

FVC VGATE

Universityof

Windsor

SharedH.261 ISDN

25 Mb ATM

OC3

Dial - up

FVC V-room

WRHWestern

Campus

HDGH

LE25

Successful CallTelevision

Video

V-Room

LE25

WEDnet uses WUC asa carrier such as a Bell or METROnet with core gear LS1010 and 7200 series forATM and IP routing

TelevisionVideo

V-Room

LE25

IBM 8274 9 slot

LE 25 SMF

P-Tel Video

FVC VGATE

Universityof

Windsor

SharedH.261 ISDN

25 Mb ATM

OC3

Dial - up

FVC V-room

WRHWestern

Campus

HDGH

LE25

Making an ISDN Call

TelevisionVideo

V-Room

LE25

WEDnet uses WUCas a carrier such as a Bellor METROnet with core gearLS1010 and 7200 series forATM and IP routing

TelevisionVideo

V-Room

LE25

IBM 8274 9 slot

LE 25 SMF

P-Tel Video

FVC VGATE

Universityof

Windsor

SharedH.261 ISDN

25 Mb ATM

OC3

Dial - up

FVC V-room

WRHWestern

Campus

HDGH

LE25

Making an ISDN Call

TelevisionVideo

V-Room

LE25

WEDnet uses WUC asa carrier such as a Bellor METROnet with coregear LS1010 and 7200series for ATM and IProuting

TelevisionVideo

V-Room

LE25

IBM 8274 9 slot

LE 25 SMF

P-Tel Video

FVC VGATE

Universityof

Windsor

SharedH.261 ISDN

25 Mb ATM

OC3

Dial - up

FVC V-room

WRHWestern

Campus

HDGH

LE25

Making an ISDN Call

TelevisionVideo

V-Room

LE25

WEDnet uses WUC asa carrier such as a Bellor METROnet with coregear LS1010 and 7200series for ATM and IProuting

TelevisionVideo

V-Room

LE25

IBM 8274 9 slot

LE 25 SMF

P-Tel Video

FVC VGATE

Universityof

Windsor

SharedH.261 ISDN

25 Mb ATM

OC3

Dial - up

FVC V-room

WRHWestern

Campus

HDGH

LE25

TelevisionVideo

V-Room

LE25

WEDnet uses WUCas a carrier such as a Bellor METROnet with core gearLS1010 and 7200 series for ATM and IP routing

TelevisionVideo

V-Room

LE25

IBM 8274 9 slot

LE 25 SMF

P-Tel Video

FVC VGATE

Universityof

Windsor

SharedH.261 ISDN

25 Mb ATM

OC3

Dial - up

FVC V-room

WRHWestern

Campus

HDGH

LE25

Making an ISDN Call

TelevisionVideo

V-Room

LE25

WEDnet uses WUCas a carrier such as a Bell or METROnet with coregear LS1010 and 7200 seriesfor ATM and IP routing

TelevisionVideo

V-Room

LE25

IBM 8274 9 slot

DEC Gigaswitch18 gbps

LE 25 SMF

P-Tel Video

FVC VGATE

Universityof

Windsor

SharedH.261 ISDN

25 Mb ATM

OC3

OC3

Dial - up

FVC V-room

WRHWestern

Campus

HDGH

LE25

TelevisionVideo

Centrex module

Leamington District Memorial Hospital

Codec Negotiation

TelevisionVideo

V-Room

LE25

WEDnet uses WUC asa carrier such as a Bellor METROnet with core gear LS1010 and 7200 series for ATM and IP routing

TelevisionVideo

V-Room

LE25

IBM 8274 9 slot

LE 25 SMF

P-Tel Video

FVC VGATE

Universityof

Windsor

SharedH.261 ISDN

25 Mb ATM

OC3

Dial - up

FVC V-room

WRHWestern

Campus

HDGH

LE25

TelevisionVideo

Centrex module

LeamingtonDistrict MemorialHospital

Successful Call

April 20, 2023 Bhavani Krishnan

AT&T and Regional Gigapop

IP Architecture

ATM interconnectivity

Router / RFC1577 Client

LAN interconnect

WEDNetSureNet

ATM/w SVC

AT&T RouteServer

AT&T Gigapop

AT&T Network

CA*net3

OCRINet /wOHIiB

GP

iBGP

iBG

P

AT&T AS iBGP

CA*Net AS iBGP

Regional IGP

Reg

iona

l IG

P

Reg

iona

l IG

P

BGPiBGP

From LAN to WAN

This server and control facility houses multiple Digital Alpha, DEL PowerEdge, IBM Netfinity and RS/6000 servers. Located at a single campus the facility supports 400 nodes locally and 800 nodes 7.5 km away. SVCs are provisioned on separate PVPs for security and LANE services provide VLANs for ADT systems, pharmacy, and document imaging. The systems use GUI interfaces to assist visual references for end-users

In the Ideal World!

Dark fibre between nodes Homogenous switched architecture with minimal breakouts Low latency at all layers

We will likely be dealing with something much different, unless there is about

$500 M available to support and sustain the network side of grids to help

minimize the synergistic effects of latency on applications Latency studies are important and the synergy of latency effects are

important from the processor to the I/O architectures, to the network layers If commercial carriers are to be used anywhere in the path, latency should

become a factor for selecting them as providers Effective monitoring and support of the extranet is important to the success

of a GRID unless the GRID middleware can accommodate different types of

latency and the variation that exists Internet routing is “best effort” with variable paths every time – not likely the

best GRID platform Research networks like CA*net 3, Internet 2, ORION, etc. are the next best

bet! However, the last mile issue still has to be addressed.

The Future• “It is conceivable that future Internet networks may be a seamless composite of a variety of transport

protocols. An Optical Internet might be used for high volume, best efforts computer to computer traffic, while IP over ATM might be used to support VPNs and mission critical IP networks, while IP over SONET would be used to aggregate and deliver traditional IP network services that are delivered via T1s, DS3s, and Gigabit uplinks”

• From, Dr. Bill St. Arnaud, CANARIE