High Performance Switching and Routingtiny-tera.stanford.edu/~nickm/talks/Telecom_Center... ·...

Nick McKeownAssistant Professor of Electrical Engineeringand Computer Science

[email protected]://ee.stanford.edu/~nickm

High PerformanceSwitching and RoutingTelecom Center Workshop: Sept 4, 1997.

High Performance Switching and Routing of 52

Our Group

Shang-tse “Da” Chuang, Ken Chang,Pankaj Gupta, Youngmi Joo, Steve Lin,

Adisak Mekkittikul, Nick McKeown,Rolf Muralt, Kanta Yamamoto (visitor).


1. The Demand for Bandwidth

2. The Shortage of

3. The Architecture of Switches

Switching/Routing Capacity

and Routers

4. Some (of our) solutions


What’s the Problem?

Time

Mos

t thi

ngs

High Performance Switching and Routing of 52Source: http://www.mfsdatanet.com/MAE/west.stats.html

Feb Apr Jun Aug Nov

200

400

600

800

460

200150

123

Agg

rega

te b

andw

idth

(M

bps)

100

620

520

720

Feb Apr

The San Jose NAPThe demand


The supplyR

oute

r P

erfo

rman

ce (

pack

ets/

seco

nd)

103

19901986 1994 1997

104105

106


1986 1992 1997

pack

ets/

seco

nd

Demand

Supply

Why we need faster switches/routers


•Exponential growth in the number of users.•Exponential growth in traffic per user per hour.•Linear growth in hours per user per day.

Why the growth?


Time of day5am 8am

100%

30%

Dialup DemandModem usage at U.C. Berkeley

“America on Hold”

% o

f mod

ems

in u

se


Traffic Inversion10 years ago


ISP

ISP

Traffic InversionToday


November 1st, 1996

Why is this a problem?

Pac

ket L

oss

(%)


The race is on...

pack

ets/

seco

nd

Demand

Supply

- Gigabit Routers- IP Switching- Tag Switching- Layer-3 switches



2. The Shortage of



and Routers



The Architecture of

Forwarding

OutputScheduler

Signaling &

Generic Packet Processor:

Decision

Processor

Switches and Routers

Mgmt

Interconnect

(e.g. IP Router, ATM Switch, LAN Switch)

Data Hdr


Performance of IP Routers

Min back-to-back packet size Packet size

ArrivalTime

CopyTime

Forwarding

Time

Time

Decision


Performance of IP Routers

Min back-to-back packet size Packet size

Arrivalrate

Copyrate

Headerprocessingtime

Most routersdo this poorly!

Most routersdo this ~ ok

Time


The Evolution of RoutersThe first shared memory routers

RoutingCPU

LineCard

LineCard

LineCard

BufferMemory

DMA DMA DMA

MAC MAC MAC


The Evolution of RoutersReducing the number of bus copies

RoutingCPU

LineCard

BufferMemory

DMA

MAC

BufferMemory

RouteCache

DMA

MAC

BufferMemory

RouteCache

DMA

MAC

BufferMemory

RouteCache


The Evolution of RoutersReducing the number of bus copies

RoutingCPU

LineCard

BufferMemory

DMA

MAC

BufferMemory

RouteCache

DMA

MAC

BufferMemory

RouteCache

DMA

MAC

BufferMemory

RouteCache

updates


The Evolution of RoutersAvoiding bus contention

MA

C

Buffer

Mem

ory

Route

Cache

MAC

BufferMemory

RouteCache

MA

C

Buf

fer

Mem

ory

Rou

teC

ache

BufferMemory

CPUROUTE

Advantage:Non-blocking backplane—

Disadvantage:Difficult to provide QoS

high throughput


Multigigabit Routing

MA

C

Buf

fer

Mem

ory

BufferMemory

CPUROUTE

MAC

BufferMemory

MA

C

Buffer

Mem

ory

BBN’s Multigigabit Router

Buffer

Memory

CPU

ROUTE

2.4Gb/s

2.4Gb/s

2.4Gb/s

5Mpps

50Gb/sCrossbar



2. The Shortage of



and Routers



Some (of our)Solutions

1. Accelerating Lookups:• Label-Swapping

• Longest-matching prefixes

2. Switched Backplanes• Input Queueing

— Theory— Unicast— Multicast

• Fast Buffering

• Speedup

3.Our main project: The Tiny Tera


Routing Lookups

212.17.9.4

Class A

Class B

Class C212.17.9.0 Port 4

Class A Class B Class C D

ExactMatch

(hash, cache,pipeline...)

Routing Table:


212.0.0.0/8

212.17.0.0/16212.17.9.0/24

212.17.9.4

CIDR uses “longest matching prefix” routing:

Hashing, caching and pipelining are hard!

Routing Lookups withCIDR (“supernetting”)


Solution 1:Label Swapping

Forwarding

OutputScheduler

Signaling &

Decision

ProcessorMgmt

Interconnect

Data Hdr Label

NewLabel Port

DirectLookup

IP Switching, Tag Switching, ARIS, Cell-switched Router,....


Solution 2:Perform Lookups Faster!

Size ofRouting Tables

Cost of Memory(per byte)

Observation #1:

Time


Performing Lookups Faster

Prefix length

Numberin routingtable

24

212.17.9.0/24

212.17.9.40 232-1

256

Observation #2:


212.17.9.1 101

0

1

1

Port 4

Port 4Port 3

look further

look furtherPort 3

256Port 4Port 5

Port 4Port 5

16Mbytes of 50ns DRAM

<1Mbyte of 50ns DRAM

Solution 2 (cont):20 million lookups per second






• Fast Buffering

• Speedup



Shared Memory:

Input Queueing:

Should we use shared memory

Advantages:SimplicityHigh Bandwidth

Disadvantages:HOL BlockingLess efficient

Advantages:Highest Throughput.

Disadvantages:N-fold internal speed-up

N p

orts

Difficult to control packet delay.

or input-queueing?

Possible to control packet delay.

Shared Memory


TimeM

emor

y S

ize

Time

Mem

ory

Spe

ed

SRAM

DRAM

Memory Bandwidth


An aside: How fast can sharedmemory operate?

Route Lookup

SharedMemory

200byte packet

5ns SRAM

How fast can a 16 port switch run with this architecture?

5ns per packet 2 memory operations per cell time×aggregate bandwidth is 160Gb/s⇒


Because of ashortage of memory bandwidth, mostmultigigabit and terabit switches and routers useeither:

1. Input Queueing, or2. Combined Input and Output Queueing.

Should we use shared memoryor input-queueing?


1 24 21 2

Inputs Outputs

ρmax 2 2– 58%= =

Input Cell Buffer

Cells CellsN

The Problem A Solution....

Head of Line Blocking

“Virtual Output Queueing”

ρmax 100%=


Input 1

Q(1,1)

Q(1,n)

A1(t)

Input m

Q(m,1)

Q(m,n)

Am(t)

D1(t)

Dn(t)

Output 1

Output n

Matching, MA1,1(t)

?

....but requires scheduling...


RequestGraph

BipartiteMatching

1234

1234

1234

1234

(Weight = 18)

25

242

7

....which is equivalent to graph matching


Practical Algorithms

1. iSLIP — Weight = 1

2. iLQF — Weight = Occupancy

— Simple to implement

— Iterative round-robin

3. iOCF — Weight = Cell Age

4. MCFF — Weight = Backlog

Simple, fast,efficient

Good fornon-uniformtraffic.Complex!Good fornon-uniformtraffic.Simple!


Multicast TrafficQueue Architecture

2. Why treat multicast differently?1. Making use of the crossbar

3. Why maintain a single FIFO queue?4. Fanout-splitting

134


0.1

1

10

100

1000

10000

0.04 0.06 0.08 0.1 0.12 0.14 0.16 0.18 0.2 0.22 0.24

Ave

rage

Cel

l Lat

ency

Offered Load

Fanout-splittingNo Fanout-splitting

Fanout-Splitting


Multicast Traffic

1. Residue Concentration

2. Tetris-based schedulers


Gigabit and TerabitRouting





• Fast Buffering

• Speedup



Fast BufferingPing-pong Memory

BufferMemory

BufferMemory

BufferMemory

M

M/2

M/2



M

t

Occupancy

M/2

M/2

Maximum “cost” = M/2


In practise, cost <5%


Buffer size, M

Loss rate with

Loss rate with

single memory: M

ping-pong: (M/2,M/2)LossRate


Gigabit and TerabitRouting





• Fast Buffering

• Speedup



Combined Input- and Output-Queueing:

Matching Output Queueingwith Input- and Output- Queueing

How much speedup is enough?

k readsand writes


Conventional wisdom suggests:


How much speedup is enough?

A speedupk 2 4–= leads to high throughput


Fact To match output queueing, with FIFO input queues:k = N


=?Traffic

Output QueuedSwitch

Combined Input

Switchand Output Queued

Fact To match output queueing, with virtual output queues:k = 4 is sufficient

Conjecture: To match output queueing, with VOQs:k = 2 is sufficient






• Fast Buffering

• Speedup


High Performance Switching and Routingtiny-tera.stanford.edu/~nickm/talks/Telecom_Center... ·...

Documents

Transcript of High Performance Switching and Routingtiny-tera.stanford.edu/~nickm/talks/Telecom_Center... ·...