Stability of Hop by Hop Congestion Control

Stephan Bohacek

Dept. of Mathematics

University of Southern California

End to End and Hop by Hop Congestion Control

End to End Congestion Control (e.g. TCP)– The sender determines the rate at which it sends data based on estimates of

congestion derived from information from the receiver.• There may be a large delay between when congestion occurs and when the

sender reacts.100KBps link with 250ms delay. TCP takes 1.5 secs to reach full speed. In this time 150 KB could have been sent. Most http files are smaller than 150KB. Hence, TCP is usually under utilizing the link.

• A large number of packets may be dropped.

Hop by Hop Congestion Control– Each router negotiates with its neighbors to decide how fast data should be sent.

• Little delay• No estimation is necessary• Few drops• Burden on the routers• Stability is an issue (distributed control)

D1

S1

Connection Oriented Approach to Hop by Hop Congestion Control

Neglect all other links except the ones involved in the connection.

Control information (e.g. sending rate, queue size) about this particular connection is exchanged between neighboring nodes.

qi,i+1(k+1) = L (qi,i+1(k) + ri-1,i(k) – ri,i+1(k))

ri-1,i

node i-1

qi-1,i

ri,i+1

node i

qi,i+1

ri+1,i+2

node i+1

qi+1,i +2

data flow from node i to node i+1

queue occupancy for flowfrom node i-1 to node i

queue dynamics

ri,i+1(k+1) = L (ri+1,i+2(k) + F(q* - qi+1,i+2(k)))rate dynamics

Objective: match rates, ri,i+1=M, and make qi+1,i+2 = q* “optimal queue size”

rN(k+1) = M

M = bandwidth of bottleneck link / number of competing flows

“bottleneck link”

=-F1

11z-1

0 -1

0 0

q

rσ-1

q

r

q

r+

temporalshift

spatialshift

It can be shown that this closed loop system is stable (Mishra 1996).

Drawbacks• The optimal rate M? (Why not just feed M back directly?)• Optimal queue occupancy q*? Hence, optimal delay?• The router must maintain flow control for each connection (per VC

congestion control).E.g., a 10Gbps router can accommodate 200,000, 56Kbps connections. If each buffer can hold 20 packets, then the total memory is 48Gb! For this reason, the ATM steering committee did not choose per VC congestion control for ABR (Jain 1996).

non-connection oriented approachPazos and Gerla (Infocom, 1999)

Kulkarni, Bohacek, and Safonov (Allerton, 1998)

ri,j = - ri,j + ri,j

The rate that data exits node i to other nodes.

The rate that data enters node i

from other nodes.

aggregate queue occupancy in node i

rate that node i sends data to

node j

rate that data enters/leaves the

network through node i

qi = - j i,,j ri,j + j j,i rj,i + ui

rate controller

queue dynamics(conservation of data)

average data rateoptimal queue size

ri,j = - (ri,j - ri,j) – F1 (qj – q*) + F2 (qi – q*)

Define i,,j = 0 otherwise

1 if node i sends data to node j

q = Ar + u

The close-loop LTI system has one eigenvalue at zero and the rest have negative real parts.

The eigenvalue at zero corresponds to the fact that if i ui > 0, then more data enters the network then leaves the network. The excess data accumulates in the queues.

non-connection oriented approach

A link is an output from one node and it is an input to another.Hence, columns of A sum to zero.

Blocking

A B

C

D E

fast linksfast link

slow link

Data flows from A to D and from B to E. Since link C, E is slow, the queue in node C fills. Both flows slow down to alleviate the filling queue.

input rates

ri,j = L (F1qi,j + F2qi,j – F3 ki,j,k qj,k – F4 ki,j,k qj,k + F5 hh,i,j qh,i + F6 hh,i,j qh,i)

Define i,j,k to be the ratio of data going from

node i to node j that will then go to node k.

qi,j = L (- ri,j + hh,i,j rh,i + ui,j)

queue occupancy for link i, j

increase rate if local queue is full or is filling

decrease rate if downstream queue is full or is filling (Back Pressure)

increase rate if upstream queue is full or is filling(Forward Pressure)

Node i, j adjust its rate only when directly responsible for congestion. This should alleviate the blocking problem.

No preset values.

Inputs and Outputs

IN 1 2 OUT

network under control

q1,2 = L( -r1,2 + rIN )

r1,2 = L(F1q1,2 + F2(-r1,2 + rIN) – F3q2,OUT - F4(-r2,OUT + r1,2))

q2,OUT = L( -r2,OUT + r1,2 )

r2,OUT = L(F1q2,OUT + F2(-r2,OUT + r1,2) + F5q1,2 + F6(-r1,2 + rIN))Like connection oriented Hop-by-Hop, this topology is stable for all Fi > 0 but queues are empty at equilibrium.

No preset values.

ki,j,k 1 the data flowing along link i,j must next either

flow along some other link j, k or must exit the network.

1 3

2

u1u3

u2

Suppose that

1,2,3 = 2,3,1 = 3,1,2 = 1.

Then all the data that enters the network remains in the network and queues overflow. We require

1,2,3 ·2,3,1 · 3,1,2 1

Define i,j,k to be the ratio of data going from

node i to node j that will then go to node k

No – Loop Condition

1 3

2

u1

u2

5

4

u4

u3u5

If 1,2,3 = 3,1,2 = 3,4,5 = 4,5,3 = 1

2,3,1 = 2,3,4 = 5,3,4 = 5,3,2 = 0.5,

then data that enters the network never leaves. We require that

1

, ,:,,

Lji Lkjkkji

for all sequence of tuples L and some > 0.

A static requirement is that = 1.

q = L((A – I) r + u)

Since ki,j,k 1 the row sums of A 1. Hence, AT is a substochastic matrix.

A(i,j),(j,k) 0 A(j,k),(i,j) = 0

r =L( (F1I – F3AT + F5A)q + (F2I – F4AT + F6A)(A – I)r )

no-loop condition

A is time varying LPV

L accounts for state saturation

Matrix Representation

a convex set

Define Fa := F1 = F3 and Fb := F2 = F4 Set F5 = 0

The system is asymptotically stable if

-Fb2 + F6 (n – n1/2) < 0

Fa > 0, Fb > 0, F6 0

However, the system might not be exponentially stable! (delay)

Asymptotic Stability

Set R = A – I

r = L(Fa (I – AT)q + Fb (I – AT)(A – I)r) ·q = L((A – I) r + u)·

Note: R varies with time

=-FbRTR-FaRT

R0

r

q·

· r

qLFor F6=0

Is x = (AT – I)x stable?

LPV/LMI approach

find P > 0 such that P(A – I)T + (A – I)P < 0 for all A

Since AT substochastic and loop-free condition

0txdt

d

Exponential Stability

0 0.2 0.4 0.6 0.8 10

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 10

0.2

0.4

0.6

0.8

1

LPV/LMI does not work2-D illustration

lines of constant cost

vector field for (A1- I)T vector field for (A2 – I)T

1- 0

1- 0A1 =

0 1-

0 1-A2 = T T

. wherethen xIAxtxstVdtd Ts

s

. then , If .111

such that Let pp

pp txptx

dtd

xIAxsp

p

0 0.2 0.4 0.6 0.8 1 1.20

0.5

1 s = 4

Lemma: Choose s > 1 such that

,0 1

11

1

11

1

1

s

ss

sns

s

s

s

Define the cost

n

i

si

ss kxtxxV

1

Exponential Stability

Choose Fi 0 such that there exists P11 P12 P22 with

nFFnpPFnnFnFpP

FnFPnpP

GnF

GPFpP

512246422

251

222

2

12

22412222 44

1

1531: where

FFnFG

,2 q

rF ,

11222

prp

qPP,31 FnF ,612322 FPFP

,02212

121

PP

PP,02 21211 FPP,0 , , 122211 PPP

size queue thebe and speedlinik maximum thebe Let qr

then the closed loop system is exponentially stable.

1

in1 in2 in3 in4

2

4 3

6

7

in5

5

in7

in8

in9

in6

A simple example of hop-by-hop control

-10 0 10 20 30 40 50 60 70 80 900

0.5

1

1.5

2

2.5x 10

4 Queue Occupancy 1 -> 2 (HBH)

Time

Qu

eu

e O

cc

up

an

cy

(b

its

)

At time 0 node in4 begins to send data to node 7.

-10 0 10 20 30 40 50 60 70 80 900

0.5

1

1.5

2

2.5

3

3.5

4

4.5x 10

5 HBH Sending Rate for Inputs

Time

Ra

te

In 1 -> 1In 4 -> 1

1

in1 in2 in3 in4

2

4 3

6

7

in5

5

in7

in8

in9

in6

At time 0 node in4 begins to send data to node 7.

-10 0 10 20 30 40 50 60 70 80 900

1

2

3

4

5

6

7

8

9x 10

5 TCP Sending Rate

Time

Ra

te

In 1 -> 1In 4 -> 1

-10 0 10 20 30 40 50 60 70 80 903

4

5

6

7

8

9

10

11

12

13x 10

4 Queue Occupancy 1 -> 2 (TCP)

Time

Qu

eu

e O

cc

up

an

cy

(b

its

)

A simple example of TCP control

Hop-by-Hop and TCP

0 0.5 1 1.5 2 2.5 3 3.5

x 107

0

10

20

30

40

50

60

70

80

90Transaction Duration

Size of Transaction (bits)

Tim

e t

o C

om

ple

te T

ran

sa

cti

on

TCPHBH

0 0.5 1 1.5 2 2.5 3 3.5

x 107

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7Average Delay

Size of Transaction (bits)

Av

era

ge

De

lay

TCPHBH

0 0.5 1 1.5 2 2.5 3 3.5

x 107

0

5

10

15x 10

5 drops

Size of Transaction (bits)

Nu

mb

er

of

Dro

ps

Du

rin

g T

ran

sa

cti

on

TCPHBH

Blocking

1 2

3

4

5 6

fast links

slow link

congestednode

i,j,k,l the ratio of data

that is traveling from i

to j and to k which

then travels to l.

1 2

3

5

6 7

slow link

congestednode

4

i,j,k,l,m

i,j,...,z

connection oriented

per VC control

Conclusions

• Hop-by-Hop congestion control is a classic control problem.• Asymptotic stability is easy.• However, exponential stability is difficult to prove and may require

high gain.• So far the result are not scalable (they depend on n, the number of

nodes, but could be changed to fan-in).• Future work

• Topology depend strategies should be investigated.• Performance issues

• Min delay/min queue size (minimize L norm of queue).• Minimize transaction time (high bandwidth).• End to End flow control and Fairness.