CoDelpresent by Van Jacobson to the
IETF-84 Transport Area Open Meeting30 July 2012
Vancouver, Canada
���2
���3
Sender Receiver
���4
Sender Receiver
���5
Sender Receiver
• Queue forms at a bottleneck
• There’s probably just one bottleneck(each flow sees exactly one)
➡ Choices: can move the queue (by making a new bottleneck) or control it.
���5
Time
Que
ue le
ngth
Good Queue / Bad Queue
���6
Time
Que
ue le
ngth
Time
Que
ue le
ngth
Good Queue / Bad Queue
���6
Time
Que
ue le
ngth
Time
Que
ue le
ngth
Good Queue / Bad Queue
• Good queue goes away in an RTT, bad queue hangs around.
➡ queue length min() over a sliding window measures bad queue ...
➡ ... as long as window is at least an RTT wide.
���7
Time
Que
ue le
ngth
Time
Que
ue le
ngth
Good Queue / Bad Queue
• Good queue goes away in an RTT, bad queue hangs around.
➡ tracking min() in a sliding window gives bad queue ...
➡ ... as long as window is at least an RTT wide.
���8
Time
Que
ue le
ngth
Time
Que
ue le
ngth
Good Queue / Bad Queue
• Good queue goes away in an RTT, bad queue hangs around.
➡ tracking min() in a sliding window gives bad queue ...
➡ ... as long as window is at least an RTT wide.
���8
How big is the queue?
• Can measure size in bytes – interesting if worried about overflow – requires output bandwidth to compute delay
• Can measure packet’s sojourn time – direct measure of delay – easy (no enque/deque coupling so works with any packet pipeline).
���9
Sojourn Time
• Works with time-varying output bandwidth (e.g., wireless and shared links)
• Better behaved than queue length – no high frequency phase noise
• Includes everything that affects packet so works for multi-queue links
���10
24.5 25.0 25.5 26.0
01
23
45
6
Time (sec.)
Q s
ize (m
s.)
24.5 25.0 25.5 26.0
01
23
45
6
Time (sec.)
Q d
elay
(ms.
)Two views of a Queue
Top graph is sojourn time, bottom is queue size.
(one ftp + web traffic; 10Mbps bottleneck;
80ms RTT; TCP Reno)���11
24.5 25.0 25.5 26.0
01
23
45
6
Time (sec.)
Q s
ize (m
s.)
24.5 25.0 25.5 26.0
01
23
45
6
Time (sec.)
Q d
elay
(ms.
)Two views of a Queue
Top graph is sojourn time, bottom is queue size.
(one ftp + web traffic; 10Mbps bottleneck;
80ms RTT; TCP Reno)���11
Two views of a Queue
Top graph is sojourn time, bottom is queue size.
(one ftp + web traffic; 10Mbps bottleneck;
80ms RTT; TCP Reno)25.00 25.05 25.10 25.15 25.20
0.0
0.5
1.0
1.5
2.0
2.5
Time (sec.)
Q s
ize (m
s.)
25.00 25.05 25.10 25.15 25.20
0.0
0.5
1.0
1.5
2.0
Time (sec.)
Q d
elay
(ms.
)
���12
Multi-Queue behavior
���13
a)Measure what you’ve got
b)Decide what you want
c)If (a) isn’t (b), move it toward (b)
Controlling Queue
���14
a)Measure what you’ve got
b)Decide what you want
c)If (a) isn’t (b), move it toward (b)
Controlling Queue
- Estimator
- Setpoint
- Control loop
���15
How much ‘bad’ queue do we want?
• Can’t let the link go idle (need one or two MTU of backlog)
• More than this will give higher utilization at low degree of multiplexing (1-3 bulk xfers) at the cost of higher delay
• Can the trade-off be quantified?
���16
0 20 40 60 80 100
7580
8590
9510
0
Utilization vs. Target for a single Reno TCP
Target (% of RTT)
Bottl
enec
k Li
nk U
tiliz
atio
n (%
of m
ax)
���17
0 20 40 60 80 100
0.70
0.75
0.80
0.85
0.90
0.95
1.00
Power vs. Target for a Reno TCP
Target (as % of RTT)
Aver
age
Powe
r (Xp
ut/D
elay
)
���18
0 20 40 60 80 100
0.70
0.75
0.80
0.85
0.90
0.95
1.00
Power vs. Target for a Reno TCP
Target (as % of RTT)
Aver
age
Powe
r (Xp
ut/D
elay
)
���18
0 5 10 15 20 25 30
0.93
0.94
0.95
0.96
0.97
0.98
0.99
1.00
Power vs. Target for a Reno TCP
Target (as % of RTT)
Aver
age
Powe
r (Xp
ut/D
elay
)
���19
0 5 10 15 20 25 30
0.93
0.94
0.95
0.96
0.97
0.98
0.99
1.00
Power vs. Target for a Reno TCP
Target (as % of RTT)
Aver
age
Powe
r (Xp
ut/D
elay
)
���19
���20
• Setpoint target of 5% of nominal RTT (5ms for 100ms RTT) yields substantial utilization improvement for small added delay.
���20
• Setpoint target of 5% of nominal RTT (5ms for 100ms RTT) yields substantial utilization improvement for small added delay.
• Result holds independent of bandwidth and congestion control algorithm (tested with Reno, Cubic & Westwood).
���20
• Setpoint target of 5% of nominal RTT (5ms for 100ms RTT) yields substantial utilization improvement for small added delay.
• Result holds independent of bandwidth and congestion control algorithm (tested with Reno, Cubic & Westwood).
➡ CoDel has no free parameters: running-min window width determined by worst-case expected RTT and target is a fixed fraction of same RTT.
���20
Algorithm & Control Law
(see I-D, CACM paper and Linux kernels >= 3.5)
���21
• provides isolation - protects low rate CBR and web for a better user experience. Makes IW10 concerns a non-issue.
• gets rid of bottleneck bi-directional traffic problems (‘ack-compression’ burstiness)
• improves flow mixing for better network performance (reduce HoL blocking)
Eric Dumazet has combined CoDel with a simple SFQ (256-1024 buckets with RR service discipline). Cost in
state & cycles is small and improvement is big.
➡ Since we’re adding code, add fqcodel, not codel.���22
• thanks to Jim Gettys and the ACM, have dead-tree publication to protect ideas
• un-encumbered code (BSD/GPL dual-license) available for ns2, ns3 & linux
• in both simulation and real deployment, CoDel does no harm – it either does nothing or reduces delay without affecting xput.
Where are we?
���23
What needs to be done• Still looking at parts of the algorithm but
changes likely to be 2nd order.
• Would like to see CoDel on both ends of every home/small-office access link but:
- We need to know more about how traffic behaves on particular bottlenecks (wi-fi, 3G cellular, cable modem)
- There are system issues with deployment
���24
Deployment Issues
RTR/AP Cable Modem HeadendHome
Gateway
���25
Deployment Issues
RTR/AP Cable Modem HeadendHome
Gateway
Protocol stack
DeviceDriver DeviceLinux
kernel
���26
Deployment Issues
RTR/AP Cable Modem HeadendHome
Gateway
Protocol stack
DeviceDriver DeviceLinux
kernel
PhoneCPU
3G Modem RAN SGSN?Cellphone
���27
Our thanks to:
• Jim Gettys
• CoDel early experimenters, particularly Dave Taht
• Eric Dumazet
• ACM Queue
• Eben Moglen
���28
Top Related