Abstract of thesis entitled “Congestion Control for ...h0683065/mphil-thesis.pdf · Congestion...
Transcript of Abstract of thesis entitled “Congestion Control for ...h0683065/mphil-thesis.pdf · Congestion...
Abstract of thesis entitled
“Congestion Controlfor Transmission Control Protocol (TCP)
in Wireless Networks”
Submitted byChengdi LAI
for the degree of Master of Philosophy
at The University of Hong Kongin August 2011
Congestion control regulates the amount of data traffic injected by end
systems into communication networks, preventing persistent network over-
load. In the Internet, it is typically realized by Transmission Control Protocol
(TCP), which backs off transmission of end systems upon detecting packet
loss, with the assistance of active queue management (AQM) algorithms,
which probabilistically drop packets at the intermediate routers based on the
buffer occupancy.
The thesis adapts the present Internet congestion control to the wire-
less environment that is gradually becoming an indispensable component of
the Internet. Firstly, we propose a novel TCP variant, known as TCP for
non-congestive loss (TCP-NCL), to perform effective congestion control, se-
quencing control, and loss recovery over wireless networks where reordering,
error-prone channels are prevalent. Conventional TCP tends to misinterpret
packet reordering and wireless loss as congestive loss, and back off unnecessar-
ily. We develop a smart TCP sender (STS) model that offers more reliable
signals of packet loss and network congestion over reordering, error-prone
channels. TCP-NCL is devised as a practical approximation of STS. Our
simulation results show that TCP-NCL can accurately differentiate among
the occurrences of packet reordering, congestive loss, and non-congestive loss,
and can thus serve as the aforementioned unified solution. Deployment of
TCP-NCL requires sender-side modification only.
Secondly, we have performed a series of simulation experiments to inves-
tigate the performance of AQM under wireless losses. Our results report that
random early detection (RED), a classical AQM, fails to maintain a stable
backlog against time-varying wireless packet error rate (WPER), resulting
in poor delay performance and unfairness towards bursty traffic. We apply
the internal model principle to devise a family of AQM enhancements, which
can automatically adjust packet dropping rate of AQM to compensate for
wireless losses, and thus stabilize the backlog. We further devise the integral
controller (IC) as an embodiment of the principle. Our simulation stud-
ies show that IC is robust against time-varying WPER, even in challenging
scenarios like fat pipes and perturbing traffic flows.
In summary, our proposed TCP and AQM enhancements can perform
effective congestion control over the heterogeneous wired/wireless Internet,
and help the latter to operate in the optimal region of low delay with small
jitters and high throughput.
(372 words)
Congestion Control
for Transmission Control Protocol (TCP)
in Wireless Networks
by
Chengdi LAI
B.Eng. (EComE) H.K.
A Thesis Submitted in Partial Fulfilment of the Requirements
for the Degree of Master of Philosophy
at The University of Hong Kong
August 2011
Declaration
I declare that the thesis and the research work thereof represents my own
work, except where due acknowledgement is made, and that it has not been
previously included in a thesis, dissertation or report submitted to this Uni-
versity or to any other institution for a degree, diploma or other qualifica-
tions.
Signed .....................................................................
Chengdi LAI
i
Acknowledgements
The thesis marks a first milestone for my research journey, which started
three years ago when my current advisors, Prof. Victor On-Kwok Li and
Dr. Ka-Cheong Leung, introduced to me the prosperous realm of networking
research. Over the years, their pursuit of excellence, encouragement of inno-
vation, and emphasis on rigorousness have greatly influenced me. Prof. Li’s
great vision and innovative insights have kept me inspired in pursuit of re-
search work of profound impact. Dr. Leung has patiently led me to formulate
my research methodology and taste via the numerous email exchanges and
conversations. I would like to express my earnest gratitude to them.
I owe my special thanks to Prof. Steven H. Low and Prof. Ricky Y.K. Kwok
for serving as my thesis examiners. Their constructive comments assist me
to improve the quality of this thesis. I thank Dr. K.P. Chan for chairing my
thesis committee.
I have been very fortunate to encounter several wonderful mentors during
the past five years in the University of Hong Kong, spanning my B.Eng. and
M.Phil. programs. I appreciate Prof. Kwok, Prof. Lawrence K. Yeung, and
Dr. Vincent Tam for their insightful advice on career and life. I would also
like to take the opportunity to thank Fr. John Coghlan for providing me three
ii
years’ enjoyable residence in Ricci Hall during my undergraduate study.
The high level of professionalism of the support staff in the Department
of Electrical and Electronic Engineering, the University of Hong Kong, has
greatly facilitated my research work. In particular, I would like to thank
Ms. Danita Lee, Ms. Julie Hung, and Ms. Lily Lo for their assistance on ad-
ministrative issues, and Mr. C.L. Chan and Mr. Andy Fok for their technical
support.
I am obligated to my colleagues, Dr. Albert Y.S. Lam, Dr. Guanghua Yang,
Dr. Yanhui Geng, Mr. Ze Zhao, Mr. Yile Yang, Mr. Jin Xu, Mr. Xiaoguan Fang,
Dr. Qiong Sun, Dr. Jun Hong, Dr. Kuang Xu, and Dr. Jialing Xu, for their
support and assistance.
Last, but not least, I am deeply indebted to my parents and my girlfriend,
Miss Shuzhen Shen, for their love, and for always being there for me through
thick and thin.
iii
Contents
Declaration i
Acknowledgements ii
Contents iv
List of Figures vi
List of Tables viii
1 Introduction 11.1 Chapter Overview . . . . . . . . . . . . . . . . . . . . . . . . . 11.2 Internet Congestion Control . . . . . . . . . . . . . . . . . . . 2
1.2.1 TCP . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.2.2 AQM . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.3 Wireless Channels . . . . . . . . . . . . . . . . . . . . . . . . . 101.4 Thesis Statement and Outline of Thesis . . . . . . . . . . . . . 121.5 Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . . 14
2 Related Work 162.1 Chapter Overview . . . . . . . . . . . . . . . . . . . . . . . . . 162.2 Wireless TCP Enhancements . . . . . . . . . . . . . . . . . . 162.3 Design and Analysis of AQM . . . . . . . . . . . . . . . . . . 192.4 Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . . 21
3 A Serialized-Timer Approach for Enhancing Wireless TCP 233.1 Chapter Overview . . . . . . . . . . . . . . . . . . . . . . . . . 233.2 STS Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
3.2.1 RD Timer . . . . . . . . . . . . . . . . . . . . . . . . . 313.2.2 CD Timer . . . . . . . . . . . . . . . . . . . . . . . . . 32
3.3 TCP-NCL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 433.3.1 NRU Process . . . . . . . . . . . . . . . . . . . . . . . 43
iv
3.3.2 RD and CD Timers . . . . . . . . . . . . . . . . . . . . 463.3.3 RTO Timer . . . . . . . . . . . . . . . . . . . . . . . . 503.3.4 Kernel Implementation . . . . . . . . . . . . . . . . . . 51
3.4 Performance Evaluation . . . . . . . . . . . . . . . . . . . . . 543.4.1 Non-congestive loss . . . . . . . . . . . . . . . . . . . . 563.4.2 Packet Reordering . . . . . . . . . . . . . . . . . . . . 613.4.3 Congestive Loss . . . . . . . . . . . . . . . . . . . . . . 63
3.5 Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . . 68
4 Enhancing AQM to Combat Wireless Losses 694.1 Chapter Overview . . . . . . . . . . . . . . . . . . . . . . . . . 694.2 AQM under Wireless Losses . . . . . . . . . . . . . . . . . . . 724.3 System Model . . . . . . . . . . . . . . . . . . . . . . . . . . . 774.4 Robust AQM based on Internal Model Principle . . . . . . . . 82
4.4.1 IC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 884.5 Performance Evaluation . . . . . . . . . . . . . . . . . . . . . 94
4.5.1 One Wireless Bottleneck Link . . . . . . . . . . . . . . 944.5.2 Two Wireless Bottleneck Links . . . . . . . . . . . . . 103
4.6 Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . . 105
5 Conclusions and Future Research 1075.1 Chapter Overview . . . . . . . . . . . . . . . . . . . . . . . . . 1075.2 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . 1075.3 Future Research . . . . . . . . . . . . . . . . . . . . . . . . . . 111
5.3.1 A Theoretical Framework for Wireless TCP . . . . . . 1115.3.2 A Unified AQM Solution for Wireless Networks . . . . 113
5.4 Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . . 115
Bibliography 116
Acronyms 125
v
List of Figures
1.1 Fast retransmit and fast recovery. . . . . . . . . . . . . . . . . 5
3.1 A flowchart for STS model. . . . . . . . . . . . . . . . . . . . 263.2 Evaluation of τ ∗
CDi. . . . . . . . . . . . . . . . . . . . . . . . . 41
3.3 Software architecture for TCP. . . . . . . . . . . . . . . . . . . 523.4 The network topologies used for performance comparison . . . 533.5 Goodput performance against packet error rate over infrastructure-
based wireless network. . . . . . . . . . . . . . . . . . . . . . . 553.6 Goodput performance against delay of the wireless link over
infrastructure-based wireless network. . . . . . . . . . . . . . . 573.7 Goodput performance against bandwidth of the wireless link
over infrastructure-based wireless network. . . . . . . . . . . . 583.8 Goodput performance over multihop ad-hoc wireless network . 623.9 Jain’s fairness index (J) over wired network with a dumbbell
topology. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 643.10 GBR over wired network with a dumbbell topology. . . . . . . 65
4.1 A wireless bottleneck link. . . . . . . . . . . . . . . . . . . . . 724.2 Queue length dynamics under varying wireless losses in the
dumbbell topology with TCP NewReno/RED. . . . . . . . . . 744.3 Queue length dynamics under varying wireless losses in the
dumbbell topology with TCP-NCL/RED. . . . . . . . . . . . 754.4 Control block diagram of the linearized congestion control sys-
tem. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 784.5 Two wireless bottleneck links. . . . . . . . . . . . . . . . . . . 924.6 Comparison of queue length dynamics with same configura-
tions as in Figure 4.2. . . . . . . . . . . . . . . . . . . . . . . . 974.7 Comparison of queue length dynamics under different wireless
losses. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 994.8 Comparison of queue length dynamics with different bottle-
neck link parameters. . . . . . . . . . . . . . . . . . . . . . . . 100
vi
4.9 Comparison of queue length dynamics with the presence ofother traffic. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
4.10 Comparison of queue length dynamics in the topology of twowireless bottleneck links. . . . . . . . . . . . . . . . . . . . . . 104
vii
List of Tables
3.1 General notations. . . . . . . . . . . . . . . . . . . . . . . . . 293.2 Notations for events. . . . . . . . . . . . . . . . . . . . . . . . 303.3 Network configurations for the infrastructure-based wireless
network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
4.1 Network configurations for the single wireless bottleneck net-work. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
viii
Chapter 1
Introduction
1.1 Chapter Overview
The thesis adapts the present Internet congestion control to the wireless en-
vironment that is gradually becoming an indispensable component of the
Internet. In this chapter, we bring the background materials on Internet
congestion control and the challenges it encounters in the wireless environ-
ment. On this basis, we present the thesis statement and outline the thesis.
Section 1.2 succinctly describes the present Internet congestion control, as
fulfilled by Transmission Control Protocol (TCP) with the assistance of active
queue management (AQM) algorithms. Section 1.3 discusses the challenges
posed by the wireless environment on the proper functioning of congestion
control. Section 1.4 presents our thesis statement, and outlines the thesis.
Section 1.5 briefly summarizes the chapter.
1
1.2 Internet Congestion Control
The modern age has witnessed the great success of communication networks
in vastly facilitating information exchange and resource sharing among ge-
ographically distributed people. Packet-switched networks are among the
most common communication networks. In a packet-switched network, end
systems are inter-connected via a group of packet switches such as routers
and network hubs. An end system groups its transmitted data into separate
segments, or packets, before delivering it over the network. Upon receiving
a packet from end systems or other packet switches, a packet switch places
the packet in its backlog, a queue of packets waiting for forwarding. It for-
wards the packet to the next switch towards the destination or directly to
the destination when the outgoing link becomes free for transmission.
The Internet is a global system of packet-switched network. Using the
standardized Transmission Control Protocol/Internet Protocol (TCP/IP) pro-
tocol suite, the Internet inter-connects numerous heterogeneous networks and
is the largest network of networks.
Network congestion or network overload occurs when the amount of data
traffic injected into the network by end systems exceeds its transmission
capacity. At packet switches, packets persistently arrive faster than can be
forwarded. The backlog grows in length, elongating the time packets spent in
queueing. This queueing delay adds to the end-to-end delay experienced by
packets. When the backlog further grows, it can exceed the buffer size of the
packet switch, thereby discarding arrived and/or backlogged packets. Under
persistent network overload, the high levels of transmission delays and losses
2
experienced by packets result in poor data delivery services to end systems
and hence users.
Historically, severe network congestion, known as congestion collapse [56],
occurred to the Internet in the 1980s [30]. Due to poorly regulated transmis-
sion of end systems, packets were held in long backlogs at packet switches.
End systems deemed these packets as being lost and retransmitted them,
exacerbating network congestion and wasting network capacity for transmit-
ting duplicate data. Consequently, the network observed up to three orders
of magnitude decrease in the rate of successful data delivery. Moreover, the
end-to-end delay became excessively long.
Congestion control regulates the transmission rate of end systems, pre-
venting persistent network overload. It is therefore crucial to ensuring the
proper operation of the network. In the present Internet, the task of con-
gestion control is implemented by TCP running at end systems with the
assistance of AQM running at routers in a distributed manner. In the fol-
lowing two subsections, we review TCP and AQM, respectively.
1.2.1 TCP
Transmission Control Protocol (TCP) [4, 30, 60] is the de facto standard for
the Internet. Using the connection-less, unreliable data delivery services of
Internet Protocol (IP), it provides end-to-end, in-order, reliable data deliv-
ery services to various applications like File Transfer Protocol (FTP), Telnet,
and Hypertext Transfer Protocol (HTTP). Initially, TCP did not incorpo-
rate end-to-end congestion control [60]. The latter was first installed in TCP
3
around 1988 in an successful attempt to combat the occurrence of conges-
tion collapse, and has since become an essential element of TCP, central to
ensuring the stable, efficient operation of the present Internet.
TCP is connection-oriented. A TCP connection is set up between two
end systems when they agree on a few parameter settings and each reserve
resources like memory space for the transmission via TCP, after which the
transmission begins. The transmission ends when the connection is closed
and the resources reserved by end systems will be released.
The TCP connection is duplex. Both end systems can transmit and re-
ceive data. Nevertheless, we can conceptually focus on one-way transmission
over a TCP connection to avoid unnecessary complications. Moreover, in re-
ality the bulk of data transfer over a TCP connection takes place in one way,
say, from a web server to a web client. Correspondingly, the end systems
that transmits and receives data are referred to as TCP sender and TCP
receiver, respectively.
A TCP sender consecutively assigns byte numbers to its transmitted data
bytes, and a TCP receiver expects the number of received data bytes to be
consecutively ordered. When receiving a data packet, a TCP receiver noti-
fies the sender of successfully delivered data via an acknowledgment (ACK)
packet that contains acknowledgment number, namely the number of next
byte it expects. The time elapsed between when a TCP sender transmits a
data packet and when the packet is acknowledged is known as round-trip time
(RTT). An example is illustrated in Figure 1.1. The right and left arrows
denote data packets and ACK, respectively. The first data packet contains
bytes numbered 201 to 300, as indicated by the two numbers above its cor-
4
Sender Receiver
301:400
201:300
401:500
501:600
601:700
ACK 301
301:400
ACK 301
ACK 301
ACK 301
Figure 1.1: Fast retransmit and fast recovery.
responding arrow. Its arrival at the receiver is acknowledged to the sender
via the first ACK containing acknowledgment number 301. These in-order
data bytes will be delivered to the applications after some extra processing.
The next packet arrived contains bytes numbered 401 to 500 and is thus out
of order. In this case, the receiver immediately sends back an ACK with
acknowledgment number 301. This duplicate acknowledgement contains the
same acknowledgment number as the previous ACK. The out-of-order data
packet will be buffered by the receiver until the arrival of all packets before it,
when the consecutively ordered data will be further processed and delivered
to the application. In-order data delivery is thus fulfilled. We come back to
the example shortly.
A TCP sender applies a window-based control to regulate the amount
of outstanding data, namely the data that has been transmitted but not
yet acknowledged by the receiver. The outstanding data consists of data
5
buffered by the receiver, data packets in transit, and data packets whose cor-
responding ACKs are in transit. Its amount shall not exceed the minimum of
receiver’s advertised window (awnd) and sender’s congestion window (cwnd).
awnd is set by the receiver and fed back to the sender via ACK for indicating
the available spaces in receiver buffer. It prevents a fast sender from over-
whelming a slow receiver. cwnd is determined by the sender, representing
the latter’s estimation of the load affordable by the network.
The acknowledgment scheme and window-based transmission provides a
basis for realizing the important features of end-to-end congestion control
and reliable transmission. In fulfilling the former feature, cwnd has to be
dynamically adjusted throughout the TCP session in accordance with the
ever-changing available network capacity. Signals of network underload and
overload are needed for triggering the increase and decrease of cwnd, respec-
tively. In fulfilling the latter feature, signals of packet loss are needed for
triggering retransmission of lost data packets to ensure the eventual delivery
of every data byte.
A TCP sender uses successful, prompt data delivery as a signal that the
network can potentially carry more load than currently offered. The slow
start and congestion avoidance algorithms are devised on this premise, and
govern the increment of cwnd when it is smaller than and greater than slow
start threshold value (ssthresh), respectively. Slow start increases cwnd by
one data packet size upon receiving an non-duplicate ACK that acknowl-
edges the successfully delivery of new data. Congestion avoidance increases
cwnd by one data packet size per RTT if all data packets transmitted in the
previously RTT are acknowledged.
6
A TCP sender uses the same set of signals for indicating packet loss
and network overload. Two types of signals are used, namely retransmission
timeout (RTO) and triple duplicate ACKs. A retransmission timer is started
when a data packet is first injected into the network, and will timeout if the
packet is still not acknowledged when the timer expires. Upon the occurrence
of an RTO, all the outstanding packets will be retransmitted. At the same
time, the network is deemed severely congested and cwnd will be forced to
reopen from one packet size by employing the slow start algorithm.
At the receiver side, each out-of-order packet will trigger a duplicate
ACK. At the sender side, when the number of duplicate ACKs reaches a
certain threshold value, say, three, the packet expected by the receiver will
be deemed as lost due to congestion. Fast retransmit and fast recovery will
be activated, retransmitting the packet expected by the receiver and halving
cwnd. ssthresh is then set to cwnd so that cwnd will reopen by employing the
congestion avoidance algorithm. Therefore, triple duplicate acknowledgment,
a direct signal of out-of-order packet events, is further used as an indication
of congestive packet loss. In the example illustrated in Figure 1.1, the sec-
ond, third, and fourth data packets arrived at the receiver are all out of order
since they do not contain the byte numbered 301 expected by the receiver,
each triggering a duplicate ACK. The sender activates fact retransmit and
fast recovery upon receiving the third duplicate ACK.
7
1.2.2 AQM
A queue management algorithm manages the backlog at routers, determining
when and how packets should be dropped. Conventionally, a router does
not drop packets until the backlog exceeds the buffer size, forcing discard of
incoming data packets, namely drop tail, or packets in the backlog. The TCP
data flows will keep on increasing cwnd and generate more data packets in
transit until a router buffer reaches its limit. Consequently,
1. The end-to-end delay for data delivery is significantly increased.
2. Insufficient buffer space incurs a much higher packet loss rate for bursty
traffic that tends to fill up the buffer space.
On the other hand, when buffer overflow occurs, the situation is likely
to persist for a while due to the delay of TCP in detecting the congestion.
Packets belonging to different TCP flows will be dropped simultaneously. All
or most TCP flows will be triggered to back off, leaving the buffer largely
unoccupied. This TCP synchronization [80] in turn creates another two
problems:
1. The backlog fluctuates severely between the full buffer size immedi-
ately before a TCP synchronization and the low level immediately af-
terwards. This creates large jitters in the queueing delay and thus
end-to-end delay.
2. The backlog may become empty after a TCP synchronization, leaving
the outgoing link idle.
8
Active queue management (AQM) [11] was first proposed in the 1990s
to tackle the aforementioned problems with conventional queue management
algorithms. AQM probabilistically drops packets before buffer overflow based
on a dropping rate determined from the past and/or present queue lengths.
The backoff of different TCP flows are therefore desynchronized and spread
across time. With the coordination of an effective AQM design, the aggregate
transmission of the TCP flows can keep a router buffer occupied with a small,
stable backlog. This significantly reduces the end-to-end delay, smooths out
delay jitters, leaves sufficient buffer space to absorb bursty traffic, and keeps
the bottleneck link backlogged and thus fully utilized.
Random early detection (RED) [11, 22] is a most classical AQM algo-
rithm. It maintains an average queue size r, say, by computing the running
average of sampled backlog size b with queue weight α:
r ← (1− α)r + αb (1.1)
The packet dropping probability p is zero if r is less than a preset threshold
value minThresh, and one if r is greater than the other preset threshold value
maxThresh. When r is between minThresh and maxThresh, p is determined
as a linear function of r:
p← pMaxr −minThresh
maxThresh−minThresh(1.2)
RED performs probabilistic packet dropping based on p upon each packet
arrival.
9
1.3 Wireless Channels
The next generation network is expected to be a heterogeneous network of
networks, including both wired and wireless components. Wireless networks
may extend beyond access networks to backhaul networks [29], and even
backbone networks. The present Internet congestion control is designed
based on the premise of conventional wired network, where data are mostly
correctly, orderly delivered. It encounters significant challenges in the wire-
less environment due to the latter’s more unpredictable nature. In particular,
non-congestive packet losses and packet reordering, while being rare events
in conventional wired networks, are common in wireless networks [44].
As compared with the wired media, the wireless medium provides much
more lossy physical links for data transmissions. Signals propagating over
wireless channels suffer from degradation, interference, and noise. Packets
received may be damaged beyond the recovery capability of error control
codes, if any. These packets are thus discarded, leading to the occurrence of
non-congestive packet losses.
Packet reordering refers to the disruption of the packet order of a TCP
flow. Despite conventional beliefs that packet reordering is a transient or
pathological network behavior, it is in fact persistently observed over modern
networks and can be caused by a myriad of reasons [44, 45]. Due to the high
transmission error rates and, in some cases, mobility in wireless networks,
packet reordering is further increased substantially when the transmission
medium evolves from physical cables to wireless. Specifically, four causes of
packet reordering are commonly observed over wireless networks, including:
10
Link-layer retransmission (LLRTX): To combat high transmission error
rates of wireless channels, some link layer retransmission mechanisms [7, 28]
have been proposed to locally retransmit damaged packets at the link layer.
As a side effect of local retransmission, the packet order of a flow is altered.
Path change: Over a mobile ad-hoc networks (MANET), a TCP flow
traverses a number of wireless nodes. The transmission path of the flow may
be altered when some nodes move. The RTT of the connection may change
too. Consequently, it is possible that some packets transmitted after the path
change arrive at the receiving end before those packets transmitted prior to
the path change.
Hand-off: In an infrastructure-based network, the complete transmission
coverage of an entire area is achieved cooperatively by a set of base stations.
When a mobile client moves from the radio range of one base station to that
of another, a hand-off between these two base stations occurs, changing the
transmission path for the flow. The resulting variation in RTT may lead to
the occurrence of packet reordering.
Packet-level multi-path routing: In wireless mesh network (WMN) and
many other networks of rich connectivity, multi-path routing have been pro-
posed for load balancing. When it is carried out at the packet level, packets
may be reordered due to the difference in time delays of distinct paths.
The present Internet congestion control therefore encounters two-fold
challenges in the wireless environment. First, TCP sender triggers fast re-
transmit and fast recovery upon triple duplicate ACKs. While the latter is
dominantly caused by congestive loss over conventional wired networks, it is
often triggered by additionally packet reordering and non-congestive loss in
11
the wireless environment. Thus, TCP tends to falsely retransmit packets and
reduce cwnd from time to time, injecting duplicated bytes into the network
and keeping cwnd unnecessarily small. Consequently, the available network
resources are wasted and under-utilized.
Second, the extra source of packet losses, namely wireless losses, may
interfere with the normal operation of AQM, which communicates implicitly
with TCP via active packet drops. This casts doubt on whether AQM can
continue to fulfill the objective of maintaining a stable, small backlog under
wireless losses.
1.4 Thesis Statement and Outline of Thesis
Our thesis statement is to demonstrate that the present Internet conges-
tion control can be systematically adapted to the wireless environment at a
minimal disruption and deployment cost.
Presently, the Internet congestion control is generally loss-based. Packet
drops are used as a signal of network overload, based on which TCP acti-
vate congestion response and AQM probabilistically trigger the congestion
responses of different TCP connections. We devise our enhancements within
the scope of loss-based TCP and AQM. This brings the inherent advantage
of minimizing both the deployment cost and the potential disruptions to the
existing congestion control deployment.
On the TCP side, we have developed a smart TCP sender (STS) model,
that uses a serialized-timer structure to offer more reliable signals of packet
loss and network congestion over general (reordering), error-prone channels.
12
Timers are serialized in the sense that the second timer is started upon the
expiration of the first timer. We have devised a novel TCP variant, known
as TCP for non-congestive loss (TCP-NCL), as a practical approximation
of the STS. TCP-NCL can accurately differentiate among the occurrences
of packet reordering, congestive loss, and non-congestive loss, and can thus
serve as a unified solution for effective congestion control, sequencing control,
and loss recovery over wireless networks. Deployment of TCP-NCL requires
sender-side modification only.
On the AQM side, we have investigated the performance of AQM under
wireless losses using simulation experiments. Our results report that RED
fail to maintain a stable backlog against time-varying wireless packet error
rate (WPER). We have applied the internal model principle to devise a family
of AQM enhancements, which can adjust packet dropping rate to compen-
sate for wireless losses without requiring exact knowledge of WPER, and
thus stabilize the backlog. We further devise integral controller (IC) as an
embodiment of the principle, which performs robustly against time-varying
WPER in various challenging scenarios like fat pipes and perturbing traffic
flows.
In summary, our proposed TCP and AQM enhancements can perform
effective congestion control over the heterogeneous wired/wireless Internet,
and help the latter to operate in the optimal region of low delay with small
jitters and high throughput.
The rest of this thesis is organized as follows.
Chapter 2 presents a literature review of existing work on wireless TCP
enhancements and on the design and analysis of AQM. It puts our work in
13
perspective.
Chapter 3 develops the STS model. TCP-NCL is then devised as a prac-
tical approximation of the STS model. Extensive simulation experiments are
performed to examine the performance of TCP-NCL against packet reorder-
ing, non-congestive packet loss, and congestive loss.
Chapter 4 performs simulation-based investigation of AQM under wire-
less losses. A linear model of TCP/AQM under wireless losses is formulated.
On this basis, the internal model principle is applied to devise a family of
AQM enhancements that stabilize the backlog. IC is introduced as an em-
bodiment of the principle, with a set of design rules developed. Extensive
simulations are conducted to evaluate IC against time-varying wireless losses
under various network scenarios, like fat pipes and perturbing traffic flows.
Chapter 5 concludes the thesis. It summarizes our major contributions,
and presents an overview of future research directions.
The work presented in this thesis has in parts been published in [39, 38,
36, 37, 47].
1.5 Concluding Remarks
In this chapter, we bring the background materials on Internet congestion
control and the challenges it encounters in the wireless environment. Con-
gestion control is crucial to ensuring the stable, efficient operation of packet-
switched networks like the Internet. The present Internet congestion control,
as fulfilled by TCP and AQM, performs well for conventional wired networks.
However, it encounters significant challenges in wireless networks, in large
14
part due to the frequent occurrences of packet reordering and non-congestive
loss.
We suggest loss-based TCP and AQM enhancements to adapt the Internet
congestion control to the wireless environment at a minimal deployment cost
and disruption.
15
Chapter 2
Related Work
2.1 Chapter Overview
In this chapter, we conduct a literature survey of related work performed
to date. Section 2.2 studies existing wireless TCP enhancements that adapt
TCP to the general (reordering), error-prone channels. Section 2.3 studies
existing work on the design and analysis of AQM. Section 2.4 summarizes
limitations of these and puts our work in perspective. The work presented
in this chapter has in parts been published in [39, 37].
2.2 Wireless TCP Enhancements
In this section, we summarize existing work for adapting TCP to perform
congestion control and loss recovery over general error-prone transmission
channels in wired/wireless networks. We confine ourselves to the scope of
loss-based TCP variants that use congestive loss as signals of network con-
16
gestion. Some TCP variants propose inferring network loading condition
based on explicit network feedbacks [32, 75] or RTT [12, 71] for adapting
TCP to high-speed networks. Yet, the former requires universal support by
all the traversed intermediate routers, which can hardly be attained due to
the heterogeneity of the Internet. The latter, known as delay-based TCP
variants, may not work well over reordering networks, where RTT can be
varied by factors other than network loading. Besides, they may not share
the bandwidth fairly with loss-based TCP in widest current usage [55, 67].
AVG, DEL, EWMA, and INC in [9], RR-TCP [81], TCP-DCR [8], and
TCP-PR [10] propose to abandon the fixed triple duplicate ACKs as sig-
nals of congestive packet loss. Instead, they proactively postpone a packet
retransmission until a corresponding timer expires or when the number of du-
plicate ACKs received reaches an adaptively evolved threshold value. This
is considered a more reliable signal of packet loss over reordered channels.
However, these variants use the same set of signals for activating congestion
response, and are incapable of differentiating between congestive and non-
congestive losses. They thus have to rely on LLRTX for making the on-going
random packet losses over wireless links transparent to the transport layer.
DSACK TCP [23], TCP-DOOR [69], and TCP-Eifel [52] are designed
with the premise that triple duplicate ACKs may be unreliable indications of
congestive loss. Their approaches differ, however, in that they try to detect
spurious retransmission after activating fast retransmission and fast recovery
upon the arrival of triple duplicate ACKs1. Upon successful detection, cwnd
1The same design philosophy is also employed by F-RTO [62], which focuses on detect-ing spurious retransmission triggered by RTO. However, false RTO is not the major reasonfor performance deterioration over wireless networks. We thus do not include F-RTO as
17
will be recovered immediately or via the slow start process. TCP-DOOR
also disables congestion control for a time interval as it assumes out-of-order
events are often caused by path change. However, these variants are generally
unable to recover cwnd unnecessarily reduced in the event of non-congestive
packet losses. Moreover, as shown by our simulation results, TCP-DOOR
tends to excessively disable congestion control actions under persistent packet
reordering, resulting in substantial deterioration in performance.
JTCP [74], TCP-Veno [24], and TCP-Westwood (TCPW) [13] focus on
differentiating congestive and non-congestive losses by using the estimated
network load upon the arrival of triple duplicate ACKs, which is still treated
by them as a credible signal of packet loss for triggering retransmission.
One of their major merits is that the binary signal of packet loss no longer
dictates the activation of congestion response. Instead, the signal is combined
with the estimated network load to decide by how much cwnd should be
reduced. Yet, the inherent assumption of nearly in-order channel obviously
limit the applicability of these variants over wireless networks and many
other modern networks, where packet reordering are common. Particularly,
persistent reordering may significantly affect the adaptation of cwnd to the
available network bandwidth by frequently activating minor reductions of
cwnd and triggering false retransmissions.
TCP-Probing [35] applies a different approach for differentiating conges-
tive and non-congestive losses upon inferring packet loss from the arrival
of triple duplicate ACKs. After cwnd is halved in response to the inferred
packet loss, probing data packets are injected into the network. The inferred
a wireless TCP enhancement.
18
packet loss is categorized as non-congestive if the RTT of the first two ac-
knowledged probing packets are smaller than the best RTT, or the minimum
of the measured RTTs during the TCP session. Again, TCP-Probing will
trigger frequent false fast retransmissions due to persistent packet reorder-
ing. Furthermore, the stored best RTT may fail to capture network changes.
Specifically, when a path change or handoff occurs within a TCP session and
increases the minimum attainable RTT, the stored best RTT will not be up-
dated. Consequently, it may become impossible for the probe packets to be
acknowledged within the best RTT even under light network loading.
The existing unified solutions require information and/or modifications of
the network protocol stack beyond the transport layer. ATCP [48] introduces
an ATCP layer between TCP and IP. The new layer switches TCP among
various pre-defined states in accordance with the network condition, trying
to avoid spurious packet retransmission and congestion response.
In summary, there is a lack of a unified solution at the transport layer with
reliable signals of network overload and packet loss for performing congestion
control and loss recovery over general, error-prone transmission channels.
Such unified solution is highly desirable, as it adapts TCP to a wide range
of wireless and, in some cases, wired transmission channels at a minimal
deployment cost.
2.3 Design and Analysis of AQM
In this section, we summarize the related work on the design and analy-
sis of AQM, with special emphasis on those applying the control-theoretic
19
approaches and/or adapting AQM to wireless networks.
First proposed in 1993, RED [22] demonstrates the inherent advantage
of AQM in helping the network to operate in the optimal region of high
throughput and low delay. However, parameter tunings and new variants of
RED invariably adopted the trial-and-error approach due to the difficulties
in understanding the dynamics of TCP/AQM.
The fluid models of TCP/AQM, as presented in, say, [33, 50, 53], provide
a basis for systematic design and analysis of AQM. The optimization-based
approach [33, 49] interprets TCP/AQM as a distributed algorithm for solv-
ing a utility maximization problem, subject to capacity constraints. The
major focus is on the optimal solutions attained at the equilibria. However,
transient responses of AQM are mostly not captured.
The control-theoretic approach views Internet congestion control as a non-
linear control system. The system is first linearized around its equilibrium
so as to analyze the dynamics of TCP/AQM around the equilibrium. This
enables parameter tuning of RED and design of new AQM algorithms based
on frequency-domain analysis [26, 27, 51, 79], which is powerful in improving
the transient response of AQM and guaranteeing the stability of the linear
system. Interested readers can refer to [65] for a survey. The work in this
area closest to our study is [26], which designs a proportional-integral (PI)
controller for the general network topology. Generally, the IC in our work
can be considered a special case of the PI controller with the proportional
part being set to zero. However, the PI controller considered in [26] assumes
the proportional part to be non-zero, so that it cannot be reduced to the IC.
Moreover, we apply the IC to combat wireless losses, whereas [26] focuses on
20
designing a PI controller in wired networks.
The global stability and region of attraction of Internet congestion control
have been studied in the context of a non-linear system. Please refer to [20,
72, 78] and the references therein. In a nutshell, these are concerned with
whether the system will converge to the equilibrium starting from a feasible
initial status not necessarily close to the equilibrium [34]. Among them,
Fan et al. [20] examine the robustness of the network flow control against
disturbances. Yet, the AQM in [20] is static rate-based, which determines the
packet dropping rate as a function of the instantaneous incoming data rate.
Our work considers buffer-based AQM variants, which are widely used in the
Internet to determine the packet dropping rate from the queue length.
In adapting AQM to wireless networks, the inter-flow fairness in ad-hoc
networks can be improved [76] by virtually aggregating packets queued in
those interfering nodes to one queue, and applying AQM to manage the
queue.
2.4 Concluding Remarks
In this chapter, we have summarized existing work on wireless TCP enhance-
ments and on the design and analysis of AQM. We find that there is a lack of
transport layer unified solution for both packet reordering and non-congestive
loss. Such solution is highly desirable for adapting TCP to the reordering,
error-prone communication channel prevalent in wireless environment at a
minimal deployment cost.
The performance of AQM in wireless environment remains to be thor-
21
oughly investigated. In particular, it is largely unexplored in the literature
whether AQM can maintain a stable, small backlog under wireless losses. If
not, an AQM enhancement for stabilizing the backlog would be highly de-
sirable so as to improve the delay performance and fairness towards bursty
traffic in wireless networks..
22
Chapter 3
A Serialized-Timer Approach
for Enhancing Wireless TCP
3.1 Chapter Overview
In this chapter, we develop a unified solution for performing effective con-
gestion control, sequencing control, and loss recovery over communication
networks with general (reordering), error-prone transmission channels like
wireless networks. The proposed modifications involved are limited to sender-
side TCP only, thereby facilitating possible future wide deployment. As we
find in the literature review in Chapter 2, most existing TCP variants only
focus on tackling with the presence of either non-congestive packet loss or
packet reordering. A few unified solutions for both problems generally re-
quire information and modifications beyond the scope of the transport layer,
hindering possible future wide deployment.
We develop an ideal TCP sender model, known as smart TCP sender
23
(STS), for constructing more reliable signals of packet loss and network con-
gestion over a general error-prone channel for activating packet retransmis-
sion and congestion response, respectively. The two signals of packet loss
and network congestion are the expirations of two serialized timers. The
first timer is started when a packet is first injected into the network, while
the second timer is started upon the expiration of the first timer. The timer
expiration periods have been optimized for minimizing quantified risks asso-
ciated with spurious congestion response and excessively delayed congestion
response.
We further devise a new TCP variant, known as TCP for non-congestive
loss (TCP-NCL), as a practical approximation of STS. TCP-NCL realizes
the serialized-timer structure economically by using timestamps. Deploy-
ment of TCP-NCL requires maintenance of a few unused bits in the existing
data structure for TCP and a few extra variables only. The maintenance is
performed upon arrival of acknowledgement packet.
We have performed extensive simulation experiments to examine the per-
formance of TCP-NCL and compare it with several well-known TCP variants
in the literature using Network Simulator (ns) Version 2.29 [19]. Our results
report that TCP-NCL attains significant performance improvement over gen-
eral, error-prone communication networks, thereby demonstrating robustness
against packet reordering and non-congestive packet loss. At the same time,
multiple competing TCP connections, with some employing TCP-NCL and
some employing SACK TCP, the popular standardized TCP variant, are
capable of sharing bandwidth fairly over conventional wired networks. TCP-
NCL can thus serve as the aforementioned unified solution for performing
24
effective congestion control, sequencing control, and loss recovery over com-
munication networks with general, error-prone transmission channels.
Section 3.2 develops the STS model. Section 3.3 presents TCP-NCL
based on the analytical results of the STS model. Section 3.4 examines the
performance of TCP-NCL and compares it with some popular TCP variants.
Section 3.5 concludes and discusses some possible extensions of this work.
The work presented in this chapter has in parts been published in [39, 38, 36].
A U.S. patent has been filed for the work [47].
3.2 STS Model
In constructing an aforementioned unified solution, we start from the follow-
ing two initial observations:
Nature of signals: Over general communication channels, there are two
prevalent approaches for inferring the occurrence of packet loss based on: 1)
the number of consecutively received duplicate ACKs reaching an adaptively
evolved threshold value or, 2) a timer1 expiration. However, one major con-
cern with the former approach is that a TCP sender may not be able to
accumulate enough duplicate ACKs to perform loss recovery because of the
insufficient number of the outstanding packets. Special measures, like limited
retransmit [2], would have to be incorporated in order to alleviate the prob-
lem by sending extra data packets upon the arrival of duplicate ACKs. By
contrast, a more elegant and robust approach for estimating the probability
1The timer is different from the retransmission timer, which triggers a packet retrans-mission and activates the slow start algorithm when it expires. TCP-PR is an exceptionsince it uses a modified retransmission timer so that fast retransmit and fast recovery areactivated upon the timer expiration.
25
Update
Expires?i CD
PiSend Packet
Pi &Resend Packet CD
Expires?
Yes
Yes
No
No
No
Yes
No
Yes
Activate Congestion Control
iRD
(Added by TCP−NCL)
& Start Timer iRD
Start Timer i
ACKi Received?
ACKi Received?
RTT
Figure 3.1: A flowchart for STS model.
26
of packet loss is to measure the time elapsed since a packet transmission (or
some other similar instances) and refer to the past recent history of RTT.
Separation of signals: Over error-prone channels, signals of packet loss
alone can only serve as prior signals of network congestion. It is necessary
to take additional information into account to confirm whether the inferred
loss can be categorized as a congestive loss.
In place of the conventional approach of activating fast retransmission and
fast recovery simultaneously, we propose to delay the decision on whether to
activate a congestion response behind that of a packet retransmission for a
short time period. The motivation behind doing so is two-fold. First, infor-
mation inferred by the occurrences of the events which happen after a packet
retransmission can be incorporated to decide whether a congestive loss has
occurred. Such information is a real-time reflection of the network loading
condition so that this helps to make an appropriate decision. Specifically, if
an ACK for a packet lags behind a retransmission of that packet by a short
duration, it should be treated as a signal of no network congestion. The ACK
packet can either be for the originally transmitted packet or the retransmit-
ted packet. In the former case, false retransmission, which can hardly be
fully eliminated in reality, is detected, thereby rejecting the necessity of a
congestion response. In the latter case, a fairly short round-trip time and
thus a lightly loaded network can be inferred.
Second, it is more “affordable” for TCP to trigger a false fast retransmis-
sion than a false congestion response. In the event of a false fast retransmis-
sion, there is at most one full-sized packet being retransmitted spuriously.
However, there is a significant portion of the available bandwidth being left
27
unused in the event of a falsely activated congestion response. For TCP vari-
ants which activate fast retransmit and fast recovery simultaneously, they
may defer both measures to avoid the penalty of a false congestion response
until a congestive loss can be confirmed highly accurately. However, over
general error-prone reordering channels, such deferment on fast retransmit
may result in an expensive RTO. For example, TCP-DCR postpones fast re-
transmit by one RTT upon receiving the first duplicate ACK,so that a packet
is retransmitted, if needed, about two RTTs after its first transmission. By
advancing packet retransmission before the activation of congestion response,
the risk of triggering an RTO can be more effectively reduced while avoiding
the heavy penalty due to a false congestion response. Nevertheless, while a
comparatively higher level of false retransmission can be tolerated with our
proposal, it should still be maintained at a minimal level to avoid significant
waste of the limited bandwidth.
Based on the above observations, we design our STS model, as illustrated
in Figure 3.1. A new retransmission decision (RD) timer RDi is started
whenever a new packet Pi is injected into the network. If ACKi is received
by the TCP sender before RDi expires, RDi will be cancelled. Otherwise,
Pi will be retransmitted and a congestion response decision (CD) timer CDi
will be started.
CDi will be cancelled if ACKi arrives before it expires. Otherwise, the
congestion control mechanisms will be activated upon the expiration of CDi.
Therefore, the installation of CDi allows the TCP sender extra time after the
packet retransmission to decide whether congestion control shall be activated.
ACKi arriving before the expiration of CDi shall be treated as an indication
28
Table 3.1: General notations.
Notation Meaning
CS Number of packets TCP sender misses to send due toSC
CT Number of packets TCP sender misses to send dueto the extra reduction of TCP sender’s throughputintroduced by RTO
F (t) Cumulative distribution function of RTT for PacketPi
G Goodput of TCP sendermaxRi Maximum attainable RTT for Packet Pi
minRi Minimum attainable RTT for Packet Pi
P oi First transmitted Packet Pi
P ri Retransmitted Packet Pi
PC Congestive loss ratePL Loss ratePW Non-congestive loss rateT Throughput of TCP senderW Average cwnd over a TCP session
of no network congestion. Thus, this eliminates the need for activating any
congestion control measures.
The expiration periods of RDi and CDi, denoted as τRDiτCDi
, are eval-
uated in Sections 3.2.1 and 3.2.2, respectively. To facilitate the subsequent
discussion, we define the notations as shown in Tables 3.1 and 3.2.
The following assumptions are made to simplify the discussion:
1. Data packets are of equal size. Accordingly, we express the size of
congestion window and slow start threshold (ssthresh) in terms of the
number of packets.
2. P (PAi(t)|CLoi ) continuously monotonically increases with time t and
29
Table 3.2: Notations for events.
Notation Meaning
CLoi P o
i lost due to congestionCLr
i P ri lost due to congestion
NLoi P o
i experiencing non-congestive lostNLr
i P ri experiencing non-congestive lost
RTXi Fast retransmission of Pi
PAi(t) Either P oi or P r
i acknowledged by time t after RTXi
PAoi (t) P o
i acknowledged by time t after retransmissionPAr
i (t) P ri acknowledged by time t after retransmission
PUi(t) Both P oi and P r
i unacknowledged by time t afterRTXi
PUoi (t) P o
i unacknowledged by time t after retransmissionPU r
i (t) P ri unacknowledged by time t after retransmission
TO Retransmission timeout
limt→∞ P (PAi(t)|CLoi ) > 0. Given that PLo
i is lost due to congestion,
the chance of Packet Pi being acknowledged relies on P ri . By allowing
P ri more time (i.e., increasing t), the chance of Pi being acknowledged
does not drop. Furthermore, the chance should be non-zero, given that
P ri is allowed sufficient time (t→∞),
3. Non-congestive loss is assumed to be random. In other words, we do not
consider correlated or burst loss for simplicity. Nevertheless, simulation
results will show that our proposed TCP variant still offers competent
performance under burst loss. Thus, NLoi is mutually independent with
any events occurring with P ri and vice versa. In particular, it follows
that:
P (NLoi ∩ PU r
i (t)) = P (NLoi ) · P (PU r
i (t)) (3.1)
30
3.2.1 RD Timer
τRDidetermines how long we have to wait before activating packet retrans-
mission. The assignment of τRDirequires careful balancing between the ob-
jectives of prompt retransmission for preventing RTO and avoiding spurious
packet retransmission.
Ideally, it should be guaranteed that, when P oi is lost due to congestion,
P ri should be promptly transmitted so that the remaining time before RTO
is sufficient for it to be acknowledged. This ensures that cwnd will be halved
instead of being reset to one at the onset of network overload. In other words,
RTO will be incurred if and only if P ri is lost:
P (TO|CLoi ) = P (PLr
i ) = PL (3.2)
On the other hand, an excessively small τRDiwill result in spurious packet
retransmissions. Let F (τRDi) = 1− ζ , where 0 ≤ ζ ≤ 1. A fast retransmit of
packet Pi will be activated upon the expiration of RDi, given Pi is not yet
acknowledged. Thus,
P (RTXi) = P (PUoi (0))
= P (PUoi (0)|PLo
i )P (PLoi )
+P (PUoi (0)|PLo
i )P (PLoi )
= PL + ζ(1− PL) (3.3)
In addition to fast retransmit, packet retransmission can also be activated
by RTO. Yet, with fast retransmit promptly activated, the occurrences of
31
RTO will be maintained at a minimal level. Thus, the difference between
throughput and goodput, without considering the protocol overhead, should
mainly be attributable to fast retransmit. Accordingly, we have:
T ≃ [1 + P (RTXi)]G (3.4)
Hence,
G
T≃ [(1 + PL) + ζ(1− PL)]−1 (3.5)
To improve efficiency, we thus have:
ζ ≪ 1 (3.6)
Since the retransmission timer only serves as a coarse upper bound for
performing loss recovery and congestion control, it is possible for (3.2) and
(3.6) to be simultaneously satisfied with τRDiappropriately set. A closed-
form expression for the optimal value of τRDiis left as part of the future work.
Nevertheless, we assume that (3.2) and (3.6) are satisfied in the following
discussion.
3.2.2 CD Timer
The setting of τCDishould guarantee that, if ACKi arrives before the ex-
piration of CDi, P oi does not experience congestive loss. Analytically, this
32
is:
P (CLoi |PAi(t)) = 0 (3.7)
for all t ∈ [0, τCDi]. In this way, we ensure that the TCP sender will not
misinterpret congestive loss as non-congestive loss.
Denote τu , suptt|P (PAi(t)|CLoi ) = 0, which is well defined by As-
sumption 2. A sufficient and necessary condition for (3.7) is given by the
following lemma.
Lemma 3.1 P (CLoi |PAi(t)) = 0 if and only if t ≤ τu.
Proof Observe that:
P (CLoi |PAi(t))P (PAi(t)) = P (PAi(t)|CLo
i )P (CLoi ) (3.8)
and
P (PAi(t)) ≥ P (PAri (0)) = (1− PL)(1− ζ) > 0 (3.9)
Thus, P (CLoi |PAi(t)) = 0 if and only if P (PAi(t)|CLo
i ) = 0. On the
other hand, by construction, P (PAi(t)|CLoi ) = 0 if and only if t ≤ τu. The
result of the lemma therefore follows.
We do not know an exact expression for τu for the general case at the
moment. In the simple case of a single flow over a bottleneck link, when a
packet is dropped due to buffer overflow, the buffer will be kept full up to
when the lost packet is retransmitted. τu is therefore maxR in this case. Yet,
33
this do not apply the the case of multiple flows since upon the occurrence
of buffer overflow some flows may back off earlier than others. Nevertheless,
a lower bound for τu can be obtained for the general case. Noting that
P (PAi(minRi)|CLoi ) ≤
P (PAri (minRi))
P (CLoi )
and P (PAri (minRi)) = 0, we have
P (PAi(minRi)|CLoi ) = 0. Thus, a general lower bound for τu is minRi.
Now, we shall seek for an optimal solution for τCDiwithin [0, τu]. Con-
sider a time period t (0 ≤ t ≤ τu) after the retransmission of packet Pi, when
a TCP sender is facing the decision of whether or not to activate a congestion
response. The risk associated with a positive decision is that the network may
not be congested (i.e. P oi is not dropped due to congestion). Consequently,
the spuriously activated congestion response will reduce cwnd unnecessarily.
On the other hand, if the network is indeed overloaded (i.e. P oi is lost due to
congestion) and the activation of congestion response is excessively delayed,
the network congestion may be exacerbated and an expensive RTO may be
incurred.
To reach a rational decision regarding the optimal τCDi, we need to com-
pare the quantified risks of activating congestion response and delaying con-
gestion response. Before doing that, we first quantify the costs of falsely
activating congestion response and excessively delaying congestion response
when network is indeed overloaded, respectively, by the following two defini-
tions.
Definition 3.1 The cost of falsely activating congestion response, denoted
as CS, is the average number of packets a TCP sender misses to send due to
the activation of a congestion response,
34
Definition 3.2 The cost of excessively delaying congestion response, de-
noted as CT , is the average number of packets a TCP sender misses to send
due to the occurrence of RTO.
Consider the activation of congestion response, when both cwnd and
ssthresh are set to 0.5 W . Thus TCP sender is at the congestion avoidance
stage to begin. cwnd is incremented by one at every RTT cycle afterwards.
Let nCA be the number of cycles during which the TCP sender is in the
congestion avoidance stage with cwnd less than W . Accordingly, we have:
nCA = ⌊0.5 W⌋ (3.10)
Therefore, when a congestion response is falsely activated, the number of
packets the TCP sender misses to send is:
CS = W nCA −nCA∑
i=1
(⌈0.5 W⌉+ i− 1)
= 0.5 ⌊0.5 W ⌋ (⌊0.5W ⌋+ 1) (3.11)
Consider the occurrence of an RTO, when cwnd is reset to one and
ssthresh is set to 0.5 W . For simplicity, we would assume that ACK ar-
rives before the occurrence of another RTO. Thus, after being reset, the
TCP sender is in the slow start stage and W will double itself at every RTT
round as long as it is less than ssthresh. Once W reaches ⌈ssthresh⌉, the TCP
sender leaves the slow start stage and enter the same congestion avoidance
stage. Let nSS be the number of cycles during which the TCP sender is in
35
the slow start stage, we have:
nSS = ⌈log2 (0.5 W )⌉ (3.12)
Upon a genuine congestive loss, it is generally desirable for a TCP sender
to halve cwnd and directly enter the congestion avoidance stage. Therefore,
if an RTO occurs instead and forces cwnd to be reset to one, the TCP sender
throughput is penalized when the slow start phase is activated. The number
of packets the TCP sender misses to send is:
CT = W nSS −nSS∑
i=1
2i−1
= ⌈log2 (0.5 W )⌉ ·W − 2⌈log2 (0.5 W )⌉ + 1 (3.13)
We are now ready to quantify the risks associated with activating a con-
gestion response and delaying congestion response. The following two metrics
are introduced, respectively, for the purpose.
Definition 3.3 The expected cost of activating a congestion response, de-
noted as ECA(t), is the product of the conditional probability of P oi not being
lost due to congestion, given that Pi is unacknowledged, and the cost of falsely
activating congestion response. In other words,
ECA(t) = P (CLoi |PUi(t)) · CS (3.14)
Definition 3.4 The expected cost of delaying a congestion response, de-
noted as ECD(t), is the product of the conditional probability that a conges-
36
tive loss and an RTO have occurred, given that Pi is unacknowledged, and
the cost of excessively delaying congestion response. In other words,
ECD(t) = P (TO ∩ CLoi |PUi(t)) · CT (3.15)
The two conditional probabilities involved in Definitions 3.3 and 3.4 are
expressed in terms of loss probabilities, namely PC , PW , and PL, and distri-
bution function of RTT, F (t), by Lemmas 3.2 and 3.3, respectively.
Lemma 3.2 For 0 ≤ t ≤ τu,
P (CLoi |PUi(T ))
=PW [1− (1− PL)F (t)]
PC + PW [1− (1− PL)F (t)](3.16)
(3.17)
Proof We begin by noting that:
P (CLoi ) = P (PUi(t)) · P (CLo
i |PUi(t))
+P (PAi(t)) · P (CLoi |PAi(t)) (3.18)
For 0 ≤ t ≤ τu, since P (CLoi |PAi(t)) = 0 by Lemma 3.1, this reduces to:
P (CLoi ) = P (PUi(t)) · P (CLo
i |PUi(t)) (3.19)
It follows that:
P (CLoi |PUi(t)) = 1− P (CLo
i |PUi(t))
37
= 1− P (CLoi )
P (PUi(t))(3.20)
where 0 ≤ t ≤ τu.
We now derive P (PUi(t)), which will complete the proof.
P (PUi(t))
= P (PUoi (t) ∩ PU r
i (t))
= P (PUoi (t) ∩ PU r
i (t)|PLoi )P (PLo
i )
+P (PUoi (t) ∩ PU r
i (t)|PLoi )P (PLo
i ) (3.21)
Since:
P (PUoi (t) ∩ PU r
i (t)|PLoi ) ≤ P (PUo
i (t)|PLoi )
= 1− F (t + τRDi)
≤ ζ (3.22)
where ζ ≪ 1 by (3.6), we have:
P (PUi(t)) ≈ P (PUoi (t) ∩ PU r
i (t)|PLoi )P (PLo
i ) (3.23)
Noting that:
P (PUoi (t) ∩ PU r
i (t)|PLoi )
= P (PU ri (t)|PLo
i )− P (PU ri (t) ∩ PAo
i (t)|PLoi )
(3.24)
38
and
P (PAoi (t) ∩ PU r
i (t)|PLoi ) ≤ P (PAo
i (t)|PLoi ) = 0 (3.25)
we further obtain:
P (PUoi (t) ∩ PU r
i (t)|PLoi ) = P (PU r
i (t)|PLoi ) (3.26)
(3.23) thus simplifies to:
P (PUi(t)) ≈ P (PU ri (t) ∩ PLo
i )
= P (PU ri (t) ∩ CLo
i ) + P (PU ri (t) ∩NLo
i )
= P (CLoi )P (PU r
i (t)|CLoi )
+P (PU ri (t)) · P (NLo
i ) (3.27)
The last step above applies (3.1)
Recall that τu = suptt|P (PAi(t)|CLoi ) = 0. Thus, for 0 ≤ t ≤ τu,
P (PU ri (t)|CLo
i ) ≥ P (PUi(t)|CLoi )
= 1− P (PAi(t)|CLoi ) = 1 (3.28)
Substituting (3.28) into (3.27) yields:
P (PUi(t)) ≃ P (CLoi ) + P (PU r
i (t)) · P (NLoi )
(3.29)
39
where P (PU ri (t)) can be derived as:
P (PU ri (t)) = P (PU r
i (t)|PLri )P (PLr
i )
+P (PU ri (t)|PLr
i )P (PLri )
= PL + [1− F (t)](1− PL) (3.30)
Substituting (3.30) into (3.29), we obtain:
P (PUi(t)) = PC + [1− (1− PL)F (t)]PW (3.31)
where 0 ≤ t ≤ τu.
Lemma 3.3 For 0 ≤ t ≤ τu,
P (TO ∩ CLoi |PUi(t)) =
PC PL
PC + PW [1− (1− PL)F (t)](3.32)
Proof Following a similar process to (3.18) - (3.20), we can obtain:
P (TO ∩ CLoi |PUi(t)) =
P (TO|CLoi )P (CLo
i )
PUi(t)
(0 ≤ t ≤ τu) (3.33)
Equ. (3.32) follows by substituting (3.31) and (3.2) into (3.33).
Thus:
ECA(t) =PW [1− (1− PL)F (t)]
2PC + PW [1− (1− PL)F (t)]×⌊0.5 W ⌋(⌊0.5W ⌋+ 1) (3.34)
40
DEC (t)
EC (t)A
tτu
(a) Case I.
τ
DEC (t)
EC (t)A
tu
(b) Case II.
τTH
EC (t)A
EC (t)D
tτu
(c) Case III.
Figure 3.2: Evaluation of τ ∗CDi
.
ECD(t) =PC PL
PC + PW [1− (1− PL)F (t)]
×(⌈log2 0.5W ⌉W − 2⌈log2 0.5W ⌉ + 1) (3.35)
Now,
∂
∂tECA(t) ≤ 0,
∂
∂tECD(t) ≥ 0 (3.36)
Thus, when the activation of congestion response is postponed further, ECD(t),
which quantifies the risk associated with delaying a congestion response, in-
creases, while ECA(t), which quantifies the risk associated with activating
a congestion response, drops. When ECA(t) is greater than ECD(t), it is
advantageous to set τCDino less than t since the operation cost of activat-
ing congestion response is ECA(t), which is larger than that of deferring it,
ECD(t). The cost may drop when t increases. Similarly, when ECD(t) is
greater than ECA(t), it is advantageous to set τCDino greater than t since
the operation cost of deferring congestion response is ECD(t), which is larger
than that of activating it, ECA(t). The cost may drop when t decreases.
41
Therefore, the evaluation of the optimal solution of τCDi, τ ∗
CDi, subject
to the constraint 0 ≤ τCDi≤ τu, can be divided into three cases, as depicted
in Figure 3.2. Case I arises when ECD(t) exceeds ECA(t) for any t ≥ 0.
The gap between the two would keep on increasing when t increases. Thus,
the congestion response should be activated at t = 0, or set τ ∗CDi
to be zero.
Case II arises when ECD(t) fails to catch up with ECA(t) for all t such that
0 ≤ t ≤ τu. Since we cannot delay a congestion response further according to
the prior constraint, we have to set τ ∗CDi
to be τu. The final case arises when
ECD(t) catches up with ECA(t) for some t such that 0 ≤ t = τTH ≤ τu. τ ∗CDi
thus corresponds to τTH since ECA(t) is greater than ECD(t) prior to τTH
and less than ECD(t) after τTH .
The optimal value of τCDican thus be determined by the following theo-
rem.
Theorem 3.1 Suppose Assumptions 1-3 hold. The optimal value for τCDi
within [0, τu] in the sense of minimizing the associated expected cost, τ ∗CDi
, is
given by:
τ ∗CDi
=
0, PC > (PL · CT
CS+ 1)−1 PL
τu, PC < PL CT
[1−(1−PL)·F (τu)]·CS+ 1−1 · PL
τTH , otherwise.
(3.37)
where τTH = F−1(
11−PL
− CT PC
CS ·(PL−PC)· PL
1−PL
)
.
Proof The proof follows from the foregoing discussion.
42
When non-congestive loss is negligible, we fall into the first condition in
Theorem 3.1. Packet retransmission and congestion response should thus be
activated simultaneously. When non-congestive loss becomes non-negligible
(say, PW > PC), it generally holds that PC < (PLCT
CS+ 1)−1 PL. It follows
that τ ∗CDi∈ [minRi, τu].
3.3 TCP-NCL
The STS model is a limited, idealized TCP sender. First, it does not pro-
vide a closed-form expression for τRD i. Second, it assumes, among other
things, prior knowledge of the RTT distribution F (t), which is impractical.
Therefore, we propose TCP-NCL to closely approximate STS.
The serialized-timer structure of the STS model is supplemented by TCP-
NCL with the NCL-RTT-Update (NRU) process for maintaining statistics of
RTT samples so as to estimate F (t). Section 3.3.1 describes the NRU pro-
cess. Section 3.3.2 explains the implementation of RD and CD timers using
timestamps. The pseudocode for these are exhibited in Algorithms 3.3.1 and
3.3.1, respectively. Section 3.3.3 describes the minor modifications involved
in the retransmission timeout (RTO) timers. Section 3.3.3 describes kernel
implementation of TCP-NCL and the overhead involved.
3.3.1 NRU Process
The NRU process is invoked if an ACK, say ACK i, arrives before the ex-
piration of its corresponding CD timer, CD i. This can be further divided
into two cases: ACK i arrives before or after RD i expires. In the latter
43
case, there is an ambiguity regarding whether ACK i is for P oi or P r
i . Thus,
some additional measures would be needed to make sure that ACK i is for
P oi before recording the corresponding RTT sample. We identify the time
instance for the most recent transmission of Pi as txTime[i]. Upon the arrival
of ACKi, if the present time (identified by now) exceeds txTime[i] by less
than β τCD i(0 < β < 1), this infers that P r
i cannot be acknowledged within
such a short interval and thus the RTT sample (τRD i+ now − txTime[i]),
is inserted into rttRcd. Otherwise, the corresponding RTT sample will be
ignored.
By updating RTT based on ACKs received in this manner, the RTT
sampling process is more robust to changes in the network environment,
especially to abrupt increases in RTT caused by path change or handoff.
When the latter occurs, it is likely that ACK i will not be received before
RTi expires. Yet, the corresponding RTT sample can still be accurately
recorded as long as it does not exceed (τRD i+ β τCD i
).
The assignment of τRD iand τCD i
, as will be discussed in Section 3.3.2,
only relies on the maximum RTT sample, maxR, and minimum RTT sample,
minR. There is thus no need to keep all the recent maximum RTT record
length (MRRL) samples in storage. Instead, when a new RTT is greater
than maxR/less than minR, it is used to update maxR/minR.
However, merely doing so may prevent us to discard some outdated sam-
ples that are very large or very small. The distribution of RTT is time-
variant over wireless networks, where the network topology and/or loading
may change over a TCP session. The time-varying property of F (t) requires
prompt discard of outdated samples. In [10], TCP-PR needs to keep track
44
of updated maximum RTT sample as well. A srtt variable is maintained
for the purpose. Upon obtaining a new RTT sample, srtt is multiplied by
α(− 1
cwnd), where α is a system designed parameter in the interval (0, 1).
The new sample is then used to update srtt if greater than the current value
of srtt. The multiplication, which reduces srtt by a factor of α per RTT,
ensures that outdated peaks in RTT will eventually be replaced. However,
it is generally computationally expensive for the kernel.
We thus apply a computationally cheaper strategy for the similar purpose
of discarding outdated maximum and minimum RTT samples. We will main-
tain two new variables maxR2 and minR2 in supplement to the prescribed
updating process of maxR and minR.
A periodic downward counter, rcdAge, is initialized as MRRL and decre-
mented by one upon obtaining a new RTT sample. Similarly as maxR, an
RTT sample is used to update maxR2 if greater than it. Yet, when rcdAge
reaches zero, maxR will be set as maxR2 before maxR2 is set as the newly
obtained RTT sample without comparison. rcdAge is then set as MRRL
again.
Thus, every time upon completing the updates on maxR, maxR2, and
rcdAge, maxR2 always maintains the maximum value within the most recent
(MRRL− rcdAge + 1) RTT samples. Regarding maxR:
1. When rcdAge is MRRL, maxR has just been updated by the old value
of maxR2. In addition, it will be set to the new RTT sample if smaller
than it. Thus, it will keep the maximum value within the most recent
(MRRL + 1) RTT samples.
45
2. Upon obtaining the next n (1 ≤ n ≤ MRRL − 1) samples, rcdAge is
decremented by one each time and thus n = MRRL − rcdAge. Since
rcdAge is greater than zero, maxR is simply updated by the new RTT
if smaller than it. Thus, it maintains the maximum value within the
most recent ((MRRL + 1) + n) RTT samples.
3. Upon obtaining one more sample, rcdAge will be decremented to zero
and then set to MRRL. We thus go back to the situation described to
1).
Overall, maxR always maintains the maximum value within the most recent
(2MRRL - rcdAge + 1) RTT samples, where 1 ≤ rcdAge ≤ MRRL. This
enables discard of outdated samples while ensuring maxR is the maximum
among a sufficiently large number (≥ MRRL+1) of most recent RTT samples.
The maintenance and usage of minR2 are carried out in an analogous
manner. It is therefore ensured that minR maintains the minimum value
within the most recent (2MRRL - rcdAge + 1) RTT samples.
3.3.2 RD and CD Timers
τRD iis set to (maxR + minR). Intuitively, this assignment is an appro-
priate choice, keeping spurious retransmission at a minimal level without
excessively delaying fast retransmit. As will be shown by our simulation re-
sults, TCP-NCL demonstrates a very efficient usage of network bandwidth in
that it almost always attains the highest connection goodput among all our
compared TCP variants. This performance can hardly be achieved if either
spurious retransmission constitutes a significant wastage of total through-
46
Algorithm 1 Procedure NRU (Packet Pi)
1: rcdAge ← rcdAge − 12: if Pi is on rcPkts then3: rtt ← now − txTime[i]4: else5: if now − txTime[i] < β ∗ tauCD then6: rtt ← now − txTime[i] + tauRD7: else8: exit from the process9: end if
10: end if11: if rcdAge = 0 then12: minR ← minR213: maxR ← maxR214: minR ← rtt15: minR ← rtt16: rcdAge ← MRRL17: end if18: maxR2 ← max(maxR2 , rtt)19: maxR ← max(maxR, rtt)20: minR2 ← min(minR2 , rtt)21: minR ← min(minR, rtt)22: tauCD ←maxR + minR23: tauCD ← K ∗maxR + (1−K) ∗minR
47
Algorithm 2 Procedure update list (Packet ACK i)
1: for each Packet Pj on cdPkts do2: if tauCD < now − txTime[j] then3: remove Pj from cdPkts4: activate congestion response5: end if6: end for7: Retrieve the Packet Pi acked by ACK i
8: if Pi is in rdPkts or cdPkts then9: NRU (Pi)
10: remove Pi from rdPkts/cdPkts11: end if12: for each Packet Pj on rdPkts do13: if tauRD < now − txTime[j] then14: remove Pj from rdPkts15: add Pj to to-be-rtxed16: end if17: end for18: wnd ← min(cwnd , awnd)19: while number of outstanding packets ≤ wnd do20: if there is a Packet Pj on to-be-rtxed then21: add Pj to cdPkts22: retransmit Pj
23: txTime[j]← now24: else25: Transmit the next new packet Pk
26: Add Pk to rdPkts27: txTime[k]← now28: end if29: end while
48
put, or RTO frequently occurs and forces cwnd to be reset to one. Thus, the
choice of setting τRD ias (maxR + minR) seems to be appropriate.
By the discussion at the end of previous section, τ ∗
CD i
∈ [minRi, τu].
minRi can be approximated by minR, whereas τu is lower bounded by minR
and can be up to maxR. Thus, we assign τCDito be (K ∗maxR + (1−K) ∗
minR), where K ∈ [0, 1] is a system-designed parameter. It is worth noting
that, while STS model specifies τ ∗
CD i
dependant on PC , we remove such
dependence in TCP-NCL. This is because congestive loss are usually rate
events in the network, and it is very difficult to measure PC accurately from
end systems.
With our approximation above, at any time instance, the expiration pe-
riods of RD/CD timers for all the RD/CD pending packets would be the
same. We thus define two variables, namely, tauRD and tauCD, for keeping
the present values of the expiration periods of RD and CD timers, respec-
tively, instead of keeping a unique copy for each packet.
To fulfill the effects of the RD and CD timers without incurring the heavy
cost of maintaining one timer per outstanding packet, we define rdPkts and
cdPkts for keeping the packets whose RD and CD timers are supposedly
pending, respectively. The two lists are updated upon arrival of an acknowl-
edgment packet as follows:
1. Activate congestion response measure if there are packets that have
stayed on cdPkts for more than tauCD, and remove the timeout’ed
packets.
2. Remove the acknowledged packet if it is on rdPkts or cdPkts.
49
3. Retransmit packets that have stayed on rdPkts for more than tauRD,
and transfer them from rdPkts to cdPkts with timestamps recording
the retransmission time instance.
4. Transmit new packets and add them to rdPkts with timestamps record-
ing the transmission time instance.
When activating the congestion control measures, we adopt an approach
similar to that used in TCP-PR, that packet losses within the same burst are
considered as a single signal about the onset of network congestion and the
reduction of cwnd will be triggered only once. Thus, when cwnd is halved,
all the present packets in rdPkts and cdPkts are exempted from causing cwnd
to be halved again.
3.3.3 RTO Timer
With standardized TCP design, severe congestion is inferred and go-back-N
retransmission is performed by retransmitting all outstanding packets upon
the occurrence of RTO. The expiration period of RTO timer is set to the sum
of mean RTT and a multiple of RTT deviation. The RTO timer is reset upon
packet (re)transmission. Since fast retransmit performs retransmission upon
receiving three duplicate acknowledgments, the (fast) retransmitted packet
is allowed sufficient time to be acknowledged before the occurrence of RTO
and the associated the go-back-N retransmission.
We keep RTO timer in the design of TCP-NCL so as to perform worst-case
recovery. Moreover, this ensures that TCP-NCL can emulate the behavior of
standardized TCP design in conventional wired networks, thereby reducing
50
its possible disruptions to the latter. Yet, the occurrence of RTO needs to
be well coordinated with the serialized-timer structure that has postponed
packet retransmission and congestion response proactively. It is undesirable
for RTO timer to fire before the RD and CD timers fires. Moreover, a packet
retransmitted upon the expiration of RD timer should be allowed sufficient
time to be acknowledged before the go-back-N retransmission is considered
necessary and activated. Thus, TCP-NCL impose two modifications to the
RTO timer:
1. The expiration period of RTO timer is modified to be (tauRD + 2 ∗
tauCD).
2. A further condition for performing go-back-N retransmission is im-
posed. Upon the occurrence of RTO, it checks whether there are
retransmitted packets that are not acknowledged within tauRD, or
whether the congestion window is no greater than one. If either is
true, rdPkts and cdPkts will be cleared, and all the outstanding pack-
ets will be retransmitted at a data rate regulated by the congestion
window.
3.3.4 Kernel Implementation
The difficulty in modifying the kernel hinders the implementation of most
TCP variants proposed. To facilitate the implementation, we devise a generic
framework for developing TCP variants in the Linux/UNIX kernel. We ob-
serve that a majority of TCP variants differ in their congestion control and
loss recovery operations, which are realized cooperatively by three TCP ma-
51
jor functions2, namely, the TCP input function tcp input, the TCP output
function tcp output, and the RTO timer function tcp timer rexmt, as illus-
trated in Figure 3.3. Thus, a TCP variant can be deployed by: 1) defining its
data structures in the TCP control block, 2) maintaining the data structures
in the three aforementioned functions, and 3) modifying/replacing the fast
retransmit, fast recovery, and/or RTO algorithms.
socket receivebuffer
tcp_input
ipintr
tcp_output
ip_output
tcp_timer_rxmt
softclock
tcp_usrreq
system calls
Figure 3.3: Software architecture for TCP.
Following the framework, we have deployed a first version of TCP-NCL .
1) The sk buff structure in TCP control block records all the outstanding
packets with timestamps, and each packet has several associated flags that are
left unused [63]. The timestamps suffices for the purpose of txTime[i], and the
unused flag bits can be set to indicate the membership of a packet in rdPkts
and cdPkts. We further define a few extra variables, namely minR, maxR,
minR2, maxR2, rcdAge, tauRD, and tauCD in TCP control block. Since each
TCP session maintains one copy of TCP control block, the implementation of
TCP-NCL requires storing a small number of extra variables per connection
only.
2) In accordance with our algorithms described in Sections 3.3.1-3.3.3,
2The architecture follows that of BSD UNIX [73]. Despite the possible differences inthe naming conventions and function calling sequences, most Linux/UNIX kernels adoptsimilar structures in their TCP/IP protocol stacks.
52
SS DBS100 Mbps 1 Mbps
50 ms5 msS
(a) Infrastructure-based wireless network.
R3 DR1SSS1 Mbps
50 ms50 ms
1 Mbps
(b) Multi-hop wireless network.
1MbpsR2R15 ms
S1
S2
100 Mbps5 ms
SN100 Mbps
5 ms
50ms...
D2100 Mbps
5 ms
100 Mbps
...
D1100 Mbps5 ms
DN100 Mbps
5 ms
(c) Wired network with a dumbbell topology.
Figure 3.4: The network topologies used for performance comparison
these variables are updated in tcp input based on the incoming acknowl-
edgements, and properly reset in tcp timer rxmt. This requires a constant
number of extra maintenance steps per data packet. Again, we stress that
the RD and CD timers are approximated via timestamps, and implementing
TCP-NCL does not require maintenance of extra timers.
3) The fast retransmit and fast recovery algorithms are replaced in tcp input
by our serialized timer structure. The major change involved is to activate
fast recovery upon detecting CD timeout’ed packets (namely, packets that
have stayed in cdPkts for more than tauCD) instead of upon activating fast
retransmit. The extra overhead involved is minimal.
53
3.4 Performance Evaluation
In this section, we present our simulation results. We have performed exten-
sive simulation experiments to evaluate the performance of TCP-NCL using
Network Simulator (ns) 2.29 [19].
We have evaluated several instances of TCP-NCL with different settings
of K. Recall that K ∈ [0, 1] is the system-designed parameter involved in
setting the expiration period of the CD timer (tauCD = (1 − K) · minR +
K · maxR). A larger value of K generally corresponds to a more aggres-
sive instance of TCP-NCL. In the following, we will focus on two instances
of TCP-NCL, TCP-NCL (K=0) and TCP-NCL (K=0.5), with K set to 0
and 0.5, respectively. TCP-NCL (K=0) therefore corresponds to the most
conservatively configured TCP-NCL.
We have also included a TCP variant degenerated from TCP-NCL, TCP-
NCL-RD, for comparison. TCP-NCL-RD differs from TCP-NCL in that it
disables the CD timer by setting the expiration period of the CD timer to
zero. Thus, it proactively postpones packet retransmission until the expira-
tion of the RD timer, when it will assume the packet is lost due to congestion
and activates congestion response and packet retransmission simultaneously.
We aim to examine the effect of the serialized-timer structure by comparing
TCP-NCL with TCP-NCL-RD.
The performance curves of four TCP variants in the literature, namely
RR-TCP [81], TCP-DCR [8], TCP-PR [10], and TCPW [13], are replicated
from our ongoing study [43] for comparison. The former three variants attain
the best goodput performance against packet reordering according to our
54
0
0.2
0.4
0.6
0.8
1
0 0.02 0.04 0.06 0.08 0.1
Con
nect
ion
Goo
dput
(M
bps)
Packet Error Rate
TCP-NCL (K=0.5)TCP-NCL (K=0)
TCPWTCP-NCL-RD
RR-TCPTCP-PR
TCP-DCR
Figure 3.5: Goodput performance against packet error rate overinfrastructure-based wireless network.
simulation-based comparison in [77], while TCPW is well known as a solution
for wireless packet losses.
The simulated network topologies include the infrastructure-based wire-
less network, the multi-hop wireless network, and the wired network with a
dumbbell topology, as illustrated in Figure 3.4. Unless specified otherwise,
the queue management algorithm for each link buffer is drop tail. The packet
size is 1000 bytes. More detailed network configurations will be described in
the following.
Sections 3.4.1 and 3.4.2 examine and compare the performance of all
the aforementioned TCP variants against non-congestive loss and packet re-
ordering over the infrastructure-based wireless network and the multi-hop
wireless network ,respectively. Section 3.4.3 examines the responsiveness of
TCP-NCL against congestive loss over the wired network with a dumbbell
topology.
55
3.4.1 Non-congestive loss
The infrastructure-based wireless network is illustrated in Figure 3.4 (a). A
TCP sender and a TCP receiver are connected through a wired with band-
width 100 Mbps and propagation delay 5 ms, and a wireless link with band-
width bw Mbps and propagation delay d ms. Random packet errors with rate
pe are deliberately introduced into the wireless link. Packets experiencing
transmission errors are discarded at the receiving end.
In each test over the topology, a total of 20 runs, each lasting 2000 seconds
and using different seeds for generating the packet error, have been performed
to compute an average value and a 95% confidence interval of TCP goodput
in Mbps. In order to remove the effect of the transient states, only the
statistics in the last 1000 seconds in each run are collected for computing the
goodput.
Figures 3.5, 3.6, and 3.7 plot the connection goodput of the TCP variants
against pe, d, and bw, respectively. In each plot, the legends for TCP variants
are sorted descendingly according to their levels of goodput at the rightmost
data point. In most cases, the confidence intervals are too narrow to be
distinguishable. The configuration settings are summarized in Table 3.3. A
hyphen (-) in an entry indicates that the corresponding parameter is being
varied to generate the plot.
Figure 3.5 exhibits the connection goodputs of the TCP variants against
packet error rate pe. The two instances of TCP-NCL attains the best per-
formance. At pe = 1%, TCP-NCL (K=0.5) and TCP-NCL (K=0) attains
around 30% and 15% performance improvement over TCP-NCL-RD, respec-
56
0
0.2
0.4
0.6
0.8
1
50 100 150 200 250 300 350 400
Con
nect
ion
Goo
dput
(M
bps)
Delay of the Wireless Link (ms)
TCP-NCL (K=0.5)TCPW
TCP-NCL (K=0)TCP-NCL-RD
TCP-DCRTCP-PRRR-TCP
(a) pe = 1%.
0
0.2
0.4
0.6
0.8
1
50 100 150 200 250 300 350 400
Con
nect
ion
Goo
dput
(M
bps)
Delay of the Wireless Link (ms)
TCP-NCL (K=0.5)TCPW
TCP-NCL (K=0)TCP-NCL-RD
TCP-DCRTCP-PRRR-TCP
(b) pe = 5%.
Figure 3.6: Goodput performance against delay of the wireless link overinfrastructure-based wireless network.
tively. Their performance gains further increase as pe increases, and reach
around 200% and 100%, respectively, at pe = 10%. These verify the ef-
fectiveness of the installation of the CD timer in combating wireless losses.
TCPW attains the third best performance. It performs similarly as TCP-
NCL (K=0) at pe (up to 2%), but performs slightly worse as pe further
increases. TCP-NCL-RD, RR-TCP, TCP-DCR, and TCP-PR demonstrates
similar drastic decrease in goodput as pe increases. They exclude the pos-
57
0
1
2
3
4
5
6
7
0 1 2 3 4 5 6 7
Con
nect
ion
Goo
dput
(M
bps)
Bandwidth of the Wireless Link (Mbps)
TCP-NCL (K=0.5)TCPW
TCP-NCL (K=0)TCP-PR
TCP-DCRTCP-NCL-RD
RR-TCP
(a) pe = 1%.
0
1
2
3
4
5
6
7
0 1 2 3 4 5 6 7
Con
nect
ion
Goo
dput
(M
bps)
Bandwidth of the Wireless Link (Mbps)
TCP-NCL (K=0.5)TCPW
TCP-NCL (K=0)TCP-PR
TCP-DCRTCP-NCL-RD
RR-TCP
(b) pe = 5%.
Figure 3.7: Goodput performance against bandwidth of the wireless link overinfrastructure-based wireless network.
sibility of non-congestive packet loss, resulting in severe under-utilization of
network resources.
Figure 3.6 exhibits the connection goodputs of the TCP variants against
propagation delay of the wireless link d, for pe = 1%, 5%. All TCP variants
generally perform worse as d increases. TCP-NCL (K=0.5), TCPW, and
TCP-NCL (K=0) attain much higher connection goodput than other TCP
variants. TCP-NCL-RD, RR-TCP, TCP-DCR, and TCP-PR demonstrates
58
Table 3.3: Network configurations for the infrastructure-based wireless net-work
Figure bw d peFigure 3.5 1 50 -
Figure 3.6(a) 1 - 1%Figure 3.6(b) 1 - 5%Figure 3.7(a) - 50 1%Figure 3.7(b) - 50 5%
similar severe performance deterioration as d increases. To fully utilize the
wireless link capacity, the size of congestion window (cwnd) has to be at least
the bandwidth-delay product, namely the product of bw and the round-trip
propagation delay. Such delay increases linearly as d increases. With a larger
value of d, it takes a TCP connection longer to resume cwnd to be above the
product upon spurious reduction of cwnd due to non-congestive losses.
Figure 3.7 exhibits the connection goodputs of the TCP variants against
bandwidth of the wireless link bw, for pe = 1%, 5%. TCP-NCL (K=0.5),
TCPW and TCP-NCL (K=0) manages to maintain a linear increase in good-
put as bw increases up to 4 Mbps, thereby maintaining a constant bandwidth
utilization. All other TCP variants essentially maintain constant connec-
tion goodput as bw increases, corresponding to a decrease in bandwidth
utilization. Similar inferences made from Figure 3.6 can apply here. The
bandwidth-delay product increases linearly proportionally to bw. With a
larger value of d, it takes a TCP connection longer to recover from spurious
cwnd reduction due to non-congestive losses.
Up to now, we can observe that the two instances of TCP-NCL attain
significant performance gains over TCP-NCL-RD across different configura-
59
tions of bw, d, for non-zero pe, by merely the installation of the CD timers.
The serialized-timer structure thus proves to be very effective in shielding
the impact of non-congestive losses.
We now draw a closer comparison between TCP-NCL and TCPW. We
focus on comparing TCP-NCL (K=0), the most conservatively configured
TCP-NCL, with TCPW. Generally, increasing K helps TCP-NCL to avoid
spurious reduction of cwnd more effectively. Thus, all inferred advantages
of TCP-NCL (K=0) over TCPW in combatting non-congestive losses will
safely hold for TCP-NCL configured otherwise.
TCP-NCL and TCPW apply distinctly different strategies for perform-
ing effective congestion avoidance in the presence of non-congestive losses.
TCP-NCL tries to construct more reliable signals of congestive loss by differ-
entiating between congestive loss and non-congestive loss, and halves cwnd
when inferring congestive loss. TCPW does not perform the differentiation.
Rather, it reduces cwnd to its estimated available network capacity upon
each inferred loss. Therefore, we can make three inferences in comparing
TCP-NCL and TCPW:
1. In the presence of non-congestive losses, TCPW tends to trigger spu-
rious reduction of cwnd more frequently than TCP-NCL. This is ex-
emplified scenarios with low bandwidth-delay product and high packet
error rates. For example, TCP-NCL (K=0) outperforms TCPW for
pe ≥ 3%, d = 50, bw = 1 in Figure 3.5.
2. When both perform a spurious reduction of cwnd due to non-congestive
losses, TCP-NCL tends to reduce cwnd more abruptly than TCPW.
60
This is exemplified in scenarios with high bandwidth-delay product
and low packet error rates. For example, TCPW outperforms TCP-
NCL (K=0) for pe = 1%, d ≥ 100, bw = 1 in Figure 3.6(a), and for
pe = 1%, d = 50, bw ≥ 2 in Figure 3.7(a).
3. The aforementioned two inferences can counteract each other in scenar-
ios with both high bandwidth-delay produce and high packet error rate.
For example, TCP-NCL (K=0) performs similarly as TCPW for pe =
5%, d ≥ 100, bw = 1 in Figure 3.6(b) and for pe = 5%, d = 50, bw ≥ 2
in Figure 3.7(b).
3.4.2 Packet Reordering
In the multi-hop wireless network as illustrated in Figure 3.4 (b), a TCP
sender is connected to a TCP receiver via six wireless links. Random chan-
nel error is introduced into the wireless links with a random packet error
rate ranging from 0 to 10%. Packets experiencing transmission errors are
discarded at the receiving end of the wireless link. Yet, the setting differs
from the infrastructure-based wireless network in that the link layer packets
thus discarded are locally retransmitted at the link layer around 110 ms after
its most recent (re)transmission. The retransmission period of 110 ms is the
sum of two-way propagation delay, (50 ms× 2), one-way transmission delay,
(1000 bytes/1 Mbps), and a delay margin of 2 ms. The retransmission will
be performed for up to three times. This greatly enhances the chance of
successful packet delivery at each hop. Yet, packet reordering will be intro-
duced since the retransmitted packets will be intermixed with later packets.
61
0
0.2
0.4
0.6
0.8
1
0 0.02 0.04 0.06 0.08 0.1
Con
nect
ion
Goo
dput
(M
bps)
Packet Error Rate
TCP-NCL(K=0.5)TCP-PR
TCP-NCL(K=0)TCP-NCL-RD
TCPWTCP-DCR
RR-TCP
(a) Four hop.
0
0.2
0.4
0.6
0.8
1
0 0.02 0.04 0.06 0.08 0.1
Con
nect
ion
Goo
dput
(M
bps)
Packet Error Rate
TCP-NCL(K=0.5)TCP-PR
TCP-NCL(K=0)TCP-NCL-RD
TCPWRR-TCP
TCP-DCR
(b) Six hop.
Figure 3.8: Goodput performance over multihop ad-hoc wireless network
The intensity of packet reordering will increase as packet error rate increases.
Moreover, under high channel error rate, some packets may still fail to be
correctly delivered after three link-layer retransmissions. Consequently, TCP
will be confronted with both packet reordering and non-congestive packet
loss.
Figure 3.8 exhibits the connection goodput of the TCP variants against
packet error rate over four-hop and six-hop wireless networks. Over the
four-hop connection (Figure 3.8(a)), the two instances of TCP-NCL, TCP-
62
PR, and TCP-DCR outperform other variants for pe up to 7%, thereby
demonstrating robustness to persistent reordering. TCP-NCL-RD performs
slightly worse. It activates congestion response if a packet not on ePkt is not
acknowledged tauRD(= maxR +minR) after packet transmission. The time
lag is generally shorter than that of TCP-NCL, TCP-PR, and TCP-DCR,
leading to a higher chance of premature congestion responses.
When pe further increases beyond 7% and non-congestive losses are no
longer transparent to the transport layer, TCP-NCL (K=0.5) outperforms
TCP-PR while the performance of TCP-DCR is seriously deteriorated. In
such scenario, the installation of the CD timers with a sufficiently long expi-
ration period helps TCP-NCL (K=0.5) increase the reliability for signals of
congestive loss.
Similar inferences can be made for the six-hop connection (Figure 3.8(b)),
and TCP-NCL (K=0.5) demonstrates more significant advantages over the
other variants.
3.4.3 Congestive Loss
In the wired network with a dumbbell topology as illustrated in Figure 3.4(c),
N pairs of TCP senders and receivers are sharing a wired bottleneck link with
bandwidth 5 Mbps, propagation delay 50 ms, and buffer size 50 packets. For
i = 1, 2, ...N , one TCP connection is set up from sender Si to receiver Di.
We examine the responsiveness of TCP-NCL against congestive loss over the
topology. In general, if a TCP variant, say, TCP-X, is competently responsive
against the occurrence of congestive loss, it is expected to fulfill the following
63
0.75
0.8
0.85
0.9
0.95
1
1.05
1.1
0 10 20 30 40 50 60 70
Jain
’s F
airn
ess
Inde
x
Number of Connections
SACK TCPTCP-NCL (K=0)
TCP-NCL (K=0.5)
(a) Drop tail.
0.75
0.8
0.85
0.9
0.95
1
1.05
1.1
0 10 20 30 40 50 60 70
Jain
’s F
airn
ess
Inde
x
Number of Connections
SACK TCPTCP-NCL (K=0)
TCP-NCL (K=0.5)
(b) RED.
Figure 3.9: Jain’s fairness index (J) over wired network with a dumbbelltopology.
two objectives:
1. (TCP-friendliness) A connection driven by standardized TCP design
(e.g. SACK TCP) can maintain similar goodput when its competing
connections are driven by TCP-X as that when its competing connec-
tions are driven by standardized TCP design. This ensures that TCP-X
will not grasp bandwidth from standardized TCP in wide current usage.
64
0
0.2
0.4
0.6
0.8
1
1.2
0 10 20 30 40 50 60 70
Rat
io o
f Mea
n G
oodp
ut to
Fai
r B
andw
idth
Number of Connections
Without TCP-NCLWith TCP-NCL (K=0)
With TCP-NCL (K=0.5)
(a) Drop tail.
0
0.2
0.4
0.6
0.8
1
1.2
0 10 20 30 40 50 60 70
Rat
io o
f Mea
n G
oodp
ut to
Fai
r B
andw
idth
Number of Connections
Without TCP-NCLWith TCP-NCL (K=0)
With TCP-NCL (K=0.5)
(b) RED.
Figure 3.10: GBR over wired network with a dumbbell topology.
2. (Intra-protocol fairness) Multiple competing connections driven by TCP-
X should be able to share links fairly.
For evaluating the TCP-friendliness of TCP-NCL, we conduct three tests
with drop tail as the buffer management algorithm for the bottleneck link.
In the first test, all connections are driven by SACK TCP. In the second
and third tests, half of the connections are driven by TCP-NCL (K=0) and
TCP-NCL (K=0.5), respectively, with the other half driven by SACK TCP.
65
In each test, suppose connections 1, 2, ..., M are SACK TCP connections and
M + 1, M + 2, ..., N are non-SACK TCP connections (and thus M = N in
the first test and M = N/2 in the remaining three tests), we compute the
goodput to bandwidth ratio (GBR):
GBR =1M
∑M
i=1 xi
1N
C(3.38)
where C denotes the capacity of the bottleneck link and equals 5 Mbps. The
GBR is the ratio of average SACK TCP connections to the fair bandwidth for
each TCP connection (namely the total link capacity divided by the number
of TCP connections), and is comparable across the three tests. Moreover, in
the first test, the multiple SACK TCP connections should be able to share
the bandwidth efficiently to generate the goodput, thereby attaining a GBR
close to one. By examining the GBR obtained in the latter two tests and
comparing them with the benchmark GBR obtained in the first case, we
can evaluate how the presence of TCP-NCL (K=0)/TCP-NCL (K=0.5) will
affect standardized TCP and thus the TCP-friendliness of TCP-NCL.
We then repeat the three tests with RED as the buffer management al-
gorithm.
Figure 3.10 plots GBR against N when the buffer management algorithm
for the bottleneck link is drop tail and RED. We observe that when compet-
ing with TCP-NCL (K=0), the GBR of SACK TCP is generally not affected
in most cases. The conservatively configured instance of TCP-NCL demon-
strates good TCP-friendliness. On the other hand, when competing with
TCP-NCL (K=0.5), SACK TCP suffers up to 10% to 40% decrease in GBR.
66
With a longer expiration period of the CD timer due to large K, some retrans-
mitted packets may be acknowledged before CD timer expires even though
the originally packet is lost due to congestion. TCP-NCL (K=0.5) may thus
occasionally miss the occurrence of congestive losses, grasping bandwidth
yielded by the backoff of SACK TCP and undermining its TCP-friendliness.
For measuring intra-protocol fairness, we introduce the Jain’s fairness in-
dex [31], J , as the performance metric. For a set of N connections, 1,2,..., N ,
sharing a bottleneck link with connection goodputs x1, x2, ..., xN , respec-
tively, J is defined as:
J ,(∑N
i=1 xi)2
N∑N
i=1 x2i
(3.39)
By construction, J is bounded between zero and one, and attains one only
when all xi’s are equal. Furthermore, it equals k/N when only k out of
the N connections share the bandwidth equally and the remaining (N − k)
connections receive a zero bandwidth.
Using drop tail as the buffer management algorithm for the bottleneck
link, we perform three simulation tests with all connections driven by SACK
TCP, TCP-NCL (K=0), TCP-NCL (K=0.5), respectively. We then repeat
the three tests using RED as the buffer management algorithm.
Figure 3.9 plots J against N . We observe that all the three variants in
comparison attain J close to one. This demonstrates that TCP-NCL is ca-
pable of maintaining similar intra-protocol fairness as the standardized TCP
design SACK TCP. In particular, even though connections driven by TCP-
NCL (K=0.5) are not very friendly towards connections driven by SACK
67
TCP, they are able to divide the bandwidth fairly among themselves.
3.5 Concluding Remarks
In this chapter, we have proposed a novel TCP variant, known as TCP-
NCL, as a unified solution for performing loss recovery, sequencing control,
and congestion control over general error-prone channels. In particular, we
propose the use of two serialized timers for obtaining more reliable signals for
packet loss and network overload separately. The STS model has been con-
structed based on the concept of expected cost and closed-form expressions
are derived as references for setting the timer expiration periods. We note
that the timers are mostly determined intuitively in existing work. Our simu-
lation investigations reveal that TCP-NCL is capable of attaining significant
performance improvement over general, error-prone channels. It also demon-
strates good responsiveness against congestive loss by maintaining effective
congestion avoidance and competent TCP-friendliness.
There are several possible extensions to our work, some of which are listed
below:
1. develop a distribution model for RTT so as to make TCP-NCL a mem-
oryless algorithm, and;
2. implement and examine the performance of TCP-NCL on experimental
testbeds.
68
Chapter 4
Enhancing AQM to Combat
Wireless Losses
4.1 Chapter Overview
In this chapter, we answer the following basic question that is largely unex-
plored in the literature:
(BQ) Can active queue management (AQM) maintain a stable, small
backlog under wireless losses?
In wireless networks, TCP tends misinterpret wireless losses as conges-
tive loss and back off spuriously. Nevertheless, when many TCP flows are
sharing a bottleneck wireless link, it is unlikely that a large fraction of them
will experience wireless losses simultaneously (unless a link is broken). Thus,
spurious backoffs due to wireless losses would affect the aggregate transmis-
sion of all flows much less severely than when there are only a few flows.
The wireless link will still be kept backlogged, as shown by our simulation
69
results.1 In our simulation experiments, we find that RED fails to maintain
a stable backlog under wireless losses.
Apparently, the problem may be made trivial by applying one of the
following two approaches:
1. Design a TCP enhancement that can fully differentiate between con-
gestive and wireless losses. Unfortunately, this is hardly attainable
due to the limited network information available to the end systems.
Our simulation experiments demonstrate that the performance prob-
lem of RED persists when the end systems are running wireless TCP
enhancements like TCP-NCL. As shown by our results in Chapter 3,
TCP-NCL is already very successful in differentiating congestive and
wireless losses.
2. Use packet marking instead of packet dropping to signal network con-
gestion, such as [6]. However, we argue that a globally enabled marking
scheme is impractical due to heterogeneity of the Internet.2
We therefore opt to tackle the problem through the loss-based AQM en-
hancements, which make use of packet drops for signalling network conges-
tion. We model the dynamics of the congestion control system around its
equilibrium by a linear control system, and study the interaction of wireless
losses and AQM in this control-theoretic framework. By employing the queue
1The observation has also been reported in the literature and shaped the design prin-ciples of several enhancements for adapting TCP to high-speed and/or wireless networks.Interested readers can refer to [15, 70] and the references therein.
2For example, explicit congestion notification (ECN) [61] is the most widely adoptedmarking scheme, but is generally disabled by default in several versions of Microsoft Win-dows [17].
70
size as the system output, wireless losses are considered as disturbances to
the system.
Furthermore, we propose a family of solutions for stabilizing the queue
size against wireless losses based on internal model principle [14]. Internal
model principle rejects disturbances, or forces the part of the output due
to disturbances to asymptotically converge to zero, by including a model of
the disturbances, or poles of the disturbances, as part of the feedback control
loop. In the case of the congestion control system, it can be realized by simply
modifying the formula of the packet dropping rate in AQM. The application
of internal model principle in combating wireless losses offers the following
advantages:
1. It enables AQM to adjust the packet dropping rate to compensate for
the packet drops due to wireless losses without any specific knowledge
about the wireless packet error rate (WPER);
2. It is realized as an independent module in AQM, and thus offers good
extensibility to incorporate other AQM enhancements, such as [76], for
improving inter-flow fairness in ad-hoc networks.
By assuming the intensity of wireless losses to be slowly time-varying, we
devise the integral controller (IC) as an embodiment of internal model princi-
ple. We have developed design rules for the IC. Our simulation results show
that IC is capable of maintaining stable queue sizes and performing effective
congestion avoidance against wireless losses. Its performance is maintained
in the presence of perturbing traffic flows (e.g. HTTP and constant bit rate
(CBR) traffic).
71
Section 4.2 discusses the BQ using several simulated examples, and ex-
poses the performance problem of RED. Section 4.3 presents our system
model, a linear control system obtained by linearizing the non-linear control
system of TCP/AQM, as the basis for our analysis. Section 4.4 proposes a
family of solutions for enhancing AQM to combat wireless losses based on
internal model principle, including the IC. The design rules are developed
for the IC. Section 4.5 presents our simulation results. Section 4.6 concludes
the paper and discusses some possible extensions to our work. The work
presented in this chapter has in parts been published in [37].
Notation conventions: we use bold fonts to denote matrices and vectors,
as in “x(t)”, except for the diagonal matrix Λ. When the time index is not
specified, such as “x”, we refer to an equilibrium value. We use x(t) and x(t)
to represent (x(t)−x) and (x(t)−x), respectively. λ(.) and σ(.) denotes the
eigenvalue and singular value of a matrix, respectively.
4.2 AQM under Wireless Losses
In this section, based on a simulation study, we discuss the BQ raised in
Section 4.1: Can AQM maintain a stable, small backlog under wireless losses?
R2R15 ms
S1
S2
100 Mbps5 ms
SN100 Mbps
5 ms
...D2
100 Mbps100 Mbps
...
D1100 Mbps
DN100 Mbps
1 ms
1 ms
1 ms
Figure 4.1: A wireless bottleneck link.
72
We use the simulation topology as exhibited in Figure 4.1, similar to [1,
15]. It models the scenario that two wired LANs are connected via a wireless
link. A total of N pairs of source and destination are connected via a wireless
bottleneck link with capacity 15 Mbps, and propagation delay 50 ms. Within
each pair (Si, Di), M long-lived TCP flows are set up from Si to Ni. The size
of a link buffer is 800 packets. We introduce vary the WPER of the bottleneck
link. Time is divided into periods, each of exponential distribution, with a
mean of 100 seconds. At the beginning of each period, the WPER of the
bottleneck link is determined from a uniform distribution over [0, 4%].3 The
error rate is then kept constant during that period, whose duration follows
an exponential distribution, and is changed again at the beginning of the
next period.
The AQM algorithm simulated is RED. The parameter settings of RED
follow the result of frequency domain analysis in [27]. The queue weight and
pMax are set to 2.66 · 10−6 and 0.1, respectively. minThresh and maxThresh
of RED are set to 150 and 700 packets, respectively.
We first consider the following question: Is a wireless link backlogged un-
der wireless losses? Figure 4.2 plots the queueing dynamics when the end sys-
tems are running TCP NewReno [21], a standard TCP variant. When there
are only 20 flows traversing the bottleneck link as shown in Figure 4.2(a),
spurious backoffs of some flows due to wireless losses affect the aggregate
transmission of the flows considerably. The aggregate connection goodput is
measured to be around 12 Mbps, or 80% of the link capacity, and the back-
3We present simulation results with more exhaustive parameter settings in Section 4.5.Nevertheless, the discussion in the following applies generally.
73
0
100
200
300
400
500
600
700
800
0 200 400 600 800 1000 1200 1400
Que
ue le
ngth
(nu
mbe
r of
pac
kets
)
Time (second)
RED
(a) N = 20, M = 1.
0
100
200
300
400
500
600
700
800
0 200 400 600 800 1000 1200 1400
Que
ue le
ngth
(nu
mbe
r of
pac
kets
)
Time (second)
RED
(b) N = 100, M = 1.
0
100
200
300
400
500
600
700
800
0 200 400 600 800 1000 1200 1400
Que
ue le
ngth
(nu
mbe
r of
pac
kets
)
Time (second)
RED
(c) N = 20, M = 5.
Figure 4.2: Queue length dynamics under varying wireless losses in the dumb-bell topology with TCP NewReno/RED.
74
0
100
200
300
400
500
600
700
800
0 200 400 600 800 1000 1200 1400
Que
ue le
ngth
(nu
mbe
r of
pac
kets
)
Time (second)
RED
(a) N = 20, M = 1.
0
100
200
300
400
500
600
700
800
0 200 400 600 800 1000 1200 1400
Que
ue le
ngth
(nu
mbe
r of
pac
kets
)
Time (second)
RED
(b) N = 100, M = 1.
0
100
200
300
400
500
600
700
800
0 200 400 600 800 1000 1200 1400
Que
ue le
ngth
(nu
mbe
r of
pac
kets
)
Time (second)
RED
(c) N = 20, M = 5.
Figure 4.3: Queue length dynamics under varying wireless losses in the dumb-bell topology with TCP-NCL/RED.
75
log is zero most of the time. In this case, the major concern is to enhance
wireless TCP to improve the link utilization efficiency.
When the number of flows increases to 100 by increasing either the num-
ber of source-destination pairs, or the number of flows per pair, the aggregate
transmission of the flows is able to keep the bottleneck link backlogged and
fully utilized, despite the presence of wireless losses. This is clearly exhibited
in Figure 4.2(b),(c). A large N naturally corresponds to a large wired LAN.
A large M can correspond to multiple simultaneous file transfers, or TCP
enhancements that use parallel TCP sockets for better performance over
wireless networks, such as [15]. In this case, our major concern shifts from
link utilization to the backlog dynamics under wireless losses. We note that
the backlog fluctuates severely across time due to the time-varying wireless
losses. Thus, the answer to the BQ for RED is negative.
Now, we explore some feasible directions for reducing the observed back-
log fluctuations. Can TCP be enhanced so as to reduce the backlog fluctua-
tions? Figure 4.3 exhibits the queueing dynamics when the end systems are
running TCP-NCL. When there are only 20 flows as shown in Figure 4.3(a),
we note that TCP-NCL has improved the link utilization over TCP NewReno.
The link is backlogged from the 800th second to the 1300th second, and the
aggregate connection goodput is measured to be around 14 Mbps, or 93% of
the link capacity.
When the number of flows increases to 100 as shown in Figure 4.3(b),(c),
the link is always backlogged. Again, we note that the backlog fluctuates
severely, despite the increased accuracy of end systems in differentiating be-
tween congestive and wireless loss. This is because it is nearly impossible
76
to fully differentiate between congestive and wireless losses in TCP due to
the limited network information available to end systems. Thus, the backlog
fluctuation cannot be reduced significantly simply by enhancing TCP only.
The problem should be tackled by enhancing AQM directly.
We now take a closer look at the cause of the backlog fluctuations under
time-varying wireless losses. In another set of conducted simulations under
time-invariant wireless losses, we have observed that different WPER essen-
tially cause the backlog to converge to different levels. Thus, time-varying
wireless losses effectively force the backlog to switch between different equi-
librium levels from time to time. Moreover, the sluggish transient response
of RED in attaining such switching induces further backlog fluctuations.
Therefore, the key to maintaining a small, stable backlog under wireless
losses is for AQM to direct the backlog to track a fixed reference level de-
spite the variations in the WPER. We now present a systematic approach to
achieve this goal.
4.3 System Model
In this section, we present our system model. We start by constructing a
non-linear system of TCP/AQM. We then linearize the system around its
equilibrium and develop its frequency domain representations. The latter
will serve as the basis for further analysis in later sections.
We begin by considering a generic communication network with a set of
TCP flows F and a set of bottleneck links L.4 We confine our discussion to
4A non-bottleneck link delays packets for almost a constant amount of time and canthus be modelled as part of the propagation delay experienced by a traffic flow traversing
77
C(s) P(s)++
−
wp
bp ~~+r=0
Figure 4.4: Control block diagram of the linearized congestion control system.
the case of unique static routing that a Flow f traverses one identified path
Lf ⊆ L.
The non-linear system of TCP/AQM is a time-delay system. There are
time lags between when a packet is dropped over Link l, and when the source
of Flow f detects the loss, and between when the source of Flow f injects data
into the network, and when Link l receives the data. We define the former
as backward delay,←−dfl(t), and the latter as forward delay,
−→dfl(t). Denoting
the round-trip time of Flow F as df(t), it can be established that [50]:
−→dfl(t) +
←−dfl(t) = df(t) (4.1)
We define the routing matrix incorporated with equilibrium forward de-
lays as:
[RF (s)]lf ,
e−s−→dfl Flow f traverses Link l
0 otherwise(4.2)
and further denote R , RF (0).
For each TCP Flow f , the dynamics of its congestion window Wf(t)
evolves in accordance with the aggregate packet dropping probability qf (t).
that link.
78
Denote the transmission rate of Flow f as xf(t). By [50], we have:
Wf(t) = xf (t− df)(1− qf(t))1
Wf (t)
−xf (t− df)qf(t)Wf (t)
2(4.3)
In wireless networks, qf (t) is determined as:
qf (t) ≈∑
l∈L
Rlf(pl(t−←−dfl) + pwl(t−
←−dfl)) (4.4)
where pl(t) and pwl(t) are the packet dropping probabilities over Link l due
to AQM and wireless losses, respectively. We note that a packet needs to be
stored in the buffer managed by AQM, before being transmitted over the link
subject to wireless losses. Thus, the delay between packet transmission and
packet drop due to wireless losses is slightly greater than the delay between
packet transmission and packet drop due to AQM. Nevertheless, we assume
the difference to be negligible for simplicity.
For each link l, the instantaneous queue size bl(t) is determined by the
aggregate incoming data rate yl(t) and the link capacity cl, following the
system dynamics:
bl(t) = yl(t)− cl (4.5)
yl(t) is the sum of the time-delayed source rates of all flows traversing Link
79
l:
yl(t) =∑
f∈F
Rlfxf (t−−→dfl) ≈
∑
f∈F
Rlf
Wf(t−−→dfl)
df(t)(4.6)
The approximation above is made based on the assumption that df(t), which
is determined by the collected transmission of many flows, varies in a larger
time scale than xf (t).
Besides data transmission and packet dropping, the dynamics of flows
and links are also coupled via the RTTs of the flows. RTT is the sum of
queueing delays at links and round-trip propagation delay, τf :
df(t) = τf +∑
l
Rlf
bl(t)
cl
(4.7)
Denote the Laplace transforms of (pl(t) : l ∈ L), (pwl(t) : l ∈ L), and
(bl(t) : l ∈ L) as p(s), pw(s), and b(s), respectively. Following a similar
procedure as [27], we linearize the non-linear system specified in (4.3), (4.4),
(4.5), and (4.6) around its equilibrium attained with pw(t) ≡ 0, and apply
the Laplace transform to the resulting linear system. Define a plant transfer
function matrix, P(s), as the transfer function from (p(s) + pw(s)) to b(s),
we obtain:
P(s) = −(
sI + R diag(Wf
d2f
)RT diag(1
cl
)
)−1
·RF (s) diag
e−sdf
(s +qfWf
df)qfd2
f
RF (−s)T
(4.8)
80
P(s) represents the interaction of the TCP dynamics, queueing dynamics,
and routing. The stability of this open-loop system can be established by
the following lemma.
Lemma 4.1 The system specified in (4.8) is stable if R is of full row rank.
Proof It suffices to show that all eigenvalues of Ω , R diag(Wf
d2f
)RTdiag( 1cl)
are negative.
Since eigenvalues are invariant under matrix commutation,
λ(Ω) = λ(diag 1
cl
R diag(Wf
d2f
)RT ) (4.9)
By definition, there exists a unit vector v such that:
λ(Ω)v = diag 1
cl
R diag(Wf
d2f
)RTv (4.10)
or
λ(Ω) =vTR diag(
Wf
d2f
)RTv
vT diagclv
=‖diag(
√Wf
df)RTv‖22
‖diag√clv‖22≥ 0 (4.11)
The equality holds only when RTv = 0, which cannot be attained since RT
is full column rank and v 6= 0.
It is time to include AQM into the model. In the time domain, AQM
determines p(t) based on b(t). In the frequency domain, it can be represented
by a transfer function matrix of dimension |L| × |L|, C(s), from the queue
81
size deviation b(s) to the deviation of packet dropping rate by AQM p(s).
The control loop is therefore closed, as shown in Figure 4.4.
The stability of this closed-loop system can be established by the following
lemma.
Lemma 4.2 The system in Figure 4.4 is stable if det(I+P(s)C(s)) contains
zeros on the open left half plane (OLHP) only.
Proof The closed-loop transfer function from the reference signal, r, to b is
given by:
Grb(s) = (I + P(s)C(s))−1P(s)C(s) (4.12)
This is stable if the set of poles for Grb(s), which is a subset of the zeros
of det(I + P(s)C(s)), are on OLHP.
Finally, the role of wireless losses is represented by pw in Figure 4.4. To-
gether with p(s), pw(s) determines the system output b(s) via P(s). How-
ever, while p(s) is in turn determined by b(s) via C(s) and thus part of
the feedback control loop, pw(s) is not in the loop, but serves as an external
disturbance to the system.
4.4 Robust AQM based on Internal Model
Principle
In this section, we develop an approach to track the backlog to a reference
level despite the presence of wireless losses, based on the frequency domain
82
representation of the TCP/AQM system under wireless losses as shown in
Figure 4.4. Per our discussion in Section 4.2, the objective is to maintain a
stable, small backlog under wireless losses.
In the time domain, the prescribed objective is met if b(t), the backlog
deviation from the equilibrium level, converges asymptotically to zero. This
means that b(s) does not contain any unstable poles in the frequency domain.
We begin by noting the following relationship between b and pw:
b(s) = (I + P(s)C(s))−1P(s)pw(s) (4.13)
On the other hand, without loss of generality, we can express the distur-
bance of wireless losses, pw, as:
pw(s) =1
φ(s)k(s) (4.14)
where φ(s) ∈ F contains all the unstable poles of pw and k(s) , φ(s)pw(s).
F denotes the set of the rational functions in s. Obviously, all entries in
k(s) will not contain any unstable poles. The unstable poles determine the
asymptotic behavior of pw(s). Thus, 1φ(s)
is often referred to as a model of
the disturbance.
It follows that:
b(s) =1
φ(s)(I + P(s)C(s))−1P(s)k(s) (4.15)
When the system in Figure 4.4 is stable, the unstable poles of b(s), if any,
come from the unstable poles contained in φ(s). To attain disturbance rejec-
83
tion of pw, or to prevent the unstable poles contained in φ(s) from becoming
poles of b(s), the internal model principle proposes to insert the model of
disturbance 1φ(s)
inside the feedback loop [14].
The design of AQM determines C(s), whereas the plant transfer function
P(s) is invariant across different AQM algorithms. Therefore, we put 1φ(s)
inside the feedback loop by selecting C(s) as:
C(s) =1
φ(s)ΛN(s)ΛD(s)−1 (4.16)
where ΛN(s), ΛD(s) ∈ F|L|·|L| are coprime diagonal matrices. det(ΛN(s)) does
not share roots with φ(s), and det(ΛD(s)) does not have any roots on closed
right half plane. We constrain C(s) to be diagonal so that each intermediate
router can execute AQM algorithm distributedly without incurring the need
of exchanging the instantaneous queue sizes with each other.
Theorem 4.1 Consider the system in Figure 4.4 with P(s) and C(s) as
specified in (4.8) and (4.16), respectively. Suppose the conditions of Lem-
mas 4.1 and 4.2 hold so that both open-loop and closed-loop stabilities are
established. Let P(s) = D−1(s)N(s) be a left coprime factorization. The
system attains disturbance rejection of (4.14) if det(N(s)) does not contain
zeros at the roots of φ(s).
Proof Substituting (4.16) into (4.15) gives:
b(s) = ΛD(s)(φ(s)D(s)ΛD(s) + N(s)ΛN(s))−1N(s)k(s) (4.17)
Disturbance rejection is attained if none of the zeros of det(φ(s)D(s)ΛD(s)+
84
N(s)ΛN(s)), which include the poles of b(s) as a subset, are on closed right
half plane (CRHP). For the roots of φ(s) (which are unstable poles of pw(s)
and thus on CRHP), the determinant reduces to:
det(N(s)ΛN(s)) = det(N(s)) det(ΛN(s)) (4.18)
This is non-zero since neither det(ΛN(s)) nor det(ΛN(s)) contains zeros at
the roots of φ(s) by assumption.
For s ∈ s|ℜ(s) ≥ 0 and φ(s) 6= 0:
det(φ(s)D(s)ΛD(s) + N(s)ΛN(s))
= det(D(s)) det(ΛD(s))
· det(φ(s)I + P(s)ΛN(s)ΛD(s)−1)
= φ(s)|L| det(D(s)) det(ΛD(s))
· det(I + P(s)C(s)) (4.19)
We now consider the three determinants in the final expression above one
by one. As R is of full row rank, P(s) does not have any unstable poles and
thus det(D(s)) does not contain any zeros on CRHP. Neither det(ΛD(s)) nor
det(I + P(s)C(s)) contain zeros on CRHP by assumption.
Since CRHP is a subset of s|φ(s) = 0⋃s|ℜ(s) ≥ 0 and φ(s) 6= 0,
we conclude that det(φ(s)I + P(s)ΛN(s)ΛD(s)−1) does not contain zeros on
CRHP.
Remark 4.1 When det(D(s)) does contain zeros at the roots of φ(s), there
will be some cancellations of unstable zero-pole pairs between C(s) and P(s).
85
This violates the condition of total stability, which means that every possible
input-output pair of the system is bounded-input-bounded-output (BIBO) sta-
ble. It is one of the physical constraints that a practical system should meet.
Interested readers can refer to [14] for details.
Remark 4.2 The attained disturbance rejection is robust against pertur-
bance to P(s), ΛN(s), and ΛD(s), as long as the mild assumptions of Theo-
rem 4.1 continue to hold after the perturbance. In practice, TCP AIMD pa-
rameters, routing topologies, and network capacities, may change over time,
leading to such perturbance. The AQM design specified in (4.16) can nonethe-
less maintain robustness against wireless losses.
Remark 4.3 Indeed, the attained disturbance rejection is not robust against
errors in implementing φ(s). Consider when, due to such errors, C(s) be-
comes
C(s) =1
φ(s)ΛN(s)ΛD(s)−1 (4.20)
where φ(s) is an approximation of φ(s). It follows that:
b(s) =
b1(s) ,︷ ︸︸ ︷
ΛD(s)(φ(s)D(s)ΛD(s) + N(s)ΛN(s))−1N(s)k(s)
+
b2(s) ,︷ ︸︸ ︷
ΛD(s)(φ(s)D(s)ΛD(s) + N(s)ΛN(s))−1N(s)k(s)φ(s)− φ(s)
φ(s)
(4.21)
Provided that the assumptions of Theorem 4.1 hold for C(s) specified in
86
(4.20), b1(s) contains stable poles only. b2(s) contains the unstable poles
of φ(s), and represents the steady state errors in tracking the backlog to a
reference size due to the inexact cancellation between φ(s) and φ(s).
Nevertheless, it is still very meaningful to apply the internal model princi-
ple with inexact implementation of φ(s). This reduces the steady state errors
than when the principle is not applied. For example, consider the simple case
that φ(s) = s− a and φ(s) = s− a + ǫ, where a and ǫ represent an unstable
pole and the error involved in implementing φ(s), respectively. b2(s) then
reduces to:
b2(s) = ΛD(s)(φ(s)D(s)ΛD(s) + N(s)ΛN(s))−1N(s)k(s)ǫ
s− a(4.22)
which is proportional to ǫ. Therefore, we can keep the steady state error
minimal with a good approximate implementation of φ(s).
Generally, Theorem 4.1 can be applied in two directions. On the one
hand, when φ(s) is known5 , we construct an appropriate C(s) to yield a
desired disturbance rejection. For example, wireless losses may vary periodi-
cally at a frequency, say, 1T, resulting in φ(s) = 1−e−Ts. An AQM with C(s)
containing (1 − e−Ts) as the denominator can potentially help to maintain
a stable, small backlog under wireless losses with a periodic time-varying
packet loss rate. This deserves further study in the future.
On the other hand, without any specific knowledge of φ(s), φ(s) can be
designed to serve as a good approximation for the general scenarios, thereby
5We note that φ(s) corresponds to the “pattern” in the variation of wireless losses.Knowledge about φ(s) is a much weaker assumption than knowledge about the exactWPER.
87
attaining good performance in disturbance rejection.
We are going to investigate the design of such a controller in the next
subsection.
4.4.1 IC
Here, we approximate the disturbance pw as a step input function. This
approximation is valid as long as the linear system in consideration converges
much faster than the variations in the WPER. It follows that φ(s) = s. From
(4.16), we can choose the diagonal element of C(s) as:
Cll(s) =gl
s, 1 ≤ l ≤ |L| (4.23)
which is essentially an integral controller (IC). It converts its input into the
integral of that input in the time domain. The integral controller will be part
of any other C(s) satisfying (4.23) and φ(s) = s, and is the essential element
fulfilling the internal model principle. Denote:
RF (·) , diagcl−1RF (·)diagWf
df
(4.24)
ωg , 0.1 minl∈L,f∈F
(
2
dfWf
,1
df
,σ2
min(RF (0)) cl
Wf
)
(4.25)
The following proposition gives the design rules for the integral controller.
Proposition 4.1 Consider the system in Figure 4.4 with P(s) and C(s)
as specified in (4.8) and (4.23), respectively. Suppose that R is of full row
88
rank. Let gl be chosen as g
cl. The system is closed-loop stable and attains
disturbance rejection of pw as specified in (4.14) with φ(s) = s if:
g ≤ ωg ·4 σ2
min(RF (0))
‖RF (0)‖11
(maxf∈F df Wf ) maxf∈F Wf
(4.26)
In the case of homogeneous flows sharing a bottleneck link, if maxf Wf ≥ 2,
a simplified condition can be obtained as:
g ≤ 0.8(N−)3
d5+c3
(4.27)
where N−, d+, and c correspond to the lower bound for the number of flows,
the upper bound for RTT, and the link capacity, respectively.
Proof By the generalized Nyquist criteria [18], it suffices to prove6 that the
eigenloci of the return ratio (−P (jω)C(jω)) does not encircle (−1, 0) on the
complex plane for ω 6= 0. By (4.24), We can rewrite (−P(jω)C(jω)) as:
−P(jω)C(jω)
=1
ω(jωdiag(
1
cl
) + RF (0)diag(1
Wf
)RF (0)T )−1RF (jω)
·diag
exp(j(−π
2− ωdf))
2(jω +qfWf
df)
RF (−jω)T diag(Ilcl)
(4.28)
If λ(jω) is an eigenvalue of P(jw)C(jw), there exists a normalized vector
6A complete proof should also cover the eigenloci of −P (δe−jω)C(δe−jω) for all ω ∈[−0.5π, 0.5π] as δ → 0. Yet, the proof is trivial.
89
v such that:
λ(jω)
=
v∗RF (jω)diag
(
j(−π2−ωdf ))
2(jω+qf Wf
df)
)
RF (−jω)Tv
ω
g(jωv∗diag(
1
cl
)v + v∗RF (0)diag(1
Wf
)RF (0)Tv)
(4.29)
Note that:
σ2max(RF (−jω)) ≤ ‖RF (jω)RF (−jω)T‖1
≤ ‖RF (jω)‖1‖RF (−jω)T‖1
= ‖RF (0)‖1 (4.30)
For ω ∈ [0, ωg], we obtain:
λ(jω)
≈v∗RF (jω)diag
(−jdf
2qfWf
)
RF (−jω)Tv
ω
gv∗RF (0)diag(
1
Wf
)RF (0)Tv
(4.31)
With the crossover frequency bounded below by ωg and |λ(jω)| decreasing
with ω, it suffices to have |λ(jωg)| < 1, which follows from (4.26).
In the case of N homogeneous flows sharing a bottleneck link of capacity
90
c, RF (0)RF (0)T = 1N
. Thus,
ωg = 0.1 minl∈L,f∈F
(2
dfWf
,1
df
,c
NWf
)
=0.2
max dfWf
(4.32)
and (4.26) reduces to
g ≤ ωg ·4
(maxf∈F df Wf ) maxf∈F Wf
=0.8
(maxf∈F df Wf )2 maxf∈F Wf
=0.8N3
c3d5f
(4.33)
The final line above follows since all the flows are homogeneous.
The RHS of (4.33) is lower bounded by 0.8(N−
)3
d5+
c3and therefore (4.27) serves
as a more stringent lower bound.
Remark 4.4 In [26], the proportional-integral (PI) controller is designed in
the form of kl(s+zl)s
, which however cannot reduce to the integral controller as
specified in (4.23).
After discretization via the bilinear transform for an IC, the packet drop-
ping rate for each link l can computed as:
pl[k] = pl[k − 1] + Gl(bl[k]− bl) (4.34)
where Gl = gl
fl. fl is the sampling frequency, i.e., the frequency at which
the queue size is sampled and (4.34) is used to update the packet dropping
rate. pl[k] and bl[k] are the step-wise packet dropping rate and queue size,
91
R3
... ...
R1
...
S3K
S32
S31
50msR2
...
S11 S12 S1N S21 S22
D11 D1ND12 D21 D22
50ms
...
...
D31
D32
D3K
S2H
D2H
12.8 Mbps 9.6 Mbps
Figure 4.5: Two wireless bottleneck links.
respectively. The computation can be realized as an independent module in
AQM, taking the queue length of a buffer as input and computing the packet
dropping rate due to the IC as the output.
Example 4.1 Consider the scenario illustrated in Figure 4.1, with the wire-
less link of capacity 15 Mbps and propagation delay 50 ms. A data packet
is 1000 bytes long, and thus cl = 1875 packets/second. A total of N = 100
homogeneous TCP flows share the link. Suppose that the equilibrium queue
size is bl = 220 packets. It follows that df is around 0.2 seconds.
Applying (4.27) using d+ = 0.2 seconds, N− = N = 100, c = cl =
1875 Packets,
g ≤ 0.8 · 1003
0.25 · 18753≈ 0.3792
Example 4.2 Consider another network scenario illustrated in Figure 4.5,
which can arise when three wired LANs are inter-connected to create a WAN
via wireless links. Three sets of flows F1, F2, and F3 share a network. There
are N1 = 60, N2 = 50, and N3 = 20 flows in F1, F2, and F3, respectively.
Flows within the same set are homogeneous. Each of those flows in Fi (i =
1, 2, 3) traverses from Sij to Dij where 1 ≤ j ≤ 60 for i = 1, 1 ≤ j ≤ 50 for
92
i = 2, and 1 ≤ j ≤ 20 for i = 3. We thus have two wireless bottleneck links,
whose dynamics are coupled via flows in F3. Suppose that the equilibrium
queue sizes of both wireless links are b1 = b2 = 200 packets. Let di and Wi be
the equilibrium RTT and window size for each of the flows in Fi, respectively.
It follows that:
c = (1600, 1200) Packets/Second
d = (0.18, 0.22, 0.35)T Seconds
W = (4.2, 4.5, 3.2)T Packets
and thus ‖RF (0)‖1 = 0.017.
Now, the matrix RF (0)RF (0)T becomes:
RF (0)RF (0)T ≈
0.014 0.00065
0.00065 0.015
(4.35)
which gives σ2min(RF (0)) = 0.0136.
By (4.25) and (4.26),
ωg ≈ 0.1 min(2
0.35 · 3.2 ,1
0.35,0.0136 · 1200
4.5) = 0.1786
g ≤ 0.1786 · 4 · 0.0136
0.017· 1
(0.35 · 3.2) · 4.5 ≈ 0.1134
93
4.5 Performance Evaluation
In this section, we evaluate the performance of IC and compare it with RED
using ns-2 simulation. This is necessitated by that the robustness and stabil-
ity of the IC have been established in the context of the linearized congestion
control system, which is nonlinear in reality. Sections 4.5.1 and 4.5.2 present
results obtained in simulation topologies illustrated in Figures 4.1 and 4.5,
respectively.
For IC, the packet dropping rate is updated at a frequency of 10 HZ. Nev-
ertheless, we have repeated the experiments with a different implementation
of IC, which updates packet dropping rate at every packet arrival as RED
does, with similar results reported.
The parameters of IC and RED follow Proposition 4.1 and references [26,
27], respectively. For RED, pMax and the queue weight are set to 0.1 and
2.6 × 10−6, respectively. The parameters minThresh and maxThresh of
RED, and the gain of IC, Gl, vary and will be specified in the following.
4.5.1 One Wireless Bottleneck Link
In the topology illustrated in Figure 4.1, a total of N pairs of source and
destination are connected via a wireless bottleneck link with capacity c Mbps,
and propagation delay τ ms. Within each pair (Si, Di), M long-lived TCP
Reno flows are sent from Si to Di. The size of the link buffer and the reference
queue size of IC are set to 800 Packets and 220 Packets, respectively. We
introduce variable WPER into the bottleneck link. Time is divided into
periods, each of exponential distribution, with a mean of lossP seconds. At
94
the beginning of each period, the WPER of the bottleneck is determined
randomly from a uniform distribution over [0, mL].
Figures 4.6-4.9 plot the traces of queue sizes under various configurations
summarized in Table 4.1. The ”HTTP” and ”CBR” columns in Table 4.1
indicates whether there are HTTP and CBR traffic flows coexisting with the
long-lived TCP flows, respectively. The final three columns are parameter
settings of IC and RED and computed from the network settings.
95
Table 4.1: Network configurations for the single wireless bottleneck network.
Figure N M c τ mL lossP HTTP CBR Gl minThresh maxThreshFigure 4.6(a) 100 1 15 50 4% 100 No No 2 · 10−5 150 700Figure 4.6(b) 20 5 15 50 4% 100 No No 2 · 10−5 150 700Figure 4.7(a) 20 5 15 50 8% 100 No No 2 · 10−5 150 700Figure 4.7(b) 20 5 15 50 4% 20 No No 2 · 10−5 150 700Figure 4.8(a) 20 5 15 200 4% 100 No No 2 · 10−7 150 400Figure 4.8(b) 20 10 15 200 4% 100 No No 1.6 · 10−6 150 400Figure 4.8(c) 20 10 30 50 4% 100 No No 1 · 10−5 150 400Figure 4.9(a) 20 5 15 50 4% 100 Yes No 2 · 10−5 150 700Figure 4.9(b) 20 5 15 50 4% 100 No Yes 2 · 10−5 150 700Figure 4.9(c) 20 5 15 50 4% 100 Yes Yes 2 · 10−5 150 700
96
0
100
200
300
400
500
600
700
800
0 200 400 600 800 1000 1200 1400
Que
ue le
ngth
(nu
mbe
r of
pac
kets
)
Time (second)
REDIC
(a) N = 100, M = 1.
0
100
200
300
400
500
600
700
800
0 200 400 600 800 1000 1200 1400
Que
ue le
ngth
(nu
mbe
r of
pac
kets
)
Time (second)
REDIC
(b) N = 20, M = 5.
Figure 4.6: Comparison of queue length dynamics with same configurationsas in Figure 4.2.
Figure 4.6(a) and Figure 4.6(b) essentially retain the same configurations
as Figure 4.2 and Figure 4.2(b), respectively. We observe that IC helps main-
tain the queue size stable throughout the simulation period. Despite the vari-
ations in WPER, the equilibrium queue size of IC tracks the reference level
(220 Packets) very well. We do observe occasional “spikes” in the queueing
dynamics with IC. This is because IC needs to converge to a new equilibrium
packet dropping rate to maintain the same equilibrium queue size when the
97
WPER changes. Nevertheless, the transition is much smoother than that of
RED since the equilibrium queue size is unchanged and IC has better tran-
sient response than RED. Moreover, while the difference between the two
equilibrium packet dropping rate should be approximately the change in the
WPER, IC is self-adaptive and does not require any knowledge about the
WPER to complete the transition. We note that this is an essential prop-
erty of wireless AQM enhancements developed based on the internal model
principle.
In addition, Figure 4.6(a) and Figure 4.6(b) exhibit similar queueing dy-
namics. Generally, in all our conducted simulation experiments, the queueing
dynamics are quite similar across different values of M for 1 ≤ M ≤ 10, as
long as the total number of flows, NM , is kept constant.
Figure 4.7(a) and Figure 4.7(b) exhibit the queueing dynamics against
wireless losses with larger variations and more frequent variations in WPER,
respectively. Correspondingly, RED exhibit more severe and more frequent
fluctuations, respectively. IC basically maintains a stable queue size around
the reference level in both cases, except that occasional spikes due to transi-
tion becomes larger in Figure 4.7(a). This is as expected, since more abrupt
“jumps” in the WPER requires IC to undergo a greater change in the equi-
librium packet dropping rate.
Figure 4.8(a)-(c) exhibit the queueing dynamics with “fatter” pipes, i.e.,
a bottleneck link of larger propagation delay or higher bandwidth. Fig-
ure 4.8(a) increases the propagation delay to 200 ms, approximately the
two-way propagation delay with a geostationary satellite [3], and maintains
the total number of flows to be 100. We observe that queue dynamics tend
98
0
100
200
300
400
500
600
700
800
0 200 400 600 800 1000 1200 1400
Que
ue le
ngth
(nu
mbe
r of
pac
kets
)
Time (second)
REDIC
(a) Larger variations in WPER.
0
100
200
300
400
500
600
700
800
0 200 400 600 800 1000 1200 1400
Que
ue le
ngth
(nu
mbe
r of
pac
kets
)
Time (second)
REDIC
(b) More frequent variations in WPER.
Figure 4.7: Comparison of queue length dynamics under different wirelesslosses.
99
0
100
200
300
400
500
600
700
800
0 200 400 600 800 1000 1200 1400
Que
ue le
ngth
(nu
mbe
r of
pac
kets
)
Time (second)
REDIC
(a) Fatter pipe due to increased link delay.
0
100
200
300
400
500
600
700
800
0 200 400 600 800 1000 1200 1400
Que
ue le
ngth
(nu
mbe
r of
pac
kets
)
Time (second)
REDIC
(b) Fatter pipe due to increased link delay is sharedby more TCP flows.
0
100
200
300
400
500
600
700
800
0 200 400 600 800 1000 1200 1400
Que
ue le
ngth
(nu
mbe
r of
pac
kets
)
Time (second)
REDIC
(c) Fatter pipe due to increased bandwidth is sharedby more TCP flows.
Figure 4.8: Comparison of queue length dynamics with different bottlenecklink parameters.
100
to fluctuate very severely with both IC and RED. In fact, the link is not
backlogged for significant portion of time.
The congestion window of TCP flows regulate the number of packets
in transit. It grows larger with fatter pipes, since more packets in transit
are needed to keep the pipe full. When a packet is lost due to wireless
losses, the corresponding flow will spuriously reduce its window by half. With
fatter pipes, the window reduction in terms of packets becomes larger. This
leads to more severe fluctuations in the number of packets in transit, which
will possibly fall below the size of the pipe (≈ 2cτ) and leave the link not
backlogged. We recall that, when a link is not backlogged, it is outside the
operational region of wireless AQM enhancements.
To mitigate the effect of fatter pipes, we can increase the number of
TCP flows so that the congestion window of a single flow becomes smaller.
Figure 4.8(b) plots the queueing dynamics when the total number of flows
is increased to 200. In this case, we observe that the link is backlogged
throughout the simulation, and IC can effectively maintain a stable queue
size.
Similarly, Figure 4.8(c) plots the queueing dynamics when 200 flows are
sharing a bottleneck link of doubled bandwidth, in which IC still demonstrate
a robust tracking of the reference queue size. The backlog fluctuation with
RED is not very significant in this case due to the reduction in maxThresh as
computed. maxThresh is the maximum level of queue size allowed by RED.
However, such reduction is generally not recommended in other scenarios due
to the stability requirement of RED [27].
Figure 4.9(a)-(c) exhibit the queueing dynamics with the presence of other
101
0
100
200
300
400
500
600
700
800
0 500 1000 1500 2000 2500 3000
Que
ue le
ngth
(nu
mbe
r of
pac
kets
)
Time (second)
REDIC
(a) HTTP.
0
100
200
300
400
500
600
700
800
0 500 1000 1500 2000 2500 3000
Que
ue le
ngth
(nu
mbe
r of
pac
kets
)
Time (second)
REDIC
(b) CBR.
0
100
200
300
400
500
600
700
800
0 500 1000 1500 2000 2500 3000
Que
ue le
ngth
(nu
mbe
r of
pac
kets
)
Time (second)
REDIC
(c) HTTP and CBR.
Figure 4.9: Comparison of queue length dynamics with the presence of othertraffic.
102
traffic. Figure 4.9(a) plots the queueing dynamics when we introduce the
HTTP traffic into the dumbbell topology. In doing so, we attach 120 web
client nodes to R1 and ten web server nodes to R2. We can see that the
traces of the queue size are more “noisy”, since the introduction of the HTTP
traffic leads to a higher fluctuation on the network load. Nevertheless, IC
demonstrates an effective and robust tracking of the reference queue level
against the disruptions from both perturbing traffic and wireless losses.
Figure 4.9(b) plots the queueing dynamics when we introduce CBR traf-
fics into the dumbbell topology. In doing so, we add attach 30 UDP sources
to R1 to and their corresponding destinations to R2, with each UDP flow
driven by long-lived CBR at the constant rate of 0.1 Mbps. The UDP flows
represent unresponsive flows that do not back off upon network congestion.
With RED, the backlog generally shifts up due to the increased long-lived
traffic loading. On the other hand, the robust tracking by IC of the reference
queue level maintains despite the presence of unresponsive flows.
Figure 4.9(c) plots the queueing dynamics when both HTTP and CBR
traffic are present. We can identify the aforementioned impact of introducing
HTTP and CBR traffic. With RED, the backlog trace becomes more “noisy”
and shifts up as well. With IC, the backlog becomes more “noisy” but still
tracks the reference level.
4.5.2 Two Wireless Bottleneck Links
In the topology illustrated in Figure 4.5, network configuration follow those
in Example 4.2. The size of the two link buffers are both 600 packets. For
103
0
100
200
300
400
500
600
700
800
0 500 1000 1500 2000 2500 3000
Que
ue le
ngth
(nu
mbe
r of
pac
kets
)
Time (second)
REDIC
(a) Link 1.
0
100
200
300
400
500
600
700
800
0 500 1000 1500 2000 2500 3000
Que
ue le
ngth
(nu
mbe
r of
pac
kets
)
Time (second)
REDIC
(b) Link 2.
Figure 4.10: Comparison of queue length dynamics in the topology of twowireless bottleneck links.
104
both links, minThresh and maxThresh of RED are set to 150 and 500 packets,
respectively. Gl’s are set to be 7 · 10−6 and 9.5 · 10−6 for Link 1 and Link
2, respectively. We introduce time-varying losses into both links, with an
average loss period of 100 s and a maximum WPER of 3 %. The simulation
results are plotted in Figure 4.10. Again, severe fluctuations can be observed
in the queue sizes with RED over both Link 1 and Link 2. On the other
hand, the IC is successful in maintaining stable queue sizes throughout the
simulation.
4.6 Concluding Remarks
In this chapter, centered around the BQ “Can AQM maintain a small, stable
queue under wireless losses?”, we carry out a series of study. We find that
RED, one of the most representative AQM, fail to maintain a stable queue
size against time-varying wireless losses. The key to solving the problem is for
the AQM to force the queue size to track a reference queue level despite the
presence of wireless losses. Based on the frequency domain representation of
TCP/AQM, we apply the internal model principle to reject the disturbance
of wireless losses, thereby attaining such tracking. We devise the integral
controller (IC) as an embodiment of the internal model principle, and develop
a set of design rules for IC. We have conducted a series of experiments to
examine the performance of IC and compare it with RED. In particular, we
have evaluated the impact of severe variations of wireless losses, fat pipes,
and HTTP traffics on the two AQM algorithms. In general, IC demonstrates
very robust performance in maintaining a small, stable backlog under wireless
105
losses.
There are several possible extensions to our work, including:
1. developing a global stability analysis for the system of TCP/buffer-
based AQM perturbed by wireless losses;
2. applying Theorem 4.1 to design an AQM that attains clean disturbance
rejection of wireless losses with known φ(s), as discussed in Section 4.4,
and;
3. developing a unified AQM solution for resolving both fairness and sta-
bility issues in ad-hoc wireless networks.
106
Chapter 5
Conclusions and Future
Research
5.1 Chapter Overview
In this chapter, we conclude the thesis and discusses some future research
directions. Section 5.2 discusses main results acquired and summarizes the
major contributions of the thesis. Section 5.3 discusses some topics worth
further investigation. Section 5.4 concludes the chapter with a discussion.
5.2 Contributions
In Chapter 2, we conduct a literature survey of the work performed to date on
wireless TCP enhancements and on design and analysis of AQM. We find that
there is a lack of transport layer unified solution for enhancing TCP against
packet reordering and non-congestive loss. We identify that the performance
107
of AQM under wireless losses is largely unexplored in the literature.
In Chapter 3, we have developed a ideal TCP sender model, known as
smart TCP sender (STS), for constructing more reliable signals of packet loss
and network congestion over general, error-prone channels. The two signals
employed are the expirations of two serialized timers, where the second timer
is started upon the expiration of the first timer. The timer expiration periods
have been optimized for minimizing quantified risks associated with spurious
congestion response and excessively delayed congestion response.
We have devised a novel TCP variant, known as TCP for non-congestive
loss (TCP-NCL), as a practical approximation of the STS model. TCP-NCL
can thus serve as a unified solution for effective congestion control, sequenc-
ing control, and loss recovery over wireless networks. The deployment of
TCP-NCL requires modification to sender-side TCP only, thereby facilitat-
ing possible future wide deployment. The deployment of TCP-NCL requires
maintenance of a few unused bits in the existing data structure for TCP and
a few extra variables only. The maintenance is performed upon arrival of
acknowledgement packet.
We have performed extensive simulation experiments to examine the per-
formance of TCP-NCL and compare it with several well-known TCP variants
in the literature using Network Simulation (ns) 2.29. Our results report that
TCP-NCL attains significant performance improvement over general, error-
prone communication networks, thereby demonstrating robustness against
packet reordering and non-congestive packet loss. At the same time, multiple
competing TCP connections, with some employing TCP-NCL and the others
employing standardized TCP design, are capable of sharing bandwidth fairly
108
over conventional wired networks. TCP-NCL is therefore responsive against
congestive loss, and backward compatible with the present TCP Internet.
In Chapter 4, we raise the basic question: Can AQM maintain a sta-
ble, small backlog under wireless losses? Our simulation results report that,
while many TCP flows can keep a wireless link backlogged despite spurious
backoffs, RED fails to maintain a stable backlog under time-varying wireless
losses. Due to the fundamentally limited information on network loading
available to end systems, the backlog stabilization should be best attained
via AQM enhancements. We further identify that the key to resolving the
problem is to robustly track the backlog to a preset reference level.
Based on a linear model of TCP/AQM, we apply the internal model
principle to realize such tracking. The application of internal model principle
in combating wireless losses enables AQM to adjust the packet dropping rate
to compensate for the packet drops due to wireless losses without any specific
knowledge about the wireless packet error rate (WPER). The principle can be
realized as an independent module in AQM, and thus offers good extensibility
to incorporate other AQM enhancements.
By assuming the intensity of wireless losses to be slowly time-varying,
we devise the integral controller (IC) as an embodiment of internal model
principle. We have developed design rules for IC. Our simulation results show
that IC is capable of maintaining stable queue sizes and performing effective
congestion avoidance against wireless losses. Its performance is maintained
in the presence of perturbing traffic flows (e.g. HTTP traffic).
The major contributions of the thesis are recapped as follows:
1. A TCP sender model for constructing reliable signals of packet loss and
109
network congestion over general, error-prone communication networks
Based on the novel serialized-timer structure, the STS model constructs
reliable signals of packet loss and network congestion. It attains an
optimal tradeoff between signal reliability and responsiveness against
congestion by minimizing quantified risks associated with spurious con-
gestion response and excessively delayed congestion response.
2. A unified solution for effective congestion control, sequencing control,
and loss recovery over wireless networks
As a transport layer solution, TCP-NCL enables accurate differentia-
tion among the occurrences of packet reordering, congestive loss, and
non-congestive loss. It significantly enhances robustness against packet
reordering and non-congestive loss while maintaining responsiveness
against congestive loss.
3. A simulation-based investigation of the performance of AQM under
wireless losses
The series of simulation experiments on the performance of RED under
time-varying losses reveal the irrobustness of RED. It helps to clarify
the different operational regions of TCP and AQM enhancements. Due
to the fundamentally limited information on network loading available
to end systems, the backlog stabilization should be best attained via
AQM enhancements. The results further indicate that the key to main-
taining a stable backlog is to robustly track the backlog to a preset
reference level.
110
4. A systematic AQM enhancement for backlog stabilization under wireless
losses
The application of internal model principle enables AQM to adjust the
packet dropping rate to compensate for wireless losses without requiring
exact knowledge of the WPER, thereby maintaining a stable backlog
under time-varying wireless losses. This significantly improves end-to-
end delay performance and fairness towards bursty traffic.
5.3 Future Research
Our major results, as summarized in Section 5.2, demonstrate that the In-
ternet congestion control fulfilled by TCP and AQM can be systematically
adapted to the wireless environment at a minimal deployment cost and dis-
ruptions, preserving the loss-based nature. However, there remain several
research directions worth thorough investigation, such as those enlisted at
the end of Chapters 3 and 4. We discuss two topics in greater depth in the
following.
5.3.1 A Theoretical Framework for Wireless TCP
The literature on wireless TCP enhancements is vast [44]. Various propos-
als generally adapt TCP to the dynamic wireless environment by exploit-
ing readily available or self-constructed signals of the occurrences of various
events. For example, the network loading conditions can be inferred from
RTT, packet losses, or explicit network feedbacks.
111
However, a theoretical framework is lacking in at least two aspects. First,
it would be highly desirable to compare the various signals in terms of quan-
tified metrics and operational regions. For example, RTT provides richer
information about network loading than packet losses, thereby enabling finer
tuning of end system transmission. The merit is particularly useful in alle-
viating the fluctuation of TCP in high-speed networks [71]. On the other
hand, RTT can be varied by non-congestive factors in many scenarios such
as reordering networks, thereby undermining its reliability as compared with
packet loss. A systematic categorization and quantified characterization of
these signals would vastly facilitate understanding about the strength and
limitations of existing TCP variants, and development of new TCP variants
that push towards fundamental limits.
Second, wireless TCP enhancements generally modify TCP at a finer
granularity, or in a more general manner, than have been modeled in the
literature. For example, many TCP variants opt to proactively postpone
packet retransmission and congestion response until congestive loss is prob-
able. In the popular control-theoretic formulation [33], the delay would not
change the converged equilibrium as long as the stability condition is ob-
served. Yet, the simulation findings reveal that it requires extra effort to
maintain TCP-friendliness of such TCP variants [10]. The discrepancy is
partly because the sophisticated impact of RTO remains to be taken into
account. Consequently, the TCP-friendliness and fairness of wireless TCP
enhancements have to be examined via extensive simulation experiments,
which cannot exhaust all possible network scenarios in reality. As another
example, some recent results on a network with heterogeneous congestion
112
control [67, 68], namely a network with multiple congestion control schemes
based on different signals, assume each congestion control scheme utilizes
one type of signal only. However, TCP variants, such as TCP-NCL proposed
in this thesis, tends to exploit information conveyed by the mix of multiple
signals to attain better performance. Therefore, it remains to develop theo-
retical understanding regarding the dynamics and equilibrium of a network
shared by TCP variants. This helps to clarify the set of constraints a TCP
proposal needs to meet to ensure inter-flow fairness.
The aforementioned two aspects are expected to be complementary. The
systematic study of signals available to end systems essentially views the
network as a black box and focuses on optimizing a single TCP flow. This
hopefully will give rise to a generalized expression of TCP dynamics, and
provides a basis for studying the network of TCP variants. A better un-
derstanding of the equilibrium and dynamics of such networks, on the other
hand, provides further performance metrics for our study of the signals.
5.3.2 A Unified AQM Solution for Wireless Networks
Wireless networks bring unique challenges to AQM that are not covered by
the vast literature on the design and analysis of AQM in wired networks.
First, as reported in Chapter 4, different WPERs can cause the backlog to
converge to different levels, and time-varying WPER can induce fluctuations
in backlog. To the best of our knowledge, our work presented in Chapter 4
is the first investigation on whether and how AQM can maintain a stable
backlog under wireless losses.
113
Second, in wireless ad-hoc networks, transmission at adjacent nodes may
interfere with each other. The implication on congestion control is two-
fold. On the one hand, TCP flows compete for “space”, namely the chance
for transmission within an interfering range, in addition to competing for
wireless bandwidth. Thus, backlogs at interfering nodes need to be aggre-
gately managed by a queue management algorithm to facilitate fair allocation
among competing flows [76]. On the other hand, packet losses occur due to
channel contention, in addition to router buffer overflow, and both should be
interpreted as signals of network congestion. However, unlike losses due to
buffer overflow, contention losses may saturate when the offered network load
exceeds a threshold value [25]. Consequently, the TCP flows tend to push the
network beyond the optimal loaded point, leading to degraded throughput.
AQM should thus be in place to deliberately tune up the packet dropping
rate to force earlier backoff of the TCP flows.
A unified AQM solution for resolving issues induced by wireless losses,
channel contention and other special characteristics of wireless networks is
much more than a trivial combination of the solutions proposed for each issue.
In particular, to stabilize the backlog under wireless losses we need to tune
down packet dropping rate to various levels to compensate for the variation
in wireless packet error rate, whereas to force TCP flows to operate at the
optimal loading point in ad-hoc networks we need to tune up packet dropping
rate to force earlier backoffs in TCP. A possible approach to attain both
objectives is to make use of some occurrences of wireless losses to deliberately
induce backoffs in TCP to restore the optimal loading point, and at the same
time to adjust the packet dropping rate of AQM to compensate for variations
114
of wireless losses. Again, the exact knowledge of wireless packet error rate
should not be assumed, or the adaptiveness to the ever-changing wireless
environment would be sacrificed. The approach can thus be best fulfilled on
the basis of a valid model of TCP/AQM under wireless losses and channel
contentions, which itself is a challenging task to be accomplished.
5.4 Concluding Remarks
In this thesis, we aim to systematically adapt the present Internet conges-
tion control to the wireless environment at a minimal deployment cost and
disruption. Our adaptations include TCP and AQM enhancements, which
jointly help the heterogeneous wired/wireless Internet to operate in the op-
timal region of low delay and high throughput.
On the TCP side, we have developed the STS model that uses a serialized-
timer structure to offer more reliable signals of packet loss and network con-
gestion over general, error-prone channels. We have devised TCP-NCL as a
practical approximation of STS, which can serve as a unified solution for ef-
fective congestion control, sequencing control, and loss recovery over wireless
networks. The deployment of TCP-NCL requires sender-side modification
only.
On the AQM side, we have investigated the performance of AQM under
wireless losses using simulation experiments. Our results report that RED fail
to maintain a stable backlog against time-varying WPER. We have applied
the internal model principle to stabilize the backlog. We further devise IC
as an embodiment of the principle, which performs robustly against time-
115
varying WPER in various simulated challenging scenarios.
There are several research directions in wireless Internet congestion con-
trol worth future thorough investigation, for which the work completed in
this thesis provide a solid basis. These include a theoretical framework for
the design and analysis of wireless TCP enhancements, and a unified AQM
solution for wireless networks.
116
Bibliography
[1] I.F. Akyidiz, C. Morabito, and S. Palazzo. TCP-Peach: A New Conges-tion Control Scheme for Satellite IP Networks. IEEE/ACM Transactionson Networking, Vol. 9, No. 3, Jun. 2001.
[2] M. Allman, H. Balakrishnan, and S. Floyd. Enhancing TCP’s Loss Re-covery using Limited Transmit. IETF RFC 3042, Jan. 2001.
[3] M. Allman, D. Glover, and L. Sanchez. Enhancing TCP Over SatelliteChannels using Standard Mechanisms IETF RFC 2488, Jan. 1999.
[4] M. Allman, V. Paxson, and E. Blanton. TCP Congestion Control. IETFRFC 5681, Sep. 2009.
[5] M. Allman, V. Paxson, and W. Stevens. TCP Congestion Control. IETFRFC 2581, Apr. 1999.
[6] S. Athuraliya, S.H. Low, V.H. Li, and Q. Yin. REM: Active QueueManagement. IEEE Network, Vol. 15, No. 3, pp. 48-53, May 2001.
[7] H. Balakrishnan, V.N. Padmanabhan, S. Seshan, and R.H. Katz. AComparison of Mechanisms for Improving TCP Performance over Wire-less Links. IEEE/ACM Transactions on Networking, Vol. 5, No. 6,pp. 756-769, Dec. 1997.
[8] S. Bhandarkar and A.L.N. Reddy. TCP-DCR: Making TCP Robust toNon-Congestion Events. Lecture Notes in Computer Science, Vol. 3042,pp. 712-724, May 2004.
[9] E. Blanton and M. Allman. On Making TCP More Robust to Packet Re-ordering. ACM SIGCOMM Computer Communication Review, Vol. 32,Issue 1, pp. 20-30, Jan. 2002.
[10] S. Bohacek, J.P. Hespanha, J. Lee, C. Lim, and K. Obraczka. A NewTCP for Persistent Packet Reordering. IEEE/ACM Transactions onNetworking, Vol. 14, No. 2, pp. 369-382, Apr. 2006.
117
[11] B. Braden, D. Clark, J. Crowsoft, B. Davie, S. Deering, D. Estrin,S. Floyd, V. Jacobson, G. Minshall, C. Patridge, L. Peterson, K. Ra-makrishnan, S. Shenker, J. Wroclawski, and L. Zhang. Recommenda-tions on Queue Management and Congestion Avoidance in the Internet.IETF RFC 2309, Apr. 1998.
[12] L. Brakmo and L. Peterson. TCP Vegas: End to End Congestion Avoid-ance on a Global Internet. IEEE Journal on Selected Areas in Commu-nication, Vol 13, No. 8, pp. 1465-1480, Oct. 1995.
[13] C. Casetti, M. Gerla, S. Mascolo, M.Y. Sanadidi, and R. Wang. TCPWestwood: End-to-End Congestion Control for Wired/Wireless Net-works. Wireless Networks, Vol. 8, No. 5, pp. 467-479, 2002.
[14] C.-T. Chen. Linear System Theory and Design. New York: Oxford Uni-versity Press, 1999.
[15] M. Chen and A. Zakhor. Flow Control over Wireless Network and Appli-cation Layer Implementation. Proc. of IEEE INFOCOM, pp. 103-113,Mar. 2006.
[16] D.M. Chiu and R. Jain. Analysis of the increase and decrease algorithmsfor congestion avoidance in computer networks. Computer Networks andISDN Systems, Vol. 17, No. 1, pp. 1-14, 1989.
[17] J. Davis. New Networking Features in Windows Server2008 and Windows Vista. http://technet.microsoft.com/en-us/library/bb726965.aspx. Feb. 2008.
[18] C.A. Desoer and Y.T. Wang. On the Generalized Nyquist Stability Cri-terion. IEEE Transactions on Automatic Control, Vol. 25, No. 2, pp. 187-196, Apr. 1980.
[19] K. Fall and K. Varadhan. The ns Manual (formerly ns Notes and Doc-umentation). The VINT Project, 6 January 2009.
[20] X. Fan, M. Arcak, and J.T. Wen. Robustness of Network Flow Controlagainst Disturbances and Time-Delay. System & Control Letters, Vol.53, No. 11, pp. 1329, Sep. 2004.
[21] S. Floyd, T. Henderson, and A. Gurtov. The NewReno Modification toTCP’s Fast Recovery Algorithm. IETF RFC 3782, Apr. 2004.
118
[22] S. Floyd and V. Jacobson. Random Early Detection Gateways for Con-gestion Avoidance. IEEE Transactions on Networking, Vol. 1, No. 4,Aug. 1993.
[23] S. Floyd, J. Mahdavi, M. Mathis, and M. Podolsky. An Extension to theSelective Acknowledgment (SACK) Option for TCP IETF RFC 2883,July 2000.
[24] C.P. Fu and S.C. Liew. TCP Veno: TCP Enhancement for Transmis-sion over Wireless Access Networks. IEEE Journal on Selected Areas inCommunication, Vol. 21, No. 2, pp. 216-228, Feb. 2003.
[25] Z. Fu, P. Zerfos, H. Luo, S. Lu, L. Zhang, and M. Gerla. The Impactof Multihop Wireless Channel on TCP Throughput and Loss. Proc. ofIEEE INFOCOM, Mar. 2003.
[26] H. Han, C.V. Hollot, Y. Chait, and V. Misra. TCP Networks Stabilizedby Buffer-Based AQMs. Proc. of IEEE INOFOCOM, Mar. 2004.
[27] C.V. Hollot, V. Misra, D. Towsley, and W.-B. Gong. Analysis and Designof Controllers for AQM Routers Supporting TCP Flows. IEEE Trans-actions on Automatic Control, Vol. 47, No. 6, June 2002.
[28] F. Hu and N. Sharma. Enhancing Wireless Internet Performance.IEEE Communication Surveys and Tutorials, Vol. 4, No. 1, pp. 2-15,Dec. 2002.
[29] Institute of Electrical and Electronics Engineers. IEEE Standard forLocal and Metropolitan Area Networks Part 16: Air Interface for FixedBroadband Wireless Access Systems. New York: Institute of Electricaland Electronics Engineers, 2009.
[30] V. Jacobson Congestion Avoidance and Control. ACM SIGCOMM Com-pututer Communication Review, Vol. 25, No. 1, pp. 157-187, Jan. 1995.
[31] R. Jain, D.M. Chiu, and W. Hawe. A Quantitative Measurement ofFairness and Discrimination for Resource Allocation in Shared Systems.Technical Report DEC-TR-301, Degital Equipment Corporation, 1984.
[32] D. Katabi, M. Handley, and C. Rohrs. Congestion Control for HighBandwidth-delay Product Networks. ACM SIGCOMM Computer Com-munication Review, Vol. 32, No. 4, Oct. 2002.
119
[33] F. Kelly, A. Maulloo, and D. Tan. Rate Control in Communication Net-works: Shadow Prices, Proportional Fairness and Stability. Journal ofthe Operational Research Society, Vol. 49, No. 3, pp. 237-252, Mar. 1998.
[34] H.K. Khalil. Nonlinear Systems. Upper Saddle River, New Jersey:Prentice-Hall, 2002.
[35] A. Lahanas and V. Tsaoussidis. Improving TCP Performance overNetworks with Wireless Components using ’Probing Devices’. Interna-tional Journal of Communication Systems, Vol. 15, No. 6, pp. 495-511,July/Aug. 2002.
[36] C. Lai, K.-C. Leung, and V.O.K. Li. Demo Abstract: A Testbed for TCPin Heterogeneous Wired/Wireless Networks. Proc. of IEEE INFOCOM2011, Apr. 2011.
[37] C. Lai, K.-C. Leung, and V.O.K. Li. Enhancing AQM to Combat Wire-less Losses. Submitted.
[38] C. Lai, K.-C. Leung, and V.O.K. Li. Enhancing Wireless TCP: ASerialized-Timer Approach. Proc. of IEEE INFOCOM 2010, Mar. 2010.
[39] C. Lai, K.-C. Leung, and V.O.K. Li. TCP-NCL: A Unified Solution forTCP Packet Reordering and Random Loss. Proc. of IEEE PIMRC 2009,Sep. 2009.
[40] T.V. Lakshman and U. Madhow. The The Performance of TCP/IPfor Networks with High Bandwidth-Delay Products and Random Loss.IEEE/ACM Transactions on on Networking, Vol. 5, No. 3, June 1997.
[41] D. Lapsley and S.H. Low. Random Early Marking for Internet Conges-tion Control. Proc. of IEEE Globecom, pp. 1747-1752, Dec. 1999.
[42] G.S. Lee, L.L.H. Andrew, A. Tang, and S.H. Low. WAN-in-Lab: Mo-tivation, Deployment and Experiments. Proc. of PFLDnet, pp. 85-90,Feb. 2007.
[43] K.-C. Leung, C. Lai, V.O.K. Li, and D. Yang. A Packet ReorderingSolution to Wireless Losses in Transmission Control Protocol (TCP).Submitted.
[44] K.-C. Leung and V.O.K. Li. Transmission Control Protocol (TCP) inWireless Networks: Issues, Approaches, and Challenges. IEEE Commu-nication Surveys and Tutorials, Vol. 8, No. 4, pp. 64-79, Fourth Quar-ter 2006.
120
[45] K.-C. Leung, V.O.K. Li, and D. Yang. An Overview of Packet Reorder-ing in Transmission Control Protocol (TCP): Problems, Solutions, andChallenges. IEEE Transactions on on Parallel and Distributed Systems,Vol. 18, No. 4, pp. 522-535, Apr. 2007
[46] V.O.K. Li, L. Cui, Q. Liu, G.-H. Yang, Z. Zhao, and K.-C. Leung. AHeterogeneous Peer-to-Peer Network Testbed. Proc. of ICUFN, pp. 46-51, Jun. 2009.
[47] V.O.K. Li, C. Lai, and K.-C. Leung. Differentiation among Occurrencesof Packet Reordering, Congestive Packet Loss, and Random PacketLoss in Communication Networks. United States Regular ApplicationNo. 12/879,880, Sep. 2010.
[48] J. Liu and S. Singh. ATCP: TCP for Mobile Ad Hoc Networks. IEEEJournal on Selected Areas in Communication, Vol. 19, No. 7, pp. 1300-1315, July 2001.
[49] S.H. Low. A Duality Model of TCP and Queue Management Algorithms.IEEE/ACM Transactions on Networking, Vol. 11, No. 4, pp. 525-536,Aug. 2003.
[50] S.H. Low, F. Paganini, and J.C. Doyle. Internet Congestion Control.IEEE Control Systems Magazine, Vol. 22, No. 1, pp. 28-43, Feb. 2002.
[51] S.H. Low, F. Paganini, J. Wang, and J.C. Doyle. Linear Stability ofTCP/RED and a Scalable Control. Computer Networks Journal, Vol. 43,No. 5, pp. 633-647, Dec. 2003.
[52] R. Ludwig and R.H. Katz. The Eifel Algorithm: Making TCP RobustAgainst Spurious Retransmissions. ACM SIGCOMM Computer Com-munication Review, Vol. 30, Issue 1, pp. 30-36, Jan. 2000.
[53] V. Misra, W.-B. Gong, and D. Towsley. Fluid-based Analysis of a Net-work of AQM Routers Supporting TCP Flows with an Applicationto RED. ACM SIGCOMM Computer Communication Review, Vol. 30,No. 4, pp. 151-160, Oct. 2000.
[54] D. Mitzel. Overview of 2000 IAB Wireless Internetworking Workshop.RFC 3002, Network Working Group, Dec. 2000.
[55] J. Mo, R.J. La, V. Anantharam, and J. Walrand. Analysis and Com-parison of TCP Reno and Vegas. Proc. of IEEE INFOCOM, Mar. 1999.
121
[56] J. Nagle. Congestion Control in IP/TCP Internetworks. RFC 896, Net-work Working Group, Jan. 1984.
[57] R. Pan, B. Prabhakar, and K. Psounis. CHOKe: A Stateless AQMScheme for Approximating Fair Bandwidth Allocation. Proc. of IEEEINFOCOM, Mar. 2000.
[58] C. Parsa and J. Garcia-Luna-Aceves. Improving TCP Congestion Con-trol over Internets with Heterogeneous Transmission Media. Proc. ofIEEE ICNP 1999, pp. 213-221, Oct./Nov. 1999.
[59] L. Peterson, T. Anderson, D. Culler, and T. Roscoe. A blueprint forintroducing disruptive technology into the Internet. ACM SIGCOMMComputer Communication Review, Vol. 33, No. 1, pp. 59-64, Jan. 2003.
[60] J. Postel. Transmission Control Protocol. RFC 793, Protocol Specifica-tion, DARPA Internet Program, Sep. 1981.
[61] K.K. Ramakrishnan, S. Floyd, and D. Black. The Addition of ExplicitCongestion. Notification (ECN) to IP. IETF RFC 3168, Sep. 2001.
[62] P. Sarolahti and M. Kojo. Forward RTO-Recovery (F-RTO): An Algo-rithm for Detecting Spurious Retransmission Timeouts with TCP andthe Stream Control Transmission Protocol (SCTP). IETF RFC 4138,Aug. 2005.
[63] S. Seth and M.A. Venkatesulu. TCP/IP Architecture, Design and Im-plementation in Linux. Hoboken, New Jersey: Wiley, 2008.
[64] S. Shenker. Fundamental Design Issues for the Future Internet. IEEEJournal on Selected Areas in Communications, Vol. 13, No. 7, pp. 1176-1188, Sep. 1995.
[65] R. Srikant. The Mathematics of Internet Congestion Control. Cam-bridge, MA:Birkhauser, 2004.
[66] W. Stevens. TCP Slow Start, Congestion Avoidance, Fast Retransmit,and Fast Recovery Algorithms. IETF RFC 2001, Jan. 1997.
[67] A. Tang, J. Wang, S.H. Low, and M. Chiang. Equilibrium of Hetero-geneous Congestion Control: Existence and Uniqueness. IEEE/ACMTransactions on Networking, Vol. 15, No. 4, pp. 1422-1435, Aug. 2007.
122
[68] A. Tang, D. Wei, S.H. Low and M. Chiang. Equilibrium of Heteroge-neous Congestion Control: Optimality and Stability. IEEE/ACM Tran-sanctions on Networking, Vol. 18, No. 3, pp. 844-857, Jun. 2010.
[69] F. Wang and Y. Zhang. Improving TCP Performance over Mobile Ad-Hoc Networks with Out-Of-Order Detection and Response. Proc. ofACM MOBIHOC 2002, pp. 217-225, June 2002.
[70] J. Wang, J. Wen, J. Zhang, and Y. Han. TCP-FIT: An Improved TCPCongestion Control Algorithm and its Performance. Proc. of IEEE IN-FOCOM, Apr. 2011.
[71] D. Wei, C. Jin, S. Low, and S. Hegde. FAST TCP: Motivation, Architec-ture, Algorithms, Performance. IEEE/ACM Transactions on Network-ing, Vol. 14, No. 6, pp.1246-1259, Dec. 2006.
[72] J.T. Wen and M. Arcak. A Unifying Passivity Framework for NetworkFlow Control. IEEE Transactions on Automatic Control, Vol. 49, No. 2,pp. 162-174, Feb. 2004.
[73] G. Wright and W. Stevens. TCP/IP Illustrated, Volume 2: The Imple-mentation. Addison-Weasley, 1995.
[74] E. H.-K. Wu and M.-Z. Chen. JTCP: Jitter-Based TCP for Heteroge-neous Wireless Network. IEEE Journal on Selected Areas in Communi-cation, Vol. 22, No. 4, pp. 757-766, May 2004.
[75] Y. Xia, L. Subramanian, I. Stoica, and S. Kalyannaramana. One MoreBit is Enough. IEEE/ACM Transactions on Networking, Vol. 16, No. 6,Mar. 2008.
[76] K. Xu, M. Gerla, L. Qi, and Y. Shu. Enhancing TCP Fairness in AdHoc Wireless Networks Using Neighborhood RED. Proc. of ACM MO-BICOM, Sep. 2003.
[77] D. Yang, K.-C. Leung, and V.O.K. Li. Simulation-based Comparisonsof Solutions for TCP Packet Reordering in Wireless Networks. Proc. ofIEEE WCNC, pp. 3240-3245, March 2007.
[78] L. Ying, G.E. Dullerud, and R. Srikant. Global Stability of InternetCongestion Controllers with Heterogeneous Delays. IEEE/ACM Trans-actions on Networking, Vol. 14, No. 3, pp. 579-591, Jun. 2006.
123
[79] H. Zhang, B. Liu, and W. Dou. Design of a Robust Active QueueManagement Algorithm Based on Feedback Compensation. ACM SIG-COMM Computer Communication Review, Vol. 33, No. 4, pp. 277-285,Oct. 2003.
[80] L. Zhang and D. Clark. Oscillating Behavior of Network Traffic: A CaseStudy Simulation. Internetworking: Research and Experience, Vol. 1,No. 2, pp. 101-112, 1990.
[81] M. Zhang, B. Karp, S. Floyd, and L. Peterson. RR-TCP: A Reordering-Robust TCP with DSACK. Proc. of IEEE ICNP 2003, pp. 95-106,Nov. 2003.
[82] Z. Zhao, Q. Liu, G.-H. Yang, V.O.K. Li, and L. Cui. A Multi-radioWireless P2P Network Testbed. Proc. of ACM WiNTECH, pp. 93-94,Sep. 2009.
124
Acronyms
ACK acknowledgment. 4, 17, 18, 25, 27, 28, 35, 43, 44
AQM active queue management. 1, 3, 9, 12–16, 19–22, 69–73, 77–79, 81–84,86, 90, 97, 100, 104–110, 112–115
awnd advertised window. 6
CBR constant bit rate. 71, 93, 102
CD congestion response decision. 28, 43, 49, 51, 54, 57, 59, 63, 66
CRHP closed right half plane. 85
cwnd congestion window. 6–8, 12, 17, 18, 29, 31, 34–36, 59, 60
ECN explicit congestion notification. 70
FTP File Transfer Protocol. 3
GBR goodput to bandwidth ratio. 65, 66
HTTP Hypertext Transfer Protocol. 3, 93, 102, 104, 108
IC integral controller. 13, 14, 20, 71, 72, 87, 90, 92–94, 96, 97, 100, 102,104, 108, 114
IP Internet Protocol. 3, 19
LLRTX link-layer retransmission. 11, 17
MANET mobile ad-hoc networks. 11
MRRL maximum RTT record length. 44–46
125
NRU NCL-RTT-Update. 43
OLHP open left half plane. 82
PI proportional-integral. 20, 21, 90
RD retransmission decision. 28, 43, 49, 51, 54
RED random early detection. 9, 13, 20, 66, 67, 70, 72, 73, 76, 77, 92–94,97, 100, 102, 104, 108, 109, 114
RTO retransmission timeout. 7, 17, 31, 43, 49–52, 111
RTT round-trip time. 4, 11, 17, 19, 27–29, 35, 37, 43–46, 50, 68, 80, 88, 91,110, 111
ssthresh slow start threshold value. 6, 7, 29, 35
STS smart TCP sender. 12, 14, 23–25, 28, 43, 49, 68, 107, 109, 114
TCP Transmission Control Protocol. 1, 3, 8–21, 23–25, 27–30, 33–36, 43,44, 46, 50–52, 54–56, 58, 59, 61–63, 65–67, 69, 70, 72, 73, 76–78, 81,83, 91, 93, 100, 104, 106–115
TCP-NCL TCP for non-congestive loss. 13, 14, 24, 25, 43, 46, 49, 51–56,59, 60, 62–64, 66–68, 70, 76, 107–109, 112, 114
TCP/IP Transmission Control Protocol/Internet Protocol. 2, 53
WMN wireless mesh network. 11
WPER wireless packet error rate. 13, 71, 73, 77, 86, 87, 93, 96, 97, 102,108, 110, 112, 114, 115
126