IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. 13, NO. 3 ... · A QoS-Oriented Distributed Routing...

16
A QoS-Oriented Distributed Routing Protocol for Hybrid Wireless Networks Ze Li, Student Member, IEEE, and Haiying Shen, Member, IEEE Abstract—As wireless communication gains popularity, significant research has been devoted to supporting real-time transmission with stringent Quality of Service (QoS) requirements for wireless applications. At the same time, a wireless hybrid network that integrates a mobile wireless ad hoc network (MANET) and a wireless infrastructure network has been proven to be a better alternative for the next generation wireless networks. By directly adopting resource reservation-based QoS routing for MANETs, hybrids networks inherit invalid reservation and race condition problems in MANETs. How to guarantee the QoS in hybrid networks remains an open problem. In this paper, we propose a QoS-Oriented Distributed routing protocol (QOD) to enhance the QoS support capability of hybrid networks. Taking advantage of fewer transmission hops and anycast transmission features of the hybrid networks, QOD transforms the packet routing problem to a resource scheduling problem. QOD incorporates five algorithms: 1) a QoS-guaranteed neighbor selection algorithm to meet the transmission delay requirement, 2) a distributed packet scheduling algorithm to further reduce transmission delay, 3) a mobility-based segment resizing algorithm that adaptively adjusts segment size according to node mobility in order to reduce transmission time, 4) a traffic redundant elimination algorithm to increase the transmission throughput, and 5) a data redundancy elimination-based transmission algorithm to eliminate the redundant data to further improve the transmission QoS. Analytical and simulation results based on the random way-point model and the real human mobility model show that QOD can provide high QoS performance in terms of overhead, transmission delay, mobility-resilience, and scalability. Index Terms—Hybrid wireless networks, multihop cellular networks, routing algorithms, quality of service Ç 1 INTRODUCTION T HE rapid development of wireless networks has stimu- lated numerous wireless applications that have been used in wide areas such as commerce, emergency services, military, education, and entertainment. The number of WiFi capable mobile devices including laptops and handheld devices (e.g., smartphone and tablet PC) has been increas- ing rapidly. For example, the number of wireless Internet users has tripled world-wide in the last three years, and the number of smartphone users in US has increased from 92.8 million in 2011 to 121.4 million in 2012, and will reach around 207 million by 2017 [1]. Nowadays, people wish to watch videos, play games, watch TV, and make long- distance conferencing via wireless mobile devices “on the go.” Therefore, video streaming applications such as Qik [2], Flixwagon [3], and FaceTime [4] on the infrastructure wireless networks have received increasing attention recently. These applications use an infrastructure to directly connect mobile users for video watching or interaction in real time. The widespread use of wireless and mobile devices and the increasing demand for mobile multimedia streaming services are leading to a promising near future where wireless multimedia services (e.g., mobile gaming, online TV, and online conferences) are widely deployed. The emergence and the envisioned future of real time and multimedia applications have stimulated the need of high Quality of Service (QoS) support in wireless and mobile networking environments [5]. The QoS support reduces end- to-end transmission delay and enhances throughput to guarantee the seamless communication between mobile devices and wireless infrastructures. At the same time, hybrid wireless networks (i.e., multi- hop cellular networks) have been proven to be a better network structure for the next generation wireless networks [6], [7], [8], [9], and can help to tackle the stringent end-to- end QoS requirements of different applications. Hybrid networks synergistically combine infrastructure networks and MANETs to leverage each other. Specifically, infra- structure networks improve the scalability of MANETs, while MANETs automatically establish self-organizing networks, extending the coverage of the infrastructure networks. In a vehicle opportunistic access network (an instance of hybrid networks), people in vehicles need to upload or download videos from remote Internet servers through access points (APs) (i.e., base stations) spreading out in a city. Since it is unlikely that the base stations cover the entire city to maintain sufficiently strong signal every- where to support an application requiring high link rates, the vehicles themselves can form a MANET to extend the coverage of the base stations, providing continuous net- work connections. How to guarantee the QoS in hybrid wireless networks with high mobility and fluctuating bandwidth still remains an open question. In the infrastructure wireless networks, QoS provision (e.g., Intserv [10], RSVP [11]) has been proposed for QoS routing, which often requires node negotiation, admission control, resource reservation, and priority scheduling of packets [12]. However, it is more IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. 13, NO. 3, MARCH 2014 693 . The authors are with the Department of Electrical and Computer Engineering, Clemson University, Clemson, SC 29634. E-mail: {zel, shenh}@clemson.edu. Manuscript received 16 Jan. 2012; revised 15 May 2012; accepted 12 Dec. 2012; published online 20 Dec. 2012. For information on obtaining reprints of this article, please send e-mail to: [email protected], and reference IEEECS Log Number TMC-2012-01-0022. Digital Object Identifier no. 10.1109/TMC.2012.258. 1536-1233/14/$31.00 ß 2014 IEEE Published by the IEEE CS, CASS, ComSoc, IES, & SPS

Transcript of IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. 13, NO. 3 ... · A QoS-Oriented Distributed Routing...

Page 1: IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. 13, NO. 3 ... · A QoS-Oriented Distributed Routing Protocol for Hybrid Wireless Networks Ze Li, Student Member, IEEE, and Haiying Shen,

A QoS-Oriented Distributed RoutingProtocol for Hybrid Wireless Networks

Ze Li, Student Member, IEEE, and Haiying Shen, Member, IEEE

Abstract—As wireless communication gains popularity, significant research has been devoted to supporting real-time transmission

with stringent Quality of Service (QoS) requirements for wireless applications. At the same time, a wireless hybrid network that

integrates a mobile wireless ad hoc network (MANET) and a wireless infrastructure network has been proven to be a better alternative

for the next generation wireless networks. By directly adopting resource reservation-based QoS routing for MANETs, hybrids networks

inherit invalid reservation and race condition problems in MANETs. How to guarantee the QoS in hybrid networks remains an open

problem. In this paper, we propose a QoS-Oriented Distributed routing protocol (QOD) to enhance the QoS support capability of hybrid

networks. Taking advantage of fewer transmission hops and anycast transmission features of the hybrid networks, QOD transforms

the packet routing problem to a resource scheduling problem. QOD incorporates five algorithms: 1) a QoS-guaranteed neighbor

selection algorithm to meet the transmission delay requirement, 2) a distributed packet scheduling algorithm to further reduce

transmission delay, 3) a mobility-based segment resizing algorithm that adaptively adjusts segment size according to node mobility in

order to reduce transmission time, 4) a traffic redundant elimination algorithm to increase the transmission throughput, and 5) a data

redundancy elimination-based transmission algorithm to eliminate the redundant data to further improve the transmission QoS.

Analytical and simulation results based on the random way-point model and the real human mobility model show that QOD can provide

high QoS performance in terms of overhead, transmission delay, mobility-resilience, and scalability.

Index Terms—Hybrid wireless networks, multihop cellular networks, routing algorithms, quality of service

Ç

1 INTRODUCTION

THE rapid development of wireless networks has stimu-lated numerous wireless applications that have been

used in wide areas such as commerce, emergency services,military, education, and entertainment. The number of WiFicapable mobile devices including laptops and handhelddevices (e.g., smartphone and tablet PC) has been increas-ing rapidly. For example, the number of wireless Internetusers has tripled world-wide in the last three years, and thenumber of smartphone users in US has increased from92.8 million in 2011 to 121.4 million in 2012, and will reacharound 207 million by 2017 [1]. Nowadays, people wish towatch videos, play games, watch TV, and make long-distance conferencing via wireless mobile devices “on thego.” Therefore, video streaming applications such as Qik[2], Flixwagon [3], and FaceTime [4] on the infrastructurewireless networks have received increasing attentionrecently. These applications use an infrastructure to directlyconnect mobile users for video watching or interaction inreal time. The widespread use of wireless and mobiledevices and the increasing demand for mobile multimediastreaming services are leading to a promising near futurewhere wireless multimedia services (e.g., mobile gaming,online TV, and online conferences) are widely deployed.The emergence and the envisioned future of real time and

multimedia applications have stimulated the need of highQuality of Service (QoS) support in wireless and mobilenetworking environments [5]. The QoS support reduces end-to-end transmission delay and enhances throughput toguarantee the seamless communication between mobiledevices and wireless infrastructures.

At the same time, hybrid wireless networks (i.e., multi-hop cellular networks) have been proven to be a betternetwork structure for the next generation wireless networks[6], [7], [8], [9], and can help to tackle the stringent end-to-end QoS requirements of different applications. Hybridnetworks synergistically combine infrastructure networksand MANETs to leverage each other. Specifically, infra-structure networks improve the scalability of MANETs,while MANETs automatically establish self-organizingnetworks, extending the coverage of the infrastructurenetworks. In a vehicle opportunistic access network (aninstance of hybrid networks), people in vehicles need toupload or download videos from remote Internet serversthrough access points (APs) (i.e., base stations) spreadingout in a city. Since it is unlikely that the base stations coverthe entire city to maintain sufficiently strong signal every-where to support an application requiring high link rates,the vehicles themselves can form a MANET to extend thecoverage of the base stations, providing continuous net-work connections.

How to guarantee the QoS in hybrid wireless networkswith high mobility and fluctuating bandwidth still remainsan open question. In the infrastructure wireless networks,QoS provision (e.g., Intserv [10], RSVP [11]) has beenproposed for QoS routing, which often requires nodenegotiation, admission control, resource reservation, andpriority scheduling of packets [12]. However, it is more

IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. 13, NO. 3, MARCH 2014 693

. The authors are with the Department of Electrical and ComputerEngineering, Clemson University, Clemson, SC 29634.E-mail: {zel, shenh}@clemson.edu.

Manuscript received 16 Jan. 2012; revised 15 May 2012; accepted 12 Dec.2012; published online 20 Dec. 2012.For information on obtaining reprints of this article, please send e-mail to:[email protected], and reference IEEECS Log Number TMC-2012-01-0022.Digital Object Identifier no. 10.1109/TMC.2012.258.

1536-1233/14/$31.00 � 2014 IEEE Published by the IEEE CS, CASS, ComSoc, IES, & SPS

Page 2: IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. 13, NO. 3 ... · A QoS-Oriented Distributed Routing Protocol for Hybrid Wireless Networks Ze Li, Student Member, IEEE, and Haiying Shen,

difficult to guarantee QoS in MANETs due to their uniquefeatures including user mobility, channel variance errors,and limited bandwidth. Thus, attempts to directly adapt theQoS solutions for infrastructure networks to MANETsgenerally do not have great success [13]. Numerousreservation-based QoS routing protocols have been pro-posed for MANETs [14], [15], [16], [17], [18], [19], [20], [21],[22] that create routes formed by nodes and links thatreserve their resources to fulfill QoS requirements.Although these protocols can increase the QoS of theMANETs to a certain extent, they suffer from invalidreservation and race condition problems [12]. Invalidreservation problem means that the reserved resourcesbecome useless if the data transmission path between asource node and a destination node breaks. Race conditionproblem means a double allocation of the same resource totwo different QoS paths.

However, little effort has been devoted to support QoSrouting in hybrid networks. Most of the current works inhybrid networks [23], [24], [25], [26], [27] focus onincreasing network capacity or routing reliability but cannotprovide QoS-guaranteed services. Direct adoption of thereservation-based QoS routing protocols of MANETs intohybrid networks inherits the invalid reservation and racecondition problems.

In order to enhance the QoS support capability of hybridnetworks, in this paper, we propose a QoS-OrientedDistributed routing protocol (QOD). Usually, a hybridnetwork has widespread base stations. The data transmis-sion in hybrid networks has two features. First, an AP canbe a source or a destination to any mobile node. Second, thenumber of transmission hops between a mobile node andan AP is small. The first feature allows a stream to haveanycast transmission along multiple transmission paths toits destination through base stations, and the second featureenables a source node to connect to an AP through anintermediate node. Taking full advantage of the twofeatures, QOD transforms the packet routing problem intoa dynamic resource scheduling problem. Specifically, inQOD, if a source node is not within the transmission rangeof the AP, a source node selects nearby neighbors that canprovide QoS services to forward its packets to base stationsin a distributed manner. The source node schedules thepacket streams to neighbors based on their queuingcondition, channel condition, and mobility, aiming toreduce transmission time and increase network capacity.The neighbors then forward packets to base stations, whichfurther forward packets to the destination. In this paper, wefocus on the neighbor node selection for QoS-guaranteedtransmission. QOD is the first work for QoS routing inhybrid networks. This paper makes five contributions.

. QoS-guaranteed neighbor selection algorithm. The algo-rithm selects qualified neighbors and employsdeadline-driven scheduling mechanism to guaranteeQoS routing.

. Distributed packet scheduling algorithm. After quali-fied neighbors are identified, this algorithm sche-dules packet routing. It assigns earlier generatedpackets to forwarders with higher queuing delays,while assigns more recently generated packets to

forwarders with lower queuing delays to reducetotal transmission delay.

. Mobility-based segment resizing algorithm. The sourcenode adaptively resizes each packet in its packetstream for each neighbor node according to theneighbor’s mobility in order to increase the schedul-ing feasibility of the packets from the source node.

. Soft-deadline based forwarding scheduling algorithm. Inthis algorithm, an intermediate node first forwardsthe packet with the least time allowed to waitbefore being forwarded out to achieve fairness inpacket forwarding.

. Data redundancy elimination based transmission. Due tothe broadcasting feature of the wireless networks,the APs and mobile nodes can overhear and cachepackets. This algorithm eliminates the redundantdata to improve the QoS of the packet transmission.

2 THE QOD PROTOCOL

2.1 Network and Service Models

We consider a hybrid wireless network with an arbitrarynumber of base stations spreading over the network. Nmobile nodes are moving around in the network. Eachnode ni ð1 � i � NÞ uses IEEE 802.11 interface with theCarrier Sense Multiple Access with Collision Avoidance(CSMA/CA) protocol [28]. Since a hybrid network wherenodes are equipped with multiinterfaces that transmitpackets through multichannels generate much less inter-ference than a hybrid network where nodes are equippedwith a single WiFi interface, we assume that each node isequipped with a single WiFi interface in order to deal with amore difficult problem. Therefore, the base stations consid-ered in this paper are access points (APs). The WiFi interfaceenables nodes to communicate with both APs and mobilenodes. For example, in a University campus, normally onlybuildings have APs. Therefore, people that do not have WiFiaccess but close to buildings can use two-hop relaytransmissions to connect to the APs in the buildings. Feeneyet al. [29] considered the similar scenario in his work.

We use Ri and R0

i to denote the packet transmissionrange and transmission interference range of node ni,respectively. We use di;j to denote the distance betweennodes ni and nj. A packet transmission from ni to nj issuccessful if both conditions below are satisfied [30]:1) di;j � Ri, and 2) any node nk satisfying dk;j � R0k is nottransmitting packets, where 0 < k < N and k 6¼ j. Table 1lists the symbols used in this paper for reference.

694 IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. 13, NO. 3, MARCH 2014

TABLE 1List of Symbols

Page 3: IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. 13, NO. 3 ... · A QoS-Oriented Distributed Routing Protocol for Hybrid Wireless Networks Ze Li, Student Member, IEEE, and Haiying Shen,

The QoS requirements mainly include end-to-end delaybound, which is essential for many applications withstringent real-time requirement. While throughput guaran-tee is also important, it is automatically guaranteed bybounding the transmission delay for a certain amount ofpackets [31]. The source node conducts admission control tocheck whether there are enough resources to satisfy therequirements of QoS of the packet stream. Fig. 1 shows thenetwork model of a hybrid network. For example, when asource node n1 wants to upload files to an Internet serverthrough APs, it can choose to send packets to the APsdirectly by itself or require its neighbor nodes n2, n3, or n4

to assist the packet transmission.We assume that queuing occurs only at the output ports

of the mobile nodes [32]. After a mobile node generates thepackets, it first tries to transmit the packets to its nearbyAPs that can guarantee the QoS requirements. If it fails (e.g.,out of the transmission range of APs or in a hot/dead spot),it relies on its neighbors that can guarantee the QoSrequirements for relaying packets to APs. Relaying for apacket stream can be modeled as a process, in whichpackets from a source node traverse a number of queuingservers to some APs [31]. In this model, the problem of howto guarantee QoS routing can be transformed to theproblem of how to schedule the neighbor resources betweennodes to ensure QoS of packet routing.

2.2 An Overview of the QOD Protocol

Scheduling feasibility is the ability of a node to guarantee apacket to arrive at its destination within QoS requirements.As mentioned, when the QoS of the direct transmissionbetween a source node and an AP cannot be guaranteed, thesource node sends a request message to its neighbor nodes.After receiving a forward request from a source node, aneighbor node ni with space utility less than a thresholdreplies the source node. The reply message containsinformation about available resources for checking packetscheduling feasibility (Section 2.4), packet arrival intervalTa, transmission delay TI!D, and packet deadline Dp of thepackets in each flow being forwarded by the neighbor forqueuing delay estimation and distributed packet schedul-ing (Section 2.5) and the node’s mobility speed fordetermining packet size (Section 2.6). Based on thisinformation, the source node chooses the replied neighborsthat can guarantee the delay QoS of packet transmission toAPs. The selected neighbor nodes periodically report theirstatuses to the source node, which ensures their schedulingfeasibility and locally schedules the packet stream to them.The individual packets are forwarded to the neighbor nodes

that are scheduling feasible in a round-robin fashion from alonger delayed node to a shorter delayed node, aiming toreduce the entire packet transmission delay. Algorithm 1shows the pseudocode for the QOD routing protocolexecuted by each node.

Algorithm 1. Pseudocode for the QOD routing protocol

executed by a source node.

1: if receive a packet forwarding request from a source

node then

2: if this.SpaceUtility < threshold then

3: Reply to the source node.

4: end if

5: end if

6: if receive forwarding request replies for neighbor

nodes then

7: Determine the packet size SpðiÞ to each neighbor i

based on Equation (5).

8: Estimate the queuing delay Tw for the packet for

each neighbor based on Equation (4).

9: Determine the qualified neighbors that can satisfy

the deadline requirements based on Tw10: Sort the qualified nodes in descending order of Tw11: Allocate workload rate Ai for each node based on

Equation (3).

12: for each intermediate node ni in the sorted list do

13: Send packets to ni with transmission intervalSpðiÞAi

.

14: end for

15: end if

The packets travel from different APs, which may lead todifferent packet transmission delay, resulting in a jitter atthe receiver side. The jitter problem can be solved by usingtoken buckets mechanism [33] at the destination APs toshape the traffic flows. This technique is orthogonal to ourstudy in this paper and its details are beyond the scope ofthis paper.

Before introducing the details of QOD in the system, wejustify that QOD is feasible to be used in a network with theIEEE 802.11 protocol in Section 2.3. We then present thedetails of QoS by answering the following questions in QoSrouting in hybrid networks.

1. How to choose qualified neighbors for packetforwarding? (Section 2.4)

2. How to schedule the packets to the qualifiedneighbor nodes? (Section 2.5)

3. How to ensure the QoS transmission in a highlydynamic situation? (Section 2.6)

4. How to schedule the packets in the relay node inforwarding to destinations? (Section 2.7)

5. How to reduce the data redundancy in transmissionto further enhance QoS? (Section 2.8)

2.3 Applicability of the QOD Distributed RoutingAlgorithm

The QOD distributed routing algorithm is developed basedon the assumption that the neighboring nodes in thenetwork have different channel utilities and workloadsusing IEEE 802.11 protocol. Otherwise, there is no need forpacket scheduling in routing, since all neighbors produce

LI AND SHEN: A QOS-ORIENTED DISTRIBUTED ROUTING PROTOCOL FOR HYBRID WIRELESS NETWORKS 695

Fig. 1. The network model of the hybrid networks.

Page 4: IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. 13, NO. 3 ... · A QoS-Oriented Distributed Routing Protocol for Hybrid Wireless Networks Ze Li, Student Member, IEEE, and Haiying Shen,

comparative delay for packet forwarding. Therefore, weanalyze the difference in node channel utilities and work-loads in a network with IEEE 802.11 protocol in order to seewhether the assumption holds true in practice.

2.3.1 Theoretical Analysis of Channel Utility and

Workload Differences

In order to avoid medium access contention and hidden

terminal problem, IEEE 802.11 uses the CSMA/CA protocol

as MAC access control protocol. Before a node sends out

packets, it sends a Request To Send (RTS) message to the

next hop node indicating the duration time of the subse-

quent transmission. The destination node responds with a

Clear To Send (CTS) message to establish a connection with

the source node. The neighbor nodes overhearing RTS and/

or CTS set their Virtual Carrier Sense indicator (i.e., Network

Allocation Vector (NAV)) to the message’s transmission

duration time, so that it can avoid transmitting data into the

channel within the time duration. We define channel utility as

the fraction of time a channel is busy over a unit time.

Assume T is a constant time interval used for channel utility

updating, by referring to NAV and update interval T , each

node ni can statistically calculate its channel utility by

UcðiÞ ¼ TNAV ðiÞT

, where TNAV ðiÞ is the number of time units

that ni is interfered that is recorded by NAV. The available

bandwidth for ni is Wi ¼ ð1� UcðiÞÞ � Ci, where Ci is the

transmission link capacity of node ni.Fig. 2 shows a graph to demonstrate the interference

between two neighboring nodes ni and nj. The solid circlesaround ni and nj denote their packet transmission ranges,and the dotted circles denote their interference ranges(sensing ranges). We use <IðniÞ to represent the interferenceregion of ni that is not overlapped with that of nj, use <IðnjÞto represent the interference region of nj that is notoverlapped with that of ni, and use <Iðni;njÞ to denote theoverlapped region of the interference regions of ni and nj.We first analyze whether the nodes in <IðniÞ, <IðnjÞ, and<Iðni;njÞ have different channel utilities. We see that when njis communicating with nk, the signal will not be received bythe nodes in <IðniÞ, i.e., they can receive or send packetswith other nodes at the same time with no interference fromnj. Similarly, when ni is communicating with nk, the nodesin <IðnjÞ can receive or send packets with other nodes at thesame time with no interference from ni. Thus, the nodes in<IðniÞ are independent from nj and the nodes in <IðniÞ areindependent from ni. As a result, the differences betweenthe time durations of transmitting packets of node ni and

node nj lead to different channel utilities of the nodes in<IðniÞ, <IðnjÞ, and <Iðni;njÞ.Proposition 2.1. The different data transmission amounts of two

neighboring nodes lead to different channel utilities of the

nodes in their common interference region and individual

independent interference regions.

We then analyze the workload difference between twoneighboring nodes ni and nj. We define the workload of anode as the accumulated number of packets received by thenode through the entire simulation period. The workloadsin ni and nj are determined by the packets received by niand nj from the nodes in <T ðniÞ and <T ðnjÞ, respectively,where <T ðniÞ denotes the packet transmission region.We use �< to represent the density of nodes in area <.Suppose each node’s transmission range is R, each node’sinterference range is R0 ¼ � �Rð� > 1Þ, and the distancebetween two neighboring nodes is d ¼ � �Rð� � 1Þ, we canget Lemma 2.1.

Lemma 2.1. In the different interference regions of two

neighboring nodes ni and nj, the ratio of the number of nodes

in <IðniÞ, <Iðni;njÞ, and <IðnjÞ that have different channelutilities equals �<IðniÞ : � � �<Iðni;njÞ : �<IðnjÞ , where � ¼ 2:46.

Proof. Let S< denote the size of < and suppose � � 1. FromFig. 2, we can get

S<IðniÞ ¼ S<IðnjÞ

¼ �� 2 arccos�

2�

� �� ��2R2 þ �R

2ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi4�2 � �2

p2

and

S<Iðni ;njÞ ¼ � � �2 � R2 � S<IðniÞ :

Therefore, the ratio of the number of nodes in <IðniÞ,<Iðni;njÞ and <IðnjÞ is �<IðniÞ : � � �<Iðni;njÞ : �<IðnjÞ .

Since � � 2 according to the specification of IEEE802.11,

� ¼S<Iðni;njÞ

S<IðniÞ¼

2 arccos��

2�

�� �2R2 � �R2

2

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi4�2 � �2

p��� 2 arccos

��

2�

��� �2R2 þ �R2

2

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi4�2 � �2

p� 2:46:

Theorem 2.1. The number of different channel utility values nu in

the system withN nodes are bounded by �ðNÞ < nu < �ð2NÞ.Proof. According to Lemma 2.1, there are three different

utility values in the areas of 1, 2, and 3 indicated in Fig. 2.In addition, there is another channel utility value in thearea of 4 in Fig. 2. Every time when we add a newtransmitting node into the system, the number ofdifferent utility values increases at least linearly becauseat least one different utility value is introduced intothe system, and at most exponentially because at mostthe same number of existing utility values are introducedinto the system. Therefore, for a system with N nodes,the number of different utilities Nu is bounded by�ðNÞ < nu < �ð2NÞ. tu

696 IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. 13, NO. 3, MARCH 2014

Fig. 2. Interference between two neighboring nodes.

Page 5: IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. 13, NO. 3 ... · A QoS-Oriented Distributed Routing Protocol for Hybrid Wireless Networks Ze Li, Student Member, IEEE, and Haiying Shen,

Suppose the probability of node ni receiving a packet

from nodes in <T ðniÞ and nj receiving a packet from <T ðnjÞis q. The packet size is Sp. Then, we can retrieve following

theorem:

Theorem 2.2. The difference of the workload in node ni and

node nj is

��T ðniÞ� �T ðnjÞ

��� 2 arccos

2

� �� �R2 þ �R

2

2

ffiffiffiffiffiffiffiffiffiffiffiffiffiffi4� �2

p� �q � Sp:

Proof. Based on the geometric calculation, we can get

S<T ðniÞ ¼ S<T ðnjÞ ¼ �� 2 arccos�

2

� �� �R2 þ �R

2

2

ffiffiffiffiffiffiffiffiffiffiffiffiffiffi4� �2

p:

The difference of the workload in node ni and node nj is

affected by the traffic from <T ðniÞ and <T ðnjÞ, which is

equal to ð�T ðniÞ � �T ðnjÞÞ � S<T ðniÞ � q � Sp. tuThe theoretical analysis show that if the source nodes

are independent and identically distributed in a system

with random packet generation rate, the nodes with the

IEEE 802.11 protocol present diversified channel utilities

and workloads, which is suitable for distributed resource

scheduling. In Section 6, in the supplemental file, which

can be found on the Computer Society Digital Library

at http://doi.ieeecomputersociety.org/10.1109/TMC.2012.

258, we present our experimental evaluation of channel

utility and workload differences.

2.4 QoS-Guaranteed Neighbor Selection Algorithm

Since short delay is the major real-time QoS requirement fortraffic transmission, QOD incorporates the Earliest DeadlineFirst scheduling algorithm (EDF) [34], which is a deadlinedriven scheduling algorithm for data traffic scheduling inintermediate nodes. In this algorithm, an intermediate nodeassigns the highest priority to the packet with the closestdeadline and forwards the packet with the highest priorityfirst. Let us use SpðiÞ to denote the size of the packet steamfrom node ni, use Wi to denote the bandwidth of node i,and TaðiÞ to denote the packet arrival interval from node ni.

Theorem 2.3. The QoS of the packets going through node ni

can be satisfied ifSpð1ÞTað1Þ þ

SpðjÞTaðjÞ þ � � � þ

SpðmÞTaðmÞ �Wi.

Proof. Liu et. al [34] proposed a job scheduling model,

where a task consists of a number of jobs. The authors

proved that for a given set of m jobs for an operating

system, the deadline-driven scheduling algorithm is

feasible for the job scheduling iff

Tcpð1ÞTgð1Þ

þ Tcpð2ÞTgð2Þ

þ TcpðjÞTgðjÞ

þ � � � þ TcpðmÞTgðmÞ

� 1; ð1Þ

where TgðjÞ denotes the job arrival interval time period

and TcpðjÞ denotes the job computing time of task j and 1

is the CPU utility when the CPU is busy all the time.Recall that space utility UsðiÞ is the fraction of time a

node ni is busy with packet forwarding over a unit time.In a communication network, the transmission time of apacket in packet stream from node nj can be regarded asthe computing time TcpðjÞ of a job from task j, the packetarrival interval Ta can be regarded as Tg, and the CPU

utility can be regarded as node space utility in the jobscheduling model. Then, (2) is reduced to

UsðiÞ ¼Spð1Þ=Wi

Tað1Þþ Spð2Þ=Wi

Tað2Þþ SpðjÞ=Wi

TaðjÞ

þ � � � þ SpðmÞ=Wi

TaðmÞ� 1

) Spð1ÞTað1Þ

þ SpðjÞTaðjÞ

þ � � � þ SpðmÞTaðmÞ

�Wi;

ð2Þ

where Wi ¼ ð1� UcðiÞÞ � Ci, SpðjÞ is the size of the packetsteam from node nj. tuEquation (2) indicates that the scheduling feasibility of a

node is affected by packet size Sp, the number of packetstreams from m neighbors, and its bandwidth Wi.

Similar to the random early detection (RED) algorithm[35], in which a queue length threshold is set to avoidqueuing congestion, we set up a space utility threshold eTUsfor each node as a safety line to make the queue schedulingfeasible. We use UasðiÞ to denote the available space utilityand UasðiÞ ¼ eTUs � UsðiÞ. In QOD, after receiving a forwardrequest from a source node, an intermediate node ni withspace utility less than threshold eTUs replies the source node.The replied node ni informs the source node about itsavailable workload rate UasðiÞ �Wi, and the necessaryinformation to calculate the queuing delay of the packetsfrom the source node. The source node selects the repliedneighbor nodes that can meet its QoS deadline for packetforwarding based on the calculated queuing delay. We willexplain the details of this step in Section 2.5.

After the source node determines the Nq nodes that cansatisfy the deadline requirement of the source node, thesource node needs to distribute its packets to the Nq nodesbased on their available workload rate UasðiÞ �Wi to makethe scheduling feasible in each of the neighbor nodes. Then,the problem can be modeled as a linear programmingprocess. Suppose the packet generating rate of the sourcenode is Wg kb/s, the available workload rate of theintermediate node i is UasðiÞ �Wi, and the workload rateallocation from source node to immediate node i isAi ¼ SpðiÞ

TaðiÞ , where 0 < i < n. Then, we need to solve thefollowing equations to get an allocation set A:

A ¼ Wg ¼XNq

i¼1

Ai

Ai � UasðiÞ �Wi:

8><>: ð3Þ

Any results that satisfy (3) can be used by the source node.If the equation cannot be solved, which means the QoS ofthe source node cannot be satisfied, then the source nodestops generating packets based on the admission controlpolicy. How to determine the packet size SpðiÞ for ni willbe introduced in Section 2.6. Based on SpðiÞ, the sourcenode can calculate the packet arrival interval for node ni:TaðiÞ ¼ SpðiÞ

Ai.

For example, suppose the bandwidth Wi of the inter-mediate node ni is 70 kb/s, the threshold of the workload is80 percent of the overall space utility, which is 56 kb/s. Nodeni schedules the packet traffic from three different sourcenodes n1, n2, and n3 periodically. The packet size of traffic

LI AND SHEN: A QOS-ORIENTED DISTRIBUTED ROUTING PROTOCOL FOR HYBRID WIRELESS NETWORKS 697

Page 6: IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. 13, NO. 3 ... · A QoS-Oriented Distributed Routing Protocol for Hybrid Wireless Networks Ze Li, Student Member, IEEE, and Haiying Shen,

from n1, n2, and n3 are 1, 10, and 20 kb with arrival interval0.1, 0.5, and 1 s, respectively. Then, S1

T1þ S2

T2þ S3

T3¼ 50 kb/s.

When another node n4 sends a request to the intermediate ni,ni checks its own available workload rate and replies to thesource node its available workload rate 6 kb/s. If n4 acceptsthe reply and sends 20 kb/s traffic to the intermediate nodeni, ni will reject the request and inform n4 to reduce thetraffic to 6 kb/s. Once the bandwidth of the intermediatenode drops to 60 kb/s because of the interference, ni’soverall space utility is reduced to 48 kb/s. Then, ni informsnode n3 under scheduling that has the largest SpðjÞTaðjÞ ¼ 20 kb/sto change its traffic to 18 kb/s.

2.5 Distributed Packet Scheduling Algorithm

Section 2.4 solves the problem of how to select intermediatenodes that can guarantee the QoS of the packet transmissionand how a source node assigns traffic to the intermediatenodes to ensure their scheduling feasibility. In order tofurther reduce the stream transmission time, a distributedpacket scheduling algorithm is proposed for packet routing.This algorithm assigns earlier generated packets to for-warders with higher queuing delays and schedulingfeasibility, while assigns more recently generated packetsto forwarders with lower queuing delays and schedulingfeasibility, so that the transmission delay of an entire packetstream can be reduced.

We use t to denote the time when a packet is generated,

and use TQoS to denote the delay QoS requirement. Let WS

and WI denote the bandwidth of a source node and an

intermediate node respectively, we use TS!I ¼ SpWS

to denote

the transmission delay between a source node and an

intermediate node, and TI!D ¼ SpWI

to denote the transmis-

sion delay between an intermediate node and an AP. Let Twdenote the packet queuing time and TwðiÞ denote the packet

queuing time of ni. Then, as Fig. 3 shows, the queuing delay

requirement is calculated as Tw < TQoS � TS!I � TI!D. As

TQoS , TS!I , and TI!D are already known, the source node

needs to calculate Tw of each intermediate node to select

intermediate nodes that can send its packets by the

deadline, i.e., that can satisfy Tw < TQoS � TS!I � TI!D.Below we introduce how to calculate Tw. Recall that QOD

incorporates the EDF [34], in which an intermediate nodeassigns the highest priority to the packet with the closestdeadline and forwards the packet with the highest priorityfirst. An intermediate node can determine the priorities ofits packets based on their deadlines Dp. A packet with asmaller priority value x has a higher priority. Beforeintroducing the details of the distributed packet scheduling

algorithm, we explain how to estimate the queuing time T ðxÞw

of a packet with priority x. It is estimated by

T ðxÞw ¼Xx�1

j¼1

�TðjÞI!D �

�T ðxÞw =T ðjÞa

��ð0 < j < xÞ; ð4Þ

where x denotes a packet with the xth priority in the

queue, and TðjÞI!D and T ðjÞa respectively denote the transmis-

sion delay and arrival interval of a packet with the jth

priority. dT ðxÞw =T ðjÞa e is the number of packets arriving

during the packet’s queuing time T ðxÞw , which are sent out

from the queue before this packet.As mentioned in Section 2.4, after receiving the reply

messages from neighbor nodes that includes the schedulinginformation of all flows in their queues, the source nodecalculates the Tw of its packets in each intermediate nodeand then chooses the intermediate node ni that satisfiesTwðiÞ < TQoS � TS!I � TI!D.

After scheduling traffics to qualified intermediate nodesbased on (3), the earlier generated packet from source nodeis transmitted to a node with longer queuing delay but stillwithin the deadline bound. Taking advantage of thedifferent Tw in different neighbor nodes, the transmissiontime of the entire traffic stream can be decreased by makingthe queuing of previous generated packets and thegenerating of new packets be conducted in parallel.

As Fig. 3 shows, a source node generates three packetsp1, p2, and p3 with the same size at times t0, t1, and t2(t0 < t1 < t2), respectively. A packet p’s total transmissiondelay equals: TS!IðiÞ þ TwðiÞ þ TI!DðiÞ. Since all thesepackets are generated from the same node, the transmissiondelay from the source node to each intermediate nodeTS!Ið1Þ, TS!Ið2Þ, and TS!Ið3Þ are almost the same. Tosimplify the analysis, we suppose TI!Dð1Þ ¼ TI!Dð2Þ ¼TI!Dð3Þ. If the queuing delay in each intermediate nodesatisfies Twð1Þ > Twð2Þ > Twð3Þ, then packet p1 should besent to the first intermediate node, packet p2 should be sentto the second intermediate node, and packet p3 should besent to the third intermediate node. As a result, the finalpacket delivery time for the three packets from theintermediate nodes to the destination node can be reduced.

Theorem 2.4. Given a certain amount of packets to transmit,QOD produces higher throughput than the single shortestpath transmission method [20], in which a source node alwaystransmits packets through a single shortest path, and thedistributed randomized method, in which packets are randomlyassigned to the selected intermediate nodes.

Proof. Suppose the packet generating rate of a source nodeni is �. That is, the packet arrival interval is Ta ¼ 1

� . Thequeuing delay time Tw in different intermediate nodesare different, and Twð1Þ > Twð2Þ > Twð3Þ > TwðjÞ > � � �>TwðmÞ (1 � i � m). According to the distributed packetscheduling algorithm in QOD, the time to transmitm packets to different APs through m qualified inter-mediate nodes is T ½d� ¼maxði � TaþTwðiÞÞþTS!I þTI!D(1 � i � m). According to the Little’s law [36], for a stablesystem, the packet queuing time is less than the packetarrival interval, i.e., TwðiÞ < Ta. As i increases, i � Taincreases and TwðiÞ decreases. Therefore, T ½d� ¼ mTa þTwðmÞ þ TS!I þ TI!D. Since a packet in a queue needs towait for a number of packets to be sent out before it can

698 IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. 13, NO. 3, MARCH 2014

Fig. 3. Distributed queuing mechanism.

Page 7: IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. 13, NO. 3 ... · A QoS-Oriented Distributed Routing Protocol for Hybrid Wireless Networks Ze Li, Student Member, IEEE, and Haiying Shen,

be forwarded, TwðmÞ � TS!I and TI!D. Therefore,T ½d� � mTa þ TwðmÞ. Similarly, the transmission time ofdistributed randomized algorithm is T ½r� � mTa þ TwðkÞ(1 � k � m). In the single shortest path transmissionmethod, the time for transmitting m packets is T ½s� ¼mTaþ

Pmi¼1TwðiÞþTS!I þTI!D �mTaþ

Pmi¼1TwðiÞ. Then,

T ½d� � T ½s� ¼ TwðmÞ �Xmi¼1

TwðiÞ � 0:

Therefore,

T ½d� � T ½s�:

Similarly, T ½d��T ½r� ¼mTaþTwðmÞ�ðmTaþTwðkÞÞ. SinceTwðmÞ � TwðkÞ, T ½d� � T ½r�. tuAs the throughput in two-hop transmission is normally

less than the throughput of direct transmission, the two-hoptransmission is only used in two cases: 1) when the packetsender is out of the range of an AP, and 2) APs in range arecongested. In these two cases, the direct communication toan AP cannot provide QoS guarantee, and the two-hoptransmission is needed.

2.6 Mobility-Based Packet Resizing Algorithm

In a highly dynamic mobile wireless network, the transmis-sion link between two nodes is frequently broken down.The delay generated in the packet retransmission degradesthe QoS of the transmission of a packet flow. On the otherhand, a node in a highly dynamic network has higherprobability to meet different mobile nodes and APs, whichis beneficial to resource scheduling. As (2) shows, the spaceutility of an intermediate node that is used for forwarding apacket p is

SpWi�Ta . That is, reducing packet size can increase

the scheduling feasibility of an intermediate node andreduces packet dropping probability. However, we cannotmake the size of the packet too small because it generatesmore packets to be transmitted, producing higher packetoverhead. Based on this rationale and taking advantage ofthe benefits of node mobility, we propose a mobility-basedpacket resizing algorithm for QOD in this section. The basicidea is that the larger size packets are assigned to lowermobility intermediate nodes and smaller size packets areassigned to higher mobility intermediate nodes, whichincreases the QoS-guaranteed packet transmissions. Speci-fically, in QOD, as the mobility of a node increases, the sizeof a packet Sp sent from a node to its neighbor nodes i

decreases as following:

SpðnewÞ ¼�

viSpðunitÞ; ð5Þ

where � is a scaling parameter and vi is the relative mobilityspeed of the source node and intermediate node andSpðunitÞ ¼ 1 kb.

Proposition 2.2. The QOD protocol can provide soft state QoS of

packet routing in a highly dynamic network.

Proof. Since the packet size of a packet that is sent from asource node to an intermediate node ni is �

vi, and the

average packet arrival interval is �ð1�Þ, the space utility ofa node ni with bandwidth Wi is

UsðiÞ ¼Sp

Wi � Ta¼

�vi

Wi � 1�

¼ � � �Wi � vi

:

As vi increase, space utility UsðiÞ decreases. Then, thetraffic from the source node is more easily to bescheduled. Therefore, based on QOD routing protocol,QoS of the traffic can be guaranteed in a highlydynamic situation. tu

2.7 Soft-Deadline-Based Forwarding Scheduling

Recall that in the EDF algorithm, an intermediate nodeforwards the packets in the order from the packets with theclosest deadlines to the packets with the farthest deadlines.If an intermediate node has no problem to meet all packets’deadlines in forwarding, that is, the packets are schedulingfeasible, the EDF algorithm works satisfactorily. However,when an intermediate node has too many packets toforward out and the deadlines of some packets must bemissed, EDF forwards out the packets with the closestdeadlines but may delay the packets with the farthestdeadlines. Therefore, EDF is suitable for hard-deadlinedriven applications (e.g., online conferences) where packetsmust be forwarded before their deadlines but may not befair to all arriving packets in soft-deadline driven applica-tions (e.g., online TV), where the deadline missing issometimes acceptable.

In order to achieve fairness in the packet forwardingscheduling for soft-deadline driven applications, a for-warding node can use the least slack first (LSF) schedulingalgorithm [37]. The slack time of a packet p is defined asDp � t� c0, where t is the current time and c0 is theremaining packet transmission time of the packet. Forexample, a packet’s remaining forwarding time is 5 s andthe time interval from current time to its deadline is 20 s.Then, its slack time equals 20� 5 ¼ 15 s. With the LSFalgorithm, an intermediate node periodically calculates theslack time of each of its packets, and forwards the packetwith the least slack time. If all packets have the same slacktime value, one packet is randomly chosen to be sent out.

Therefore, the objective of LSF is different from that ofEDF. LSF does not aim to complete transmitting the packetflows before their deadlines. Rather, it aims to make delaysand the sizes of delayed part in the delayed packets(delayed size in short) of different packet flows almost thesame. If the packets are scheduling feasible according to (2),the LSF algorithm can meet all deadlines of packets.Otherwise, the forwarding node takes turns to forwardthe packets based on their slack times. Therefore, LSF canachieve more fairness than EDF. QOD can choose either LSFor EDF based on the applications and we can apply (4) forLSF the same as EDF. The priorities of the packets aredetermined by the chosen policy.

Below, we use an example of an intermediate node’spacket forwarding scheduling to compare the effect of EDFand LSF. To help readers understand the process moreclearly, in the supplementary file, which is available online,we present a more complex example where each packet flowhas multiple packet arrivals during a certain time period.

Fig. 4 shows the parameters in the example. Aforwarding node is receiving packets from three packetflows: a, b, and c. Because of the resizing algorithm, theincoming packets from different source nodes may have

LI AND SHEN: A QOS-ORIENTED DISTRIBUTED ROUTING PROTOCOL FOR HYBRID WIRELESS NETWORKS 699

Page 8: IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. 13, NO. 3 ... · A QoS-Oriented Distributed Routing Protocol for Hybrid Wireless Networks Ze Li, Student Member, IEEE, and Haiying Shen,

different packet sizes, hence have different packet trans-mission time periods. For example, the first row indicatesthat the forwarding node receives packets from packet flowa with arrival interval 32 s. Each packet has the deadlineequals “arrival time + 2 s,” and needs 2 s to be transmittedto its destination.

The tables in Fig. 5 show the deadline and slack time ateach second for each packet, and the figures below show theforwarding scheduling results of EDF and LSF. In the tables,we use the superscript to denote the packet sequencenumber. For example, in the deadline table, in the line of“packet flow a,” 21 at time 0 means that the deadline of thefirst packet of packet flow a is 2 at time 0. Similarly, in theslack time table, in the line of “packet flow a,” 01 at time 0means that the slack time of the first packet of packet flow ais 0 at time 0. The slack time 01 at time 0 is calculated bysubtracting current time 0 and the remaining transmissiontime 2 from the deadline 2. The gray color means that thispacket has the smallest deadline or slack time and is chosento be forwarded. A bold value with underscore means it isfor the newly arrived packet.

In EDF, packet b and packet c missed deadline for delay 2and 4 s, respectively. The variance is 1. Their delayed sizesare 2 and 2 kb, respectively. In LSF, all of the packets misseddeadline with delays equal to 2, 4, and 3 s, respectively. Thevariance is 0.67. In LSF, suppose the forwarding rate of anode is 1 kb/s, the delayed size of flows a, b, and c are 1, 2,and 1 kb, respectively. Comparing the results of LSF withthose of EDF, we see that LSF distributes delayed time anddelayed size to more packet flows, achieving fairness inforwarding scheduling while EDF reduces the number ofdelayed packets.

2.8 Data Redundancy Elimination

As we introduced in Section 2.3.1, the mobile nodes set theirNAV values based on the overhearing message’s transmis-sion duration time. A large NAV leads to a small availablebandwidth and a small scheduling feasibility of the mobile

nodes based on (2). Therefore, by reducing the NAV value,we can increase the scheduling feasibility of the intermedi-ate nodes and sequentially increase the QoS of the packettransmission. Due to the broadcasting feature of thewireless networks, in a hybrid network, the APs andmobile nodes can overhead and cache packets, we use anend-to-end traffic redundancy elimination (TRE) algorithm[38] to eliminate the redundancy data to improve the QoS ofthe packet transmission in QOD. TRE uses a chunkingscheme to determine the boundary of the chunks in a datastream. The source node caches the data it has sent out andthe receiver also caches its received data.

In QOD with TRE, the AP and mobile nodes overhearand cache packets. From the overhearing, the nodes knowwho have received the packets. When a source node beginsto send out packets, it scans the content for duplicatedchunks in its cache. If the sender finds a duplicated chunkand it knows that the AP receiver has received this chunkbefore, it replaces this chunk with its signature (i.e., SHA-1hash value). When the AP receives the signature, it searchesthe signature in its local cache. If the AP caches the chunkassociated with the signature, it sends a confirmationmessage to the sender and replaces the signature with thematched data chunk. Otherwise, the AP requests the chunkof the signature from the sender.

Fig. 6 shows an example of packet redundant eliminationin hybrid networks. As shown in Fig. 6a, when node n1

sends a message “abcd” to a nearby AP through n2, n3

overhears the message and caches a chunk “abc” in its localmemory. The AP receiver also caches the message. Later on,when n3 sends a message “abcf” and n1 sends a message“abce” to the AP, since they know the AP has cached “abc,”they only need to send “1f” or “1e,” where “1” is thesignature of the chunk “abc.” Then, the AP is able toreconstruct the full chunk using its signature. The reductionin the size of the message increases the schedulingfeasibility of the mobile nodes, which further enhances theQoS performance of the system.

3 PERFORMANCE EVALUATION

This section demonstrates the distinguishing properties ofQOD compared to E-AODV [20], S-Multihop [39], Two-hop[27] through simulations on NS-2 [40]. E-AODV is aresource reservation-based routing protocol for QoS routingin MANETs. This protocol extends AODV by addinginformation of the maximum delay and minimum availablebandwidth of each neighbor in a node’s routing table. Toapply E-AODV in hybrid networks, we let a source node

700 IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. 13, NO. 3, MARCH 2014

Fig. 4. An example of packets received by the forwarder.

Fig. 5. Comparison of the forwarding scheduling methods.

Fig. 6. Example of packet redundant elimination.

Page 9: IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. 13, NO. 3 ... · A QoS-Oriented Distributed Routing Protocol for Hybrid Wireless Networks Ze Li, Student Member, IEEE, and Haiying Shen,

search for the QoS guaranteed path to an AP. Theintermediate nodes along the path reserve the resourcesfor the source node. In S-Multihop, a node always forwardsa packet to a next hop node that has small buffer usage thanitself until the packet reaches an AP. In Two-hop, the sourcenode adaptively chooses direct transmission (i.e., directlytransmit packets to the AP) and forward transmission (i.e.,transmit packets through a forwarding node) to forwardpackets to APs.

In the simulation, the setup was the same as Section 6.Six APs with IEEE 802.11 MAC protocol are uniformlydistributed in the area. We randomly selected two sourcenodes to send packets to APs in every 10 s. A node’s trafficis generated with constant bit rate (CBR) sources. Thegeneration rate of the CBR traffic is 100 kb/s. Unlessotherwise specified, the speeds of the nodes were randomlyselected from [1-40]m/s. Since the number of successfullydelivered packets within a certain delay is critical to theQoS of video streaming applications, we define a newmetric, namely QoS guaranteed throughput (QoS through-put in short), that measures the throughput sent from asource node to a destination node satisfying a QoS delayrequirement as 1 s. This metric can simultaneously reflectdelay, throughput, and jitter features of packet transmis-sion. We directly use the threshold parameter in RED queue[35] as our space utility threshold. We run each experimentfor 10 times. We first selected the simulation results withinconfidence interval of 95 percent of the ten simulations, andthen calculated the average result as the final result. Thewarmup time was set to 100 s and the simulation time wasset to 200 s per round.

3.1 Performance with Different Mobility Speeds

In this experiment, a node’s mobility speed was randomlyselected from ½1; x�m/s ðx ¼ 1; 10; 20; 30; 40Þ. Fig. 7 plots theQoS throughputs of all systems versus the node mobilityspeed. It shows that the QoS throughputs of all systemsdecrease as node mobility increases. This is because highermobility causes higher frequent link breakages, whichleads to more packet drops. Reestablishing the broken linksresults in a long transmission delay for subsequent packets.We can also see that the QoS throughputs of QODand Two-hop slightly decrease, but those of E-AODV andS-Multihop decrease sharply. E-AODV and S-Multihophave much more hops in the routing paths from the sourcenodes to APs than QOD and Two-hop. A longer routingpath produces higher probability of link breakdown duringthe packet transmission. As Two-hop and QOD only havetwo hops in the routing paths to APs, the short paths havelower probability to break down. Even if a link breaksdown, the source node can quickly choose another

forwarder. Therefore, node mobility does not greatly affectthese two protocols.

E-AODV has much smaller QoS throughput thanothers with different node mobility speeds. This is becausein E-AODV, the routing resources in each link are reservedfor QoS traffic. In a highly dynamic network, the reservedlinks constantly break down, which leads to the invalidreservation problem, forcing the source node to search for anew path to an AP. The delay resulted from the pathsearching degrades the ability to meet the QoS require-ments. The race condition problem further decrease the QoSthroughput as the same resources are reserved for differentsource nodes at the same time. Then some source nodescannot obtain the resources as scheduled. Therefore, theQoS of the packet traffic in E-AODV is very difficultto guarantee in a highly dynamic network. Since a node inS-Multihop directly forwards a packet to the next hop withsmaller buffer usage without reserving resource, it gener-ates higher QoS throughput than E-AODV that suffers fromdelay from path discovery. Meanwhile, in S-Multihop, asseveral source nodes may send packets to the node withsmaller buffer usage at the same time, the node is veryeasily congested. Although the routing path length in Two-hop is always two as QOD, as Two-hop only concerns nodebandwidth in packet forwarding rather than buffer usage, itmay suffer severe buffer congestion in the selected nodewith high bandwidth. Therefore, S-Multihop produceshigher QoS throughput than Two-hop in a low-mobilitynetwork. However, S-Multihop suffers severely from nodemobility due to it long paths while Two-hop is mobilityresilient due to its short path. Thus, S-Multihop generatesless QoS throughput than Two-hop in a high node mobility.

In QOD, rather than reserving the resources in eachtransmission link, the intermediate nodes periodicallyreport their queuing status to the source node. The sourcenode adaptively schedules the packets to the neighbornodes based on their current space utilities. In this way,there is no need for retransmission caused by invalidresource reservation. Moreover, since every intermediatenode can receive scheduled packets for the forwardingtransmission, and the same scheduled resource will not beallocated to more than one source node at the same time,the race contention problem can be avoided. Furthermore, apacket resizing algorithm is used for traffic scheduling byleveraging the various mobility of the nodes for QoSguaranteed routing. It can increase the scheduling feasi-bility of an intermediate node on a packet. Because of theincreased overhead due to more packet heads in QOD in ahigher dynamic network, its QoS throughput decreasesslightly as node mobility increases. Although both QODand Two-hop have at most two hops from source nodes toAPs, QOD constantly generates higher QoS throughputthan Two-hop. This is caused by two reasons. First, QODdynamically schedules the packets to the neighbors that canguarantee QoS routing based on (2), while Two-hopforwards the packets to the nodes with high bandwidthwhich may become congested. Second, Two-hop does nottake advantage of low-bandwidth nodes which may stillsupport the QoS routing due to lower queue delay, whileQoD makes full use of the resources of the nodes around a

LI AND SHEN: A QOS-ORIENTED DISTRIBUTED ROUTING PROTOCOL FOR HYBRID WIRELESS NETWORKS 701

Fig. 7. QoS throughput versus mobility.

Page 10: IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. 13, NO. 3 ... · A QoS-Oriented Distributed Routing Protocol for Hybrid Wireless Networks Ze Li, Student Member, IEEE, and Haiying Shen,

source node, and distributively forwards the packets to theAPs, improving the QoS throughput of the system.

Though both QOD and S-Multihop schedule the

packet forwarding to the APs, QOD constantly outper-forms S-Multiple with different mobility speeds due tothree reasons. First, QOD is more mobility-resilient due toits short path length and its mobility-based segment

resizing algorithm. While S-Multiple is not mobility-resilient due to its long path length. Second, QOD providesa QoS-guaranteed neighbor selection algorithm to ensure

that the selected next hop node can guarantee QoS routing.It also uses admission control mechanism to prevent sourcenodes from generating packets if no neighbor nodes cansatisfy the QoS requirement. S-Multihop only focuses on

buffer usage of the next hop node, which increases the QoSof packet transmission to a certain extent but cannot ensurethe QoS of the forwarding. Consequently, the QoSthroughput in QOD is higher than that in S-Multihop.

We define the fraction of QoS throughput (QoS fraction in

short) as the ratio of QoS throughput to total packetthroughput. This metric shows the effectiveness of differentsystems in supporting QoS routing. Fig. 8 shows the QoS

fraction of all the systems. The figure shows that as the nodemobility speed increases, the fraction of QoS throughput ofall systems decreases, i.e., more received packets cannotmeet their QoS requirements. Specifically, the QoS fraction

in S-Multihop and E-AODV drops sharply, while that ofQOD and Two-hop drops marginally as the averagemobility of the nodes in the system increases. This is due

to the reason that S-Multihop and E-AODV generate longerpath lengths than QOD and Two-hop, and hence sufferfrom more severe link breakdown, which prevents packetsfrom arriving at APs in time. Since QOD’s distributed

packet scheduling algorithm can avoid race contention aspreviously explained, and the packet resizing algorithm canincrease the scheduling feasibility of the intermediatenodes, its QoS fraction decreases only slightly. This

decrease is caused by the increased overhead in the systemwith higher node mobility. The packet resizing algorithmgenerates smaller packet size in higher node mobility, thus

producing more packets for a given data stream and hencemore transmission overhead. We see that the QoS fractionin Two-hop also slightly decreases as the node mobilityincreases. This is because faster mobility leads to higher

frequency of link breakdown and hence more droppedpackets on the fly. Fig. 8 also shows that QOD has thehighest fraction of QoS throughput, and Two-hop con-

stantly outperforms S-Multihop and E-AODV that exhibitthe worst performance due to the same reasons as in Fig. 7.

We define the overhead rate as the size of all controlpackets generated by the system in 1 s. The controlpackets include all control packets and packet headersexcluding the data packets. Fig. 9 plots the overheadrates of different systems with different node mobilityspeeds. We see that the overhead rates of all systemsincrease as node mobility increases, and the resultfollows S-Multihop>E-AODV>QOD>Two-hop when nodemobility is larger than 15 m/s. The overhead of QODmainly consists of two parts. The first part is caused byperiodical status information exchanges. A source nodeneeds to exchange its status information with its neighbornodes periodically during the packet transmission time forpacket scheduling. With higher node mobility, a sourcenode meets more nodes, leading to more exchangedinformation. The second part is caused by the packetheads. Although the packet size of each packet is reducedas node mobility increases, more packets are generated fora given data stream. The extra packet heads increase theoverhead of QOD. Consequently, as node mobilityincreases, the overhead of QOD also increases.

E-AODV does not need to periodically exchangechannel information for packet scheduling, and its over-head is mainly caused by routing reestablish. With lownode mobility, routing paths break down infrequently.Thus, E-AODV has the least overhead rate in a system withlow mobility. The overhead in S-Multihop mainly consistsof two parts: routing reestablish overhead and bufferinformation exchange overhead. Therefore, the overhead ofS-Multihop is significantly higher than that in E-AODVwhen the mobility is low. With high node mobility, theoverhead rate of S-Multihop is only slightly higher thanthat E-AODV. This is because higher node mobility makesthe path maintenance overhead rate dominate the entireoverhead rate.

The overhead in Two-hop mainly resulted from channelinformation exchange for the dynamic packet forwardingand path reestablish overhead. As the forwarding pathlength from source nodes to the APs is only two, which ismuch less than those of E-AODV and S-Multihop, theoverhead rate of Two-hop is lower than those of E-AODVand S-Multihop, especially in the system with nodemobility. Because QOD produces more exchanged controlpackets than Two-hop for packet scheduling, it generatesslightly larger overhead rate than Two-hop. However, withthis slightly additional overhead, QoD greatly increases theQoS throughput of Two-hop as shown in Fig. 7.

3.2 Performance with Different Number of APs

Fig. 10 shows the QoS throughput versus the number ofAPs in the different systems. The figure shows that the

702 IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. 13, NO. 3, MARCH 2014

Fig. 8. Fraction of QoS throughput versus mobility. Fig. 9. Overhead versus mobility.

Page 11: IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. 13, NO. 3 ... · A QoS-Oriented Distributed Routing Protocol for Hybrid Wireless Networks Ze Li, Student Member, IEEE, and Haiying Shen,

increase of APs leads to higher QoS throughput in allsystems. This is because more APs help to reduce pathlengths and physical distances between source nodes andAPs, leading to lower packet transmission than the signalpower, leading to higher data transmission rate. More APssignificantly reduce the lengths of originally long paths tothe APs in E-AODV and S-Multihop, thus dramaticallyincreasing their QoS throughput. In contrast, as QOD andTwo-hop two-hop short path length, their QoS throughputincrease rate is smaller than those of S-Multihop due tothe same reasons explained in Fig. 7, E-AODV producesless QoS throughput than S-Multihop. When the numberof the APs in the system is small, the routing path lengthsof S-Multihop and E-AODV are longer than those ofQOD and Two-hop. Therefore, the QoS throughputs ofQOD and Two-hop are larger than those of S-Multihopand E-AODV. It is very interesting to see that S-Multihophas higher QoS throughput than Two-hop when thenumber of APs in the system is larger than 6. In thiscase, S-Multihop generates much shorter path lengths.Also, S-Multihop uses scheduling algorithm that considersbuffer usage for packet routing, which reduces the packetqueuing delay. However, Two-hop only considers channelcondition for the packet routing and ignores the bufferusage, making high-bandwidth nodes easily congested. Asa result, S-Multihop produces higher QoS throughput thanTwo-hop. Since E-AODV also suffers from congestion onthe nodes close to the APs and its average path length islarger than Two-hop, its QoS throughput is less than Two-hop. As QOD can effectively schedule the channelresources around the source node for packet forwarding,its QoS throughput remains constantly the highest.

3.3 Performance with Different Workloads

Figs. 11a and 11b plot the QoS throughput of the systemswith different number of source nodes when the averagenode mobility is 0 and 20 m/s, respectively. Each node’smobility speed is randomly chosen from the range 0 m/sto the average mobility. More source nodes generate moreworkload in the system. We see from both figures that asthe number of source nodes increases from 0 to 3, the QoSperformance of QOD increases almost linearly. In thesecases, the capacity of the system is not saturated, andhence the QoS throughput increases almost linearly as theworkload grows. When the number of source nodesincreases to 5, the QoS throughput increases at a slowerrate. In QOD, when a source node finds that all of itsneighbors cannot guarantee the QoS of its packets, it stopsgenerating new packet flows into the system based on theadmission control policy. Generating more packets into thenetworks may further decrease the QoS performance of

other source nodes. S-Multihop produces less QoSthroughput increase rate than QOD, which means thesystem with S-Multihop saturates much earlier than thatwith QOD. Although S-Multihop schedules the packetforwarding by forwarding a packet to the next hop withless buffer usage, which can reduce packet bufferinglatency, it does not have a mechanism to prevent a nodewith a full buffer from receiving packets from other nodesin order to ensure the forwarding QoS. Also, as shown inFig. 7, QOD is much more mobility resilient to S-Multihop,which increases QOD’s QoS throughput. In addition,QOD’s distributed packet scheduling algorithm can furtherreduce packet transmission delay, which enhances QOD’scapability to handle the increasing workload in the system.As a result, QOD constantly produces higher QoSthroughput than S-Multihop.

The figure also shows that Two-hop has less QoSthroughput increase rate than S-Multihop as the number ofsource nodes increases. In Two-hop, the packets are alwaysforwarded to the nodes with higher transmission link rate.Without any buffer management strategy, the nodes withhigher transmission links are very easily overloaded as theworkload in the system increases. It is very intriguing tosee, as the number of source nodes increases, E-AODV’sQoS throughput increases initially but decreases later. Thisis because in E-AODV, when the workload of the systemincreases, the probability that two or more source nodessimultaneously reserve the same resources at a nodeincreases due to the race condition problem. Also, thenodes close to the APs are more likely to be congested asE-AODV does not have a resource scheduling mechanism.Therefore, the QoS throughput of E-AODV decreases in ahighly loaded system. Comparing Figs. 11a and 11b, wecan find that the increasing mobility of the nodes in thesystem leads to a decrease of QoS throughput of allprotocols. The reason is the same as in Fig. 7.

3.4 Performance with Different Network Sizes

Figs. 12a and 12b illustrate the QoS throughput of thesystems with different number of nodes at the averagemobility speed of 0 and 20 m/s, respectively. Both figuresshow that as the number of nodes in the system increases,the QoS throughput of QOD increases, that of Two-hopremains constant, but those of E-AODV and S-Multihopdecrease. The throughput increase in QOD is caused by theincreasing number of nodes in the system, which leads to anincreasing number of neighbors of a node, enabling it tohave more available resources for packet traffic scheduling.As the Two-hop always lets the source node forward thepackets to the next hop node with high link rate without

LI AND SHEN: A QOS-ORIENTED DISTRIBUTED ROUTING PROTOCOL FOR HYBRID WIRELESS NETWORKS 703

Fig. 10. QoS throughput versus number of APs.

Fig. 11. QoS throughput versus number of source nodes.

Page 12: IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. 13, NO. 3 ... · A QoS-Oriented Distributed Routing Protocol for Hybrid Wireless Networks Ze Li, Student Member, IEEE, and Haiying Shen,

any resource scheduling as used in QOD, the source nodescannot take advantage of those increased resource nodesaround themselves as the number of nodes in the systemincreases, leading to constant QoS throughput.

In S-Multihop, although more resources are available totransmit packets based on the buffer usage as the networksize increases, the average number of routing hops from thesource node to destination node also increases, which leadsto a higher frequency of link breakdown. The reduced QoSthroughput due to the longer routing path dominates theincreased QoS throughput due to the more availableresources. Thus, the QoS throughput of S-Multihop de-creases slightly. In E-AODV, as the number of nodes in thesystem increases, the average path length grows, whichincreases the probability of path breakdown and decreasesits QoS throughput.

Comparing Figs. 12a and 12b, we see that as the mobilityof the nodes increases from 0 to 20 m/s, the QoSthroughput of E-AODV and S-Multihop decreases moresignificantly as the network size increases. For QOD andTwo-hop, the increase of mobility does not affect their QoSthroughput significantly because of their mobility-resiliencedue to short paths. More details of the reasons arepresented as in Fig. 7.

The figure also shows that in QOD, a system with alarger number of nodes and source nodes has higherthroughput increasing rate. This is due to the reason that alarger number of neighbor nodes can provide moreresources for the packet scheduling. Therefore, even thoughthere is more workload in the system, QOD can effectivelydigest the workload. In Two-hop and S-Multihop, becauseof the inefficient forwarding node selection, their workloaddigest ability is not as good as QOD. Therefore, their QoSthroughput decreases. The QoS throughput in Two-hopwith two source nodes decreases slightly when the numberof nodes in the system increases because more nodes nearthe source nodes generate more interference between eachother. However, as the number of source nodes increases

to 4, its QoS throughput increases. In this case, other idlenodes can be used for packet transmission, and theincreased QoS throughput due to idle nodes surpassesthe decreased QoS throughput due to the interference,leading to overall QoS throughput increase. In E-AODV, asthe number of nodes in the system increases, its QoSthroughput decreases. Moreover, as the number of sourcenodes in the system increases, its QoS throughput exhibitsdramatic decrease. This is caused by two reasons: 1) a largenumber of source nodes produce a high workload in thesystem, resulting in a high probability of race contentionand link breakdown, and 2) a larger number of nodesgenerate more transmission hops, resulting in a highprobability of link breakdown.

3.5 Comparison of EDF and LSF

In this section, we compare EDF with LSF for packetforwarding scheduling in QOD. We let the forwardingnodes receive as many packets from neighbor nodes aspossible without admission control to show the perfor-mance of EDF and LSF when the packets are schedulinginfeasible. In each experiment, during 50 s, we continuallyselected a certain number of random nodes to transmitpackets to their randomly selected destinations for a timeperiod randomly chosen from [1 to 5]s. The link rate fromsource nodes to relay nodes and from relay nodes to BSswas set to 2 m/s. A forwarding node calculates the deadlineand slack time every 1 ms.

Fig. 13a shows the percentage of the delayed packets ofEDF and LSF. When there are two or three source nodes, allpackets are scheduling feasible, thus there is no delayedpacket in both EDF and LSF. As the number of source nodesin the system increases, the percentage of the delayedpackets increases. This is because as more packets aregenerated, every packet in the scheduling queue needs towait for more time to be forwarded out, which leads tohigher delay and hence more delayed packets. We also seethat the percentage of the delayed packets in LSF is higherthan that of EDF. This is because EDF always tries to meetthe deadlines of packets with the earliest deadlines, whileLSF tries to balance the delay among the packets. Therefore,EDF is able to meet more deadlines than LSF.

Fig. 13b shows the first percentile, median, and99th percentile of the delay after deadline of LSF andEDF. We see that EDF has lower first percentile delayand higher 99th percentile delay than LSF, i.e., EDF has agreater variance than LSF. This is because EDF forwards thepackets with the earliest deadlines but significantly delaysthe packets with the farthest deadlines. LSF always tries to

704 IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. 13, NO. 3, MARCH 2014

Fig. 13. Comparison of EDF and LSF for packet forwarding scheduling.

Fig. 12. QoS throughput versus network size and node mobility.

Page 13: IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. 13, NO. 3 ... · A QoS-Oriented Distributed Routing Protocol for Hybrid Wireless Networks Ze Li, Student Member, IEEE, and Haiying Shen,

balance the delay between different packets, so it has muchsmaller delay variance than EDF. This result is alsoconfirmed by Fig. 13c, which shows that the average delayvariance of EDF is higher than that of LSF. Therefore, withthe admission control policy that prevents schedulinginfeasible packets from being generated in the system,EDF can be used to ensure more packets to meet their hard-deadlines. Without the admission control policy, LSF can beused to provide more fairness for the packet forwardingusing soft-deadlines. Fig. 13d shows the first percentile,median, 99th percentile of the delayed packet size of LSFand EDF. We can see that the median delayed size of LSF issmaller than that of EDF and LSF also has smaller variancethan EDF. This is because EDF aims to meet the hard-deadlines of packets while LSF tries to balance the delayedsize among different packet flows, which provides higherfairness for different packet flows.

3.6 Trace-Driven Experiments

In this section, we evaluate the performance of QOD using amore realistic human mobility model based on the tracedata set from the MIT Reality mining project [41] involving94 students and staffs at MIT. We used the records of theconnections with cellular towers in the real trace to infereach node’s mobility for the simulation. We also used sixAPs that are uniformly distributed in the system in thissimulation. All other setups are the same as in Section 6.Fig. 14 shows the QoS throughput versus the number ofsource nodes. We see that the QoS throughput followsQOD>S-Multihop>Two-hop>E-AODV. Also, unlike othersthat produce more QoS throughput with more sourcenodes, E-AODV’s QoS throughput decreases when thenumber of source nodes is more than 3. The reasons forthese results are the same as in Fig. 11.

Comparing Figs. 14 and 11, we can find that eachsystem produces less QoS throughput in the humanmobility model than in the random movement model.As nodes in the human mobility model are likely to beclustered and move together due to the communityclustering property, the communication interference be-tween the nodes in the same cluster may increase thechannel utility of each node, thus reducing the schedulingfeasibility of the nodes. Therefore, the QoS throughput ofthe nodes is reduced in the human mobility model. Sincethe performance of E-AODV does not depend on thechannel utility and space utility of the nodes as it alwaysfinds a routing path to an AP through flooding, the QoSthroughput of E-AODV does not change greatly.

Fig. 15 shows the QoS throughput versus differentnumber of APs. We can see that QOD still generatesthe highest QoS throughput with different number of

APs, which is followed by Two-hop and S-Multihop, andE-AODV produces the smallest amount of QoS through-put. The reasons for the results are the same as in Fig. 10.Comparing Figs. 15 and 10, we find that the QoSthroughputs of QOD, S-Multihop, and Two-hop inFig. 15 are lower than those in Fig. 10. This is also becausethe community clustering property reduces the channelutilization of each node due to the interference betweennodes within a community. As the routing in E-AODVdoes not reply on the states of the neighboring nodes, itsQoS throughput does not decrease significantly in thehuman mobility model. We also find that for QOD, its QoSthroughput in the human mobility model increases muchfaster than in the random movement model as the numberof APs increases. The nodes in human mobility modelsuffer more interference than in random movement modelas explained above. As APs can reduce the influence of theinterference between nodes in the same community sincethe nodes can directly transmit packets to the APs, theincreasing rate of QoS throughput in human mobilitymodel is much faster than in random movement model.Since a node in S-Multihop and Two-hop depends onfewer neighbors in routing, the routing in S-Multihop andTwo-hop suffers less influence from interference thanQOD. Thus, as the number of APs increases, their QoSthroughput is smaller than that of QOD.

Fig. 16 shows the QoS throughput of nodes versusdifferent network sizes and workloads. We see that QODstill generates higher QoS throughput than S-Multihop andTwo-hop with different network sizes and workloads, andE-AODV still has the least QoS throughput due to the samereasons as in Fig. 12.

3.7 Evaluation of TRE Based Transmission

In this section, we evaluate the performance of QOD-TREusing a real Internet traffic trace we captured from anaccess link from a large university in China to the backbone.We still used the MIT trace as the mobility trace. TheInternet trace is 120 s long and contains 1.9 GB HTTP traffic

LI AND SHEN: A QOS-ORIENTED DISTRIBUTED ROUTING PROTOCOL FOR HYBRID WIRELESS NETWORKS 705

Fig. 14. QoS throughput versus number of source nodes. Fig. 15. QoS throughput versus number of APs.

Fig. 16. QoS throughput versus network size and workload.

Page 14: IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. 13, NO. 3 ... · A QoS-Oriented Distributed Routing Protocol for Hybrid Wireless Networks Ze Li, Student Member, IEEE, and Haiying Shen,

between 8,277 host pairs. We used the Internet trace tosimulate the web access behaviors of the mobile nodes inthe hybrid network. The cache size of each mobile node andAP was set to 250 KB and 100 MB, respectively. Thesignature of a chunk is 32 bytes. There are about 5 percentredundancy data in the trace after being parsed with theTRE algorithm [38].

Figs. 17 and 18 show the QoS throughput versus thenumber of source nodes and the number of nodes,respectively. We see as the number of source nodes or thenetwork size increases, the QoS throughput of both QoDand QOD-TRE increases. We can also see that the systemwith higher node mobility has less throughput. The reasonsare the same as in Figs. 14 and 16. We can also see that theQoS throughput of QOD-TRE is consistently larger than theQoS throughput of QOD in both figures. The reason is thatinstead of sending out a whole packet stream chunks,sending out the chunk signature with smaller packet sizecan greatly increase the scheduling feasibility of the nodes,leading to an increase in QoS throughput.

4 RELATED WORK

4.1 Infrastructure Networks

Existing approaches for providing guaranteed services inthe infrastructure networks are based on two models:integrated services (IntServ) [10] and differentiated service(DiffServ) [42]. IntServ is a stateful model that uses resourcereservation for individual flow, and uses admission control[10] and a scheduler to maintain the QoS of traffic flows. Incontrast, DiffServ is a stateless model which uses coarse-grained class-based mechanism for traffic management. Anumber of queuing scheduling algorithms have beenproposed for DiffServ to further minimize packet droppingsand bandwidth consumption [43], [44], [45], [46], [47]. Stoicaet al. [48] proposed a dynamic packet service (DPS) modelto provide unicast IntServ-guaranteed service and Diffserv-like scalability.

4.2 MANETs

A majority of QoS routing protocols are based on resourcereservation [12], in which a source node sends probemessages to a destination to discover and reserve pathssatisfying a given QoS requirement. Perkins et al. [20]extended the AODV routing protocol [49] by addinginformation of the maximum delay and minimum availablebandwidth of each neighbor in a node’s routing table. Jianget al. [15] proposed to reserve the resources from the nodeswith higher link stability to reduce the effects of nodemobility. Liao et al. [50] proposed an extension of the DSRrouting protocol [51] by reserving resources based on time

slots. Venataramanan et al. [39] proposed a schedulingalgorithm to ensure the smallest buffer usage of the nodesin the forwarding path to base stations. However, theseworks focus on maximizing network capacity based onscheduling but fail to guarantee QoS delay performance.

Some works consider providing multipath routing toincrease the robustness of QoS routing. Conti et al. [16]proposed to use nodes’ local knowledge to estimate thereliability of routing paths and select reliable routes. Theworks in [17] and [18] balance traffic load among multipleroutes to increase routing reliability. Shen et al. [19]proposed to let a source node fetch the lost packets fromits neighbors to recover the multicast traffic. Shen andThomas [21] proposed a unified mechanism to maximizeboth the QoS and security of the routing. Li et al. [22]proposed a centralized algorithm to optimize the QoSperformance by considering cross-layer design among thephysical layer, MAC layer, and network layer.

4.3 Wireless Sensor Networks (WSNs)

RAP [52] and SPEED [53] give a high delivering priority tothe packets with longer distance/delay to the destination.However, both methods require each sensor to know itsown location, thus they are not suitable for a highlydynamic environment. Felemban et al. [54] and Deb et al.[55] proposed to improve routing reliability by multipathrouting. However, the redundant transmission of thepackets may lead to high power consumption.

4.4 Hybrid Wireless Networks

Very few methods have been proposed to provide QoS-guaranteed routing for hybrid networks. Most of therouting protocols [23], [24], [25], [26], [27] only try toimprove the network capacity and reliability to indirectlyprovide QoS service but bypass the constraints in QoSrouting that require the protocols to provide guaranteedservice. Jiang et al. [56] proposed a resource provisionmethod in hybrid networks modeled by IEEE802.16e andmobile WiMax to provide service with high reliability.Ibrahim et al. [23] and Bletasa et al. [24] also tried to select“best” relay that has the maximum instantaneous value of ametric which can achieve higher bandwidth efficiency fordata transmission. Ng and Yu [25] considered cooperativenetworks that use physical layer relaying strategies, whichtake advantage of the broadcast nature of wireless channelsand allow the destination to cooperatively “combine”signals sent by both the source and the relay to restorethe original signal. Cai et al. [26] proposed a semidistrib-uted relaying algorithm to jointly optimize relay selectionand power allocation of the system. Wei et al. [57] proposedto use the first-order finite state Markov channels to

706 IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. 13, NO. 3, MARCH 2014

Fig. 17. QoS throughput versus number of source nodes Fig. 18. QoS throughput versus network size.

Page 15: IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. 13, NO. 3 ... · A QoS-Oriented Distributed Routing Protocol for Hybrid Wireless Networks Ze Li, Student Member, IEEE, and Haiying Shen,

approximate the time variations of the average receivedsignal-to-noise ratio (SNR) for the packet transmission anduse the adaptive modulation and coding scheme to achievehigh spectral efficiency. Lee et al. [58] presented a frame-work of link capacity analysis for optimal transmission overuplink transmission in multihop cellular networks. Weiet al. [27] proposed a two-hop packet forwarding mechan-ism, in which the source node adaptively chooses directtransmission and forward transmission to base stations.Unlike the above works, QOD aims to provide QoS-guaranteed routing. QOD fully takes advantage of thewidely deployed APs, and novelly treats the packet routingproblem as a resource scheduling problem between nodesand APs.

5 CONCLUSIONS

Hybrid wireless networks that integrate MANETs andinfrastructure wireless networks have proven to be a betternetwork structure for the next generation networks. How-ever, little effort has been devoted to supporting QoSrouting in hybrid networks. Direct adoption of the QoSrouting techniques in MANETs into hybrid networksinherits their drawbacks. In this paper, we propose a QoS-oriented distributed routing protocol (QOD) for hybridnetworks to provide QoS services in a highly dynamicscenario. Taking advantage of the unique features of hybridnetworks, i.e., anycast transmission and short transmissionhops, QOD transforms the packet routing problem to apacket scheduling problem. In QOD, a source node directlytransmits packets to an AP if the direct transmission canguarantee the QoS of the traffic. Otherwise, the source nodeschedules the packets to a number of qualified neighbornodes. Specifically, QOD incorporates five algorithms. TheQoS-guaranteed neighbor selection algorithm chooses qua-lified neighbors for packet forwarding. The distributedpacket scheduling algorithm schedules the packet transmis-sion to further reduce the packet transmission time. Themobility-based packet resizing algorithm resizes packetsand assigns smaller packets to nodes with faster mobility toguarantee the routing QoS in a highly mobile environment.The traffic redundant elimination-based transmission algo-rithm can further increase the transmission throughput. Thesoft-deadline-based forwarding scheduling achieves fair-ness in packet forwarding scheduling when some packetsare not scheduling feasible. Experimental results show thatQOD can achieve high mobility-resilience, scalability, andcontention reduction. In the future, we plan to evaluate theperformance of QOD based on the real testbed.

ACKNOWLEDGMENTS

This work was supported in part by the US National ScienceFoundation under grants OCI-1064230, CNS-1249603, CNS-1049947, CNS-1156875, CNS-0917056, CNS-1057530, CNS-1025652, CNS-0938189, CSR-2008826, and CSR-2008827,Microsoft Research Faculty Fellowship 8300751, and theUS Department of Energy’s Oak Ridge National Laboratoryincluding the Extreme Scale Systems Center located atORNL and DoD 4000111689. An early version of this workwas presented in the Proceedings of MASS ’10 [59].

REFERENCES

[1] “A Majority of U.S. Mobile Users Are Now Smartphone Users,”http://adage.com/article/digital/a-majority-u-s-mobile-users-smartphone-users/241717, 2013.

[2] Qik, http://qik.com, 2013.[3] Flixwagon, http://www.flixwagon.com, 2013.[4] Facebook, http://www.facebook.com, 2013.[5] H. Wu and X. Jia, “QoS Multicast Routing by Using Multiple

Paths/Trees in Wireless Ad Hoc Networks,” Ad Hoc Networks,vol. 5, pp. 600-612, 2009.

[6] H. Luo, R. Ramjeey, P. Sinhaz, L. Liy, and S. Lu, “UCAN: AUnified Cell and Ad-Hoc Network Architecture,” Proc. ACMMobiCom, 2003.

[7] P.K. Mckinley, H. Xu, A. Esfahanian, and L.M. Ni, “Unicast-BasedMulticast Communication in Wormhole-Routed Direct Net-works,” IEEE Trans. Parallel Data and Distributed Systems, vol. 5,no. 12, pp. 1252-1265, Dec. 1992.

[8] H. Wu, C. Qiao, S. De, and O. Tonguz, “Integrated Cell andAd Hoc Relaying Systems: iCAR,” IEEE J. Selected Areas in Comm.,vol. 19, no. 10, pp. 2105-2115, Oct. 2001.

[9] J. Zhou and Y.R. Yang, “PAR CelS: Pervasive Ad-Hoc Relaying forCell Systems,” Proc. IFIP Mediterranean Ad Hoc Networking Work-shop (Med-Hoc-Net), 2002.

[10] R. Braden, D. Clark, and S. Shenker, Integrated Services in theInternet Architecture: An Overview, IETF RFC 1633, 1994.

[11] E. Crawley, R. Nair, B. Rajagopalan, and H. Sandick, ResourceReservation Protocol RSVP, IETF RFC 2205, 1998.

[12] I. Jawhar and J. Wu, “Quality of Service Routing in Mobile Ad HocNetworks,” Network Theory and Applications, Springer, 2004.

[13] T. Reddy, I. Karthigeyan, B. Manoj, and C. Murthy, “Quality ofService Provisioning in Ad Hoc Wireless Networks: A Survey ofIssues and Solutions,” Ad Hoc Networks, vol. 4, no. 1, pp. 83-124,2006.

[14] X. Du, “QoS Routing Based on Multi-Class Nodes for MobileAd Hoc Networks,” Ad Hoc Networks, vol. 2, pp. 241-254, 2004.

[15] S. Jiang, Y. Liu, Y. Jiang, and Q. Yin, “Provisioning ofAdaptability to Variable Topologies for Routing Schemes inMANETs,” IEEE J. Selected Areas in Comm., vol. 22, no. 7,pp. 1347-1356, Sept. 2004.

[16] M. Conti, E. Gregori, and G. Maselli, “Reliable and EfficientForwarding in Ad Hoc Networks,” Ad Hoc Networks, vol. 4,pp. 398-415, 2006.

[17] G. Chakrabarti and S. Kulkarni, “Load Balancing and ResourceReservation in Mobile Ad Hoc Networks,” Ad Hoc Networks, vol. 4,pp. 186-203, 2006.

[18] A. Argyriou and V. Madisetti, “Using a New Protocol to EnhancePath Reliability and Realize Load Balancing in Mobile Ad HocNetworks,” Ad Hoc Networks, vol. 4, pp. 60-74, 2006.

[19] C. Shen and S. Rajagopalan, “Protocol-Independent MulticastPacket Delivery Improvement Service for Mobile Ad Hoc Net-works,” Ad Hoc Networks, vol. 5, pp. 210-227, 2007.

[20] C.E. Perkins, E.M. Royer, and S.R. Das, Quality of Service in Ad HocOn-Demand Distance Vector Routing, IETF Internet draft, 2001.

[21] Z. Shen and J.P. Thomas, “Security and QoS Self-Optimization inMobile Ad Hoc Networks,” IEEE Trans. Mobile Computing, vol. 7,pp. 1138-1151, Sept. 2008.

[22] Y. Li and A. Ephremides, “A Joint Scheduling Power Control andRouting Algorithm for Ad Hoc Networks,” Ad Hoc Networks, 2008.

[23] S. Ibrahim, K. Sadek, W. Su, and R. Liu, “Cooperative Commu-nications with Relay-Selection: When to Cooperate and Whom toCooperate With?” IEEE Trans. Wireless Comm., vol. 7, no. 7,pp. 2814-2827, July 2008.

[24] A. Bletsas, A. Khisti, D.P. Reed, and A. Lippman, “A SimpleCooperative Diversity Method Based on Network Path Selection,”IEEE J. Selected Areas in Comm., vol. 24, no. 3, pp. 659-672, Mar.2006.

[25] T. Ng and W. Yu, “Joint Optimization of Relay Strategies andResource Allocations in Cellular Networks,” IEEE J. Selected Areasin Comm., vol. 25, no. 2, pp. 328-339, Feb. 2004.

[26] J. Cai, X. Shen, J.W. Mark, and A.S. Alfa, “Semi-Distributed UserRelaying Algorithm for Amplify-and-Forward Wireless RelayNetworks,” IEEE Trans. Wireless Comm., vol. 7, no. 4, pp. 1348-1357, Apr. 2008.

[27] Y. Wei and D. Gitlin, “Two-Hop-Relay Architecture for Next-Generation WWAN/WLAN Integration,” IEEE Wireless Comm.,vol. 11, no. 2, pp. 24-30, Apr. 2004.

LI AND SHEN: A QOS-ORIENTED DISTRIBUTED ROUTING PROTOCOL FOR HYBRID WIRELESS NETWORKS 707

Page 16: IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. 13, NO. 3 ... · A QoS-Oriented Distributed Routing Protocol for Hybrid Wireless Networks Ze Li, Student Member, IEEE, and Haiying Shen,

[28] Wireless LAN Medium Access Control (MAC) and Physical Layer(PHY) Specifications, IEEE-SA Standards Board, 1999.

[29] L. Feeney, B. Cetin, D. Hollos, M. Kubisch, S. Mengesha, and H.Karl, “Multi-Rate Relaying for Performance Improvement inIEEE 802.11 WLANS,” Proc. Fifth Int’l Conf. Wired/WirelessInternet Comm., 2007.

[30] P. Gupta and P.R. Kumar, “The Capacity of Wireless Net-works,” IEEE Trans. Information Theory, vol. 46, no. 2, pp. 388-404, Mar. 2000.

[31] H. Zhang, “Service Disciplines for Guaranteed PerformanceService in Packet-Switching Networks,” Proc. IEEE, vol. 83,no. 10, pp. 1374-1396, Oct. 1995.

[32] M. Karol, M. Hluchyj, and S. Mogan, “Input versus OutputQueueing on a Space-Division Packet Switch,” IEEE Trans. Comm.,vol. 35, no. 12, pp. 1347-1356, Dec. 1987.

[33] S. Sahu, P. Nain, C. Diot, V. Firoiu, and D. Towsley, “OnAchievable Service Differentiation with Token Bucket Marking forTCP,” Proc. ACM SIGMETRICS Int’l Conf. Measurement andModeling of Computer Systems (SIGMETRICS), 2000.

[34] C. Liu and J. Layland, “Scheduling Algorithms for Multiprogram-ming in a Hard Real-Time Environment,” J. ACM, vol. 20, pp. 46-61, 1973.

[35] D. Lin and R. Morris, “Dynamics of Random Early Detection,”Proc. ACM Special Interest Group Data Comm. (SIGCOMM),1997.

[36] D.P. Bertsekas and R.G. Gallager, Data Networks. Prentice Hall,1992.

[37] A. Cheng, Real-Time Systems: Scheduling Analysis, and Verification,first ed. Wiley-Interscience, 2002.

[38] E. Zohar, I. Cidon, and O. Mokryn, “The Power of Prediction:Cloud Bandwidth and Cost Reduction,” Proc. ACM Special InterestGroup Data Comm. (SIGCOMM), 2011.

[39] V. Venkataramanan, X. Lin, L. Ying, and S. Shakkottai, “OnScheduling for Minimizing End-to-End Buffer Usage over Multi-Hop Wireless Networks,” Proc. IEEE INFOCOM, 2010.

[40] “The Network Simulator - ns-2,” http://www.isi.edu/nsnam/ns,2013.

[41] “The Reality Mining Dataset,” http://reality.media.mit.edu, 2013.[42] Y.E. Sung, C. Lund, M. Lyn, S. Rao, and S. Sen, “Modeling and

Understanding End-to-End Class of Service Policies in Opera-tional Networks,” Proc. ACM Special Interest Group Data Comm.(SIGCOMM), 2009.

[43] J. Kurose and K. Ross, Computer Networking: A Top-Down ApproachFeaturing the Internet. Addison Wesley, 2004.

[44] L. Kleinrock, “Queueing System,” Computer Applications, 1976.[45] A. Parekh and R. Gallager, “A Generalized Processor Sharing

Approach to Flow Control,” Proc. IEEE INFOCOM, 1992.[46] J.C.R. Bennett and H. Zhang, “WF 2Q: Worst-Case Fair Weighted

Fair Queueing,” Proc. IEEE INFOCOM, 1996.[47] S. Golestani, “A Self-Clocked Fair Queueing Scheme for Broad-

band Applications,” Proc. IEEE INFOCOM, 1994.[48] I. Stoica and H. Zhang, “Providing Guaranteed Services without

Per Flow Management,” Proc. ACM Special Interest Group DataComm. (SIGCOMM), 1999.

[49] C. Perkins, E. Belding-Royer, and S. Das, Ad Hoc on DemandDistance Vector (AODV) Routing, IETF RFC 3561, 2003.

[50] W.H. Liao, Y.C. Tseng, and K.P. Shih, “A TDMA-BasedBandwidth Reservation Protocol for QoS Routing in a WirelessMobile Ad Hoc Network,” Proc. IEEE Int’l Conf. Comm., 2002.

[51] D.B. Johnson and D.A. Maltz, “Dynamic Source Routing inAd Hoc Wireless Networks,” Mobile Computing, vol. 353, pp. 153-181, 1996.

[52] C. Lu, B. Blum, T. Abdelzaher, J. Stankovic, and T. He, “RAP: AReal-Time Communication Architecture for Large-Scale WirelessSensor Networks,” Proc. IEEE Real-Time and Embedded TechnologyApplications Systems, 2002.

[53] T. He, J. Stankovic, C. Lu, and T. Abdelzaher, “SPEED: A StatelessProtocol for Real-Time Communication in Sensor Networks,”Proc. 23rd Int’l Conf. Distributed Computing Systems, 2003.

[54] E. Felemban, C. Lee, and E. Ekici, “MMSPEED: Multipath Multi-Speed Protocol for QoS Guarantee of Reliability and Timeliness inWireless Sensor Networks,” IEEE Trans. Mobile Computing, vol. 5,no. 6, pp. 738-754, June 2006.

[55] B. Deb, S. Bhatnagar, and B. Nath, “ReInForm: Reliable Informa-tion Forwarding Using Multiple Paths in Sensor Networks,” Proc.IEEE 28th Ann. Int’l Conf. Local Computer Networks, 2003.

[56] P. Jiang, J. Bigham, and J. Wu, “Scalable QoS Provisioning andService Node Selection in Relay Based Cellular Networks,” Proc.Fourth Int’l Conf. Wireless Comm. Networking and Mobile Computing(WiCOM), 2008.

[57] Y. Wei, M. Song, F.R. Yu, Y. Zhang, and J. Song, “DistributedOptimal Relay Selection for QoS Provisioning in Wireless Multi-hop Cooperative Networks,” Proc. IEEE 28th Conf. GlobalTelecomm. (GlobeCom), pp. 1946-1951, 2009.

[58] S. Lee and S. Lee, “Optimal Transmission Methodology for QoSProvision of Multi-Hop Cellular Network,” J. Wireless Networks,vol. 16, pp. 1313-1327, 2010.

[59] Z. Li and H. Shen, “A QoS-Oriented Distributed Routing Protocolfor Hybrid Networks,” Proc. IEEE Seventh Int’l Conf. Mobile Adhocand Sensor Systems, 2010.

[60] J. Li, C. Blake, S. Douglas, H. Lee, and R. Morris, “Capacity ofAd Hoc Wireless Networks,” Proc. ACM MobiCom, 2001.

Ze Li received the BS degree in electronics andinformation engineering from the HuazhongUniversity of Science and Technology, China,in 2007. He is currently working toward the PhDdegree in the Department of Electrical andComputer Engineering, Clemson University.His research interests include distributed net-works with an emphasis on peer-to-peer andcontent delivery networks. He is a studentmember of the IEEE.

Haiying Shen received the BS degree incomputer science and engineering from TongjiUniversity, China, in 2000, and the MS and PhDdegrees in computer engineering from WayneState University in 2004 and 2006, respectively.She is currently an assistant professor in theDepartment of Electrical and Computer Engi-neering, Clemson University. Her researchinterests include distributed computer systemsand computer networks with an emphasis on

P2P and content delivery networks, mobile computing, wireless sensornetworks, and cloud computing. She became a Microsoft Faculty Fellowin 2010. She is a member of the IEEE.

. For more information on this or any other computing topic,please visit our Digital Library at www.computer.org/publications/dlib.

708 IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. 13, NO. 3, MARCH 2014