On the packet allocation of Multi-Band AggregationWireless Networks

Sanjay Goyal, Tan Le, Amith Chincholi, Tariq Elkourdi, Alpaslan DemirInterDigital Communications, LLC

2 Huntington Quadrangle, Meville, NY 11747{Sanjay.Goyal, Tan.Le, Amith.Chincholi, Tariq.Elkourdi, Alpaslan.Demir}@InterDigital.com

Abstract—Bandwidth aggregation across multiple radio accesstechnologies (multi-RAT) has become one of the most promis-ing aspects for next generation wireless networks. This paperproposes a system called Opportunistic Multi-MAC Aggregation(OMMA) which enables aggregation of multiple radio accesstechnologies below the IP layer. The problem of optimal packetscheduling to split the incoming IP traffic across the RATsis considered so as to minimize average per packet latency,minimize out-of-order packet reception at the receiver and thusmaximize throughput. We present a theoretical framework toobtain the optimal packet distribution over multiple RATs. Wealso propose a packet scheduling algorithm, called OMMA LeakyBucket, to achieve the optimal packet distribution scheme. A briefdescription of the OMMA system architecture is also presentedwhich includes functional description, discovery and associationprocess between multi-RAT devices and dynamic RAT updatemanagement. We finally present simulation results which showsignificant performance gain with the proposed OMMA LeakyBucket scheme compared to some existing mechanisms.

I. INTRODUCTION

The widespread use of multi-RAT capable devices has at-tracted many researchers from academia and industry towardsthe concept of multi-RAT aggregation. Simultaneous use ofmultiple RATs is a viable solution to improve throughput. Thedifferent types of RATs available in a single wireless device areIEEE 802.11 based Wi-Fi RATs like IEEE 802.11n, cellulartechnologies like UMTS/WCDMA, HSPA, CDMA20001x-EVDO, WIMAX, LTE, GSM. Bandwidth aggregation acrossmultiple RATs could be implemented at different layers suchas at the application layer, transport layer, or between IP andMAC layers. The aggregation solutions at transport or applica-tion layers may not be very efficient in terms of performancesince it is difficult for these aggregation schemes to effectivelywork in an environment with varying channel conditions due tothe lack of instantaneous channel information at these layers.Aggregation schemes at a layer between IP and MAC is morepromising when feedback information from MAC about theinstantaneous channel conditions of the RAT is available.

One of the bandwidth aggregation systems proposed in[1] and [2] proposes aggregation at a layer between IP andMAC called Generic Link Layer (GLL). GLL is responsiblefor multi-radio cooperation, which integrates different RATsat the link layer and efficiently maps user service demandsto multiple radio access networks. These papers on GLLinvestigated multi-radio transmission diversity and multi-radiomulti-hop schemes. In [3], a mechanism to switch radio

resources between different available RATs in a multi-RATdevice is given. A controller (Digital Unit Controller) monitorsthe entire resource utilization for all available RATs. If in anyRAT, average packet loss rate and average channel utilizationare not sufficient to fulfill the QoS requirement, it borrowsresources from another RAT which has unused resources.If no such RAT is available then traffic will be dropped.On the standardization size, IEEE 802 HetNet group hasstarted working on a Open Mobile Network Interface (OMNI),a common module below IP layer enabling simultaneousoperation of any IEEE 802 access technology [4].

Besides work on the system design, there is an increasingnumber of research papers on the topic of data allocation overmulti-RAT systems. Recent work in [5] gives the analysisand survey on aggregation schemes over multi-RAT systems.Aggregation at transport layer described in [6][7][8][9] ag-gregate multipath TCP traffic of multi IP flows over non-contiguous and contiguous frequency bands. In [10], DanluZhang et al proposed a load balancing scheme based onqueuing theory when aggregating traffic between RATs. [11]presents the resource allocation schemes over multiple RATsfor voice and video communication sevices, which maxi-mizes network capacity while maintaining the requirementsfor individual users’ quality of service. In [12], YohsukeKon et al propose an autonomous parameter optimizationscheme using a machine learning algorithm to maximize thethroughput of the heterogeneous RAN aggregation system.Shiwen Mao et al, in [13], proposed an analytical frame-work to minimize the end to end delay on general wirelessmulti path aggregation systems for realtime multimedia traffic.This delay includes the delay along the paths and the re-sequencing delay at the receiver. Georgios P. Koudouridis et alin [14] evaluated, through simulations, the effects of differentMulti-Radio Transmission Diversity schemes for TCP flowsover heterogeneous radio links. Results indicate that MRTDschemes provide substantial gains in terms of goodput andshow a significant reduction in file download times. Thesegains are due to the diversity obtained and the suppressionof the unwanted duplicate acknowledgements that frequentlycause degradation in TCP performance over a radio channel.

In our previous work [15], we looked into the analyticalframework for data allocation at the OMMA layer for singleAP, single STA with single type of QoS traffic. We investigatedthe problem of minimizing the average packet latency (the sum

of queuing delay and serving delay) under certain constraints.We also proposed an algorithm to compute the ratio of optimalpacket distribution across RATs to be implemented at theopportunistic multi-MAC aggregation (OMMA) layer whichis common to all RATs and resides just below the IP layerbut above the radio protocol stacks as shown in figure 1.Moreover, we statistically characterized the reordering delayfor this system and proposed an algorithm to compute amodified optimal packet distribution ratio which minimizesthe packet end-to-end delay (the sum of average packet latencyand reordering delay).

Fig. 1. Multi-RAT Aggregation using OMMA Layer

In this paper, we propose an analytical framework for thegeneral case where there are multiple STAs in the system andIP traffic corresponds to more than one QoS category. Themain contributions of this paper are as follows:

• Propose an analytical framework for multi-RAT systemsto derive the optimal packet allocation ratio over multipleRATs.

• Propose a smart packet allocation algorithm for multi-RAT systems which maintains the above optimal packetallocation ratio and also minimizes the re-sequencingdelay at the receiver due to packet coming out of order.

• Propose the architecture and functional description of theOMMA system which includes discovery and associationprocess between multi-RAT devices, and, dynamic RATupdate management at OMMA.

• Present simulations results showing the performance ofOMMA system with the proposed algorithms and com-pare it with other schemes.

The rest of the paper is organized as follows. Section IIdescribes the system model. Section III describes the problemstatement. Section IV describes the analytical framework foroptimal packet allocation over multiple RATs. Section Vpresents a packet scheduling algorithm for minimizing the re-sequencing delay. Section VI presents the architecture withfunctional design of the OMMA system. Section VII describesthe flow management at both OMMA Sender and Receiver for

IP packets. Section VIII presents OMMA simulation resultsand analysis. Conclusions of this paper are presented inSection IX.

II. SYSTEM MODEL

The wireless system under consideration is a Wi-Fi system.The system consists of an access point (AP), and a numberof Wi-Fi stations (STAs). AP and STAs have the capabilityof supporting multiple RATs (say K RATs), where all RATsoperate on different spectral bands which are orthogonal andsignals on different bands do not interfere with each other. TheRATs belong to the IEEE 802.11 protocol suite i.e. 802.11n,802.11ac, etc. A common layer called OMMA resides belowthe IP layer but above the protocol stacks of all RATs. At theAP, a stream of incoming IP packets arrive at an average rateof λ (packets/unit time)at the OMMA layer. This incoming IPpacket stream is then split by the OMMA layer into K sub-streams each of which is assigned to a corresponding transmitbuffer in each RAT.

The incoming IP packets at AP may belong to one ofmany different IP QoS classes. Each RAT which supportsEDCA independently performs mapping of IP QoS classes to802.11 QoS classes(access categories). The packets sent fromthe OMMA layer to MAC layers i.e. sub-streams mentionedabove, will be stored in one of four different queues corre-sponding to four ACs in the MAC. Inside each AC queue,there are multiple virtual sub-queues corresponding to eachSTA that AP need to send data to. This queueing structurecould be modeled as a two dimensional queuing system asshown in the figure 2.

Fig. 2. MAC queueing mechanism in EDCA mode

Let Qi,k denote the queue at the AP which stores packetscorresponding to Access Category k to be sent to Station

i. Each queue Qi,k is modeled as M/G/1 queue with thefollowing assumptions:

• Arriving IP packets follow the Poisson process.• Serving time of IP packets follows the general distri-

bution. This is defined as the contention time of theCSMA/CA process plus the transmission time (includingretransmissions if required) for the packet confirmed tobe sent out successfully.

• Packet are served in the order they arrive in the queuei.e. first-in-first-out (FIFO).

• Serving time of IP packet is assumed to be identicallydistributed, mutually independent and independent of theinter-arrival time.

When data from different ACs need to be transmitted byMAC, the CSMA/CA process of each AC works in parallel tocontend for the channel. All ACs has a priorities assignedto them for sending out data so as to support the QoSrequirements of different types of traffic. The priority orderof ACs is enabled by setting different parameters of theCSMA/CA processes. The AC, which wins the contentionprocess will get the channel to send out the data from itsown virtual queues. Packets in the other ACs will remain intheir respective queues for this duration, and then resume theirown CSMA/CA processes after the current transmission iscompleted. Assume AC k is the winner and currently contendsthe channel. Inside this AC, there are N independent virtualqueues, which store data packets for N different STAs. APhas different mechanisms to select the STA for serving at thistime. If the virtual queue to be scheduled is Qi,k, AP will onlysend out data corresponding to STA i and AC k. This channelaccess duration may be used to transmit one packet or multiplepackets for the queue. After this transmission is completed, theCSMA/CA processes of all the ACs are resumed. The next ACto send out data depends on which virtual CSMA/CA winschannel access. When the channel access turn returns to AC kagain, the next Station i+ 1 will be served if they follow theRound-Robin Scheme. In other words, packets of all virtualqueues corresponding to different STAs need to be served atleast once before the first STA is served again. The detail ofthis process in the EDCA mode could be found in [16].

Because of this queueing system in the EDCA mode, eachqueue Qi,k can be modeled as M/G/1 queue with vacations.The vacation time with the queue Qi,k is the duration APserves other STAs or AP sending data of other ACs.

III. PROBLEM STATEMENT

In this paper, the problem of packet scheduling of IP trafficover multiple RATs in a multi-RAT device is considered. Asdescribed in section II, the main IP stream is split into K sub-streams each of which is assigned to a corresponding transmitbuffer in each RAT.

When a system with a single STA, single AP and singleIP flow from AP to STA is considered, the challenge is todetermine how to optimally distribute packets correspondingto the IP flow across RATs such that the average end-to-end delay per packet is minimized. Placing all packets in

the transmit buffer of the RAT with the lowest latency mayincrease the average packet queuing delay. On the other hand,dispersing them across all RATs randomly may decrease theaverage packet queuing delay and serving delay, however itmay result in out-of-order reception of packets at the receiverdue to differences in link latencies. This can cause longerqueuing delays at the receiver to rearrange packets beforesending them up to the IP layer. A smart packet assignmentstrategy to minimize both average end-to-end packet latencyand out-of-order packet reception delay, and thus maximizethroughput is essential.

In a system with multiple STAs, single AP and multipleIP flows from AP to each STA, although sender IP packetsare forwarded to the sender MAC layer as a single streamby OMMA module, MAC layer sorts them according to theircorresponding access categories and UT addresses. The mainIP stream for each STA is split into multiple QoS streamsthat are queued in separate buffers in the sender MAC layer.Each buffer corresponds to an access category that has acertain priority. However, this set up complicates the trafficshaping because, for any RAT, the average packet delay notonly depends on the queuing delay due to unserved packetsin the same buffer, but also on the delay due to bufferingand channel access by packets of the other buffers. This isbecause all packets queued in the MAC of a particular RATshare the same MAC scheduler and physical layer. In otherwords, the channel access scheme that is used to schedulemedium access between different AC buffers ties the waitingtimes for all packets in the buffers. Thus a modified packetassignment strategy (compared to the single STA single IPflow case) to minimize average end-to-end packet latency isessential.

IV. OPTIMAL SCHEDULING SCHEME AT OMMA LAYER

A. Terminology and Assumptions

We use the following terminology shown in Table I below.Note that the analytical framework developed in this sectionis for the general case of scheduling data corresponding to ACk from AP to STA i. To keep the notation simple, we omitthe subscripts i and k. The terms defined below correspond toRAT index j where (1 6 j 6 M ) and M is the total numberof RATs at AP.

Note that:

• Vj is defined as the average time duration that RAT j ofAP stops serving queue Qi,k and serves queues belongingto other STAs ( ̸= j) or other ACs (̸= k).

• V 2j is the second moment of the average vacation time

of queue Qi,k at RAT j.• T 2

j is the average second moment of serving time at RATj of AP.

B. M/G/1 Queing Model with vacations

We first look at the queueing model of queue Qi,k. Theaverage service time of one packet sent over RAT j is the

λj Average arrival rate of IP packets at RAT j at AP

µj Average serving rate of RAT j at AP

ρj Fraction of arrival rate and service rate λj

µj

Xj Average packet serving time of RAT j at AP

Tj Average packet delay at MAC layer of AP

Wj Average packet queuing delay at queue Qi,k of AP

Vj Average vacation time of the queue Qi,k

TABLE INOTATION FOR ACCESS CATEGORY k TOWARD STATION i

inverse of service rate, which is shown in Equation 1:

Xj = E{Xj} =1

µj(1)

The average second moment of the service time of packetssent over RAT j could be written as:

X2j = E{X2

j } (2)

We model queue Qi,k as an M/G/1 queue with vacations.Using the derivation and proof of Pollaczek-Khinchin(P-K)formula for this model as shown in [17], the average per packetdelay at queue Qi,k of AP corresponding to RAT j can bewritten as:

Wj =λX2

j

2(1− ρj)+

V 2j

2Vj

(3)

The average delay of one packet in a system is defined asthe sum of queuing delay and serving delay. Since Wj is theaverage queuing delay at MAC layer of AP for RAT j, Xj isthe average serving time at AP for RAT j, the total delay foreach packet at MAC layer of AP for RAT j is given by:

Tj = Wj +Xj =λjX2

j

2(1− λj

µj)+

V 2j

2Vj

+Xj (4)

Note that, for each queue Qi,k, the parametersXj , X2

j , µ, Vj , V 2j could be measured and fed back by

RAT j to OMMA layer. So, if the arrival rate λj is known,the average packet delay Tj of the Access Category k towardStation i could be calculated by Equation 4.

C. Optimization Problem Statement

In this section, we formulate the optimization problem tofind the optimal scheme at OMMA layer to distribute theincoming IP traffic corresponding to Access Category k towardStation in across multiple RATs. Assuming there are M RATsat AP to send data to N different Stations. Data sent fromqueue Qi,k of AP needs to be scheduled to be sent out on M

separate RATs. We use subscript j in the following Equationsto indicate RAT index j.

The summation of all arrival rates of IP packets, correspond-ing to Access Category k toward Station i, from OMMA layerinto each of the RATs is equal to the IP packet arrival rateinto OMMA.

λ =M∑j=1

λj (5)

To ensure that the queues do not overflow, we impose thefollowing constraint.

λj < µj for 1 6 j 6 M (6)

Since Tj , shown in Equation 4, is the average packet delayat MAC layer of RAT j at AP, the average packet delay overall M RATs is the weighted average delay of all Tj for 1 6j 6 M . The weighting factor for each RAT j is the ratio ofthe packet arrival rate on RAT j to the total packet arrival rateinto the OMMA layer i.e. λj

λ . Thus the average packet delayover all M RATs is:

F =

∑Mj=1

((

λjX2j

2(1−λjµj

)+

V 2j

2Vj+ 1

µj) ∗ λj

)λ

(7)

This expression for the average packet delay over all MRATs is the objective function that need to be minimized.The optimization problem can now be stated as follow:

Minimize F =

∑Mj=1

((λjX2

j

2(1−λjµj

)+

V 2j

2Vj+ 1

µj

)∗ λj

)λ

(8)

Subject to:

∑Mj=1 λj = λ

λj > 0 for 1 6 j 6 M

−λj > −µj for 1 6 j 6 M

(9)

D. The convexity of the objective function

In this section, we prove that F (λ1, λ2, ..., λM ) is a convexfunction. We can rewrite F as F =

∑Mj=1 f(λj), where:

f(λj) =

(λjX2

j

2(1−λjµj

)+

V 2j

2Vj+ 1

µj

)∗ λj

λ(10)

To prove that F is convex, it should be sufficient to provethat f(λj) is convex. The second derivative of f(λj) is:

∂2f

∂λ2j

= −µ3jX

2j

(λj − µj)3 ∗ λ(11)

In Equation 11, all of the variables are positive. Sinceλj < µj for 1 6 j 6 M based on constrain 9 of theoptimization problem, the second derivatives ∂2f

∂λ2j

for 1 6j 6 M are always positive for any value of λj . Since f(λj)has positive second derivative, it is strictly convex and so isF (λ1, λ2, ..., λM ).

E. The Lagrangian optimization method

Since F (λ1, λ2, ..., λM ) is a convex function, we can usethe Lagrangian optimization method to solve the optimizationproblem in equations 8 and 9. The Lagrangian function canbe written as

L(λ, γ, β, δ) = F (λ1, λ2, ..., λM )− γ ∗ (M∑j=1

λj − λ)−

M∑j=1

βj ∗ (λj)−M∑j=1

δj ∗ (−λj + µj) for 1 6 j 6 M (12)

In this Lagrangian function, λj for 1 6 j 6 M are the un-known variables. γ, βj , δj for 1 6 j 6 M are the Lagrangianmultipliers. Since the objective function F is a convex func-tion, there is an optimal solution set (λ∗

j , γ∗, β∗

j , δ∗j ), for 1 6

j 6 M for the optimization problem (using the result of theKarush-Kuhn-Tucker conditions)[18].

The optimal solution has to satisfy the following set ofequations:

∂F∂λj

− γ − βj + δj = 0 for 1 6 j 6 M (a)

γ ∗ (∑M

j=1 λj − λ) = 0 (b)

βj ∗ λj = 0 for 1 6 j 6 M (c)

δj ∗ (µj − λj) = 0 for 1 6 j 6 M (d)

(13)

In the equation set 13, there are totaly 3M+1 formulations.We also have 3M + 1 variables in the equation set whichincludes the solution set Λ∗ = (λ∗

1, λ∗2, ..., λ

∗M ), the La-

grangian variable sets γ∗ and vectors β∗ = (β∗1 , β

∗2 , ..., β

∗M ),

δ∗ = (δ∗1 , δ∗2 , ..., δ

∗M ). It is feasible to solve the equations and

find a unique solution set Λ∗ = (λ∗1, λ

∗2, ..., λ

∗M ).

From the equation 13(c), since λj > 0, we have βj =0 for 1 6 j 6 M .

From the equation 13(d), since µj > λj , we have δj =0 for 1 6 j 6 M .

Also λj < µj for 1 6 j 6 M , implies that the arrival rate issmaller than the equivalent service rate. So the total incomingtraffic will always be served optimally such that

∑Mj=1 λj = λ.

Substituting these values into equation 13, we get:

∂F∂λj

− γ = 0 for 1 6 j 6 M (a)

∑Mj=1 λj − λ = 0 (b)

βj = 0 for 1 6 j 6 M (c)

δj = 0 for 1 6 j 6 M (d)

(14)

Equation 14(a) equals to:

(µjV 2j + 2Vj − µ2

jVjX2j − 2µjVjλγ) ∗ λ2

j+

(−2µ2jV

2j + 2µ3

jVjX2j − 4µjVj + 4µ2

jVjλγ) ∗ λj+

(µ3jV

2j + 2µ2

jVj − 2µ3jVjλγ) = 0 (15)

The solutions for equation 15 are:

λ∗j =

µj ±µ2j

√VjX2

jõ2jVjX2

j − µjV 2j + (2λγµj − 2)Vj

(16)

Since the objective function is strictly convex, it has aunique non-negative globally optimal solution λj , 1 6 j 6 M .Note that in equation 16, the solution λj still depends on theunknown variable γ.

Substituting these values of λj into equation 13(b), we get:

M∑j=1

µj ±µ2j

√VjX2

jõ2jVjX2

j − µjV 2j + (2λγµj − 2)Vj

= λ

(17)where γ is the only unknown variable. This is equivalent

to:

M∑j=1

∓ 1√2λµ3jγ +

(1µ2j− V 2

j

µ3jVjX2

j

− 2

µ4jX

2j

) =

M∑j=1

µj − λ

(18)Assume 2λ ≫ µj for 1 6 j 6 M , then

we got(

1µ2j− V 2

j

µ3jVjX2

j

− 2

µ4jX

2j

)≪

(2λµ3j

). The term(

1µ2j− V 2

j

µ3jVjX2

j

− 2

µ4jX

2j

)in equation 18 becomes negligible.

Equation 18 could be approximated as:

M∑j=1

∓ 1√2λµ3jγ

≈M∑j=1

µj − λ (19)

So finally we get:

γ ≈

(∑Mj=1 ∓µ

32j

)22λ(∑M

j=1 µj − λ)2 (20)

Substituting the value of γ from equation 20 into equation16, we get the values of λ∗

i for 1 6 j 6 M shown in equation21.

λ∗j = µj±

µ2j

√VjX2

j√√√√√µ2jVjX2

j − µjV 2j +

(∑Mj=1 ∓µ

32j

)2

(∑M

j=1 µj−λ)2 ∗ µj − 2

∗ Vj

(21)Note that we need to try upto 2M different combinations

of the sign ∓ in equation 21 to find the solutions λ∗j . Each

candidate solution set Λ = (λ1, λ2, ..., λM ) should be checkedwith the constraint set 9. Once we find a local minimumsolution,we could stop the trial process since the objectivefunction is strictly convex, as proved in section IV-D, and sothe discovered local minimum is also the global minimumsolution Λ∗.

V. PACKET FLOW CONTROL

In multi-RAT systems such as OMMA, which performaggregation on a packet basis, re-sequencing delay is a criticalfactor that needs to be addressed. Re-sequencing delay foreach packet i is defined as the time packet i has to wait at theOMMA layer of the receiver node for all of the earlier packetscoming to the receiver (all packets with the order smaller thani coming successfully to the OMMA layer of the receiver). Ithappens when data packets are received out of order due topackets traversing multiple links, each with different packetlatency. At the transmitter side, packets of the main stream aresplit into multiple sub streams for transmission over differentlinks which may possibly have different latencies. As OMMAlayer at the receiver receives packets one after the other, itsends packets to the IP layer in an ordered fashion. Thisrequires that some packets to be held at OMMA for re-sequencing purposes, which incurs re-sequencing delay. Thisre-sequencing problem has a severe impact on both UDP andTCP applications. For example, the QoS of real-time UDPapplications like voice over IP or live video streaming couldsuffer. That is because out of order packets would be countedas lost packets and get ignored at the receiver side. For TCP,it is even more serious because packets out of order couldgenerate the duplicate ACK issue, which triggers unnecessarycongestion control mechanism that reduces the end to endthroughput.

Therefore, an optimal packet assignment strategy at theOMMA transmitter side would strive to maintain the correctordering of packet reception at the receiving end such thatthe ordering delay is minimized. The following proposedAlgorithm 1 will minimize the average re-sequencing delayper packet for multi RAT aggregations.

In this algorithm we maintain M token variables, one foreach RAT. Initially, the token for each RAT is assigned tobe 0. Token for each RAT j is incremented iteratively byλ∗j

µj, where λ∗

j is the optimal rate for RAT i calculated by the

Algorithm 1 OMMA Leaky Bucket Algorithm1: for j = 1 to M do2: Tj ⇐ 03: end for4: while Unscheduled packets set ̸= ∅ do5: for j = 1 to M do6: Update µj from Meta Data Feedback of MAC layer7: Update λ∗

j by equation 21

8: Rj ⇐λ∗j

µj

9: end for10: while ∀Tj < 1, 1 6 j 6 M do11: for j = 1 to M do12: Tj ⇐ Tj +Rj

13: end for14: end while15: find i with Ti =max T {T1, T2, ..., TM}16: send current packet on RAT i17: Ti ⇐ Ti − 118: end while

minimum delay algorithm presented in the last section, till atleast one of the tokens exceeds 1. The RAT corresponding tothe token which exceeds 1 is chosen to send the next incomingunscheduled packet at OMMA. Then this token is decrementedby 1 and the process of incrementing tokens is continued asbefore. This algorithm is run “ahead” of every packet arrivingat OMMA, i.e. OMMA always knows which packet ID will bescheduled on which RAT. Since the λ∗

j values are chosen so asto minimize average system delay per packet, this algorithmensures that the RAT chosen to send each packet is such thatthe packet experiences the minimum delay and also arrives inthe correct order with respect to its preceding and succeedingpackets at the receiver.

VI. OMMA ARCHITECTURE

This section describes the architecture of the OMMA layerwith description of its main functional modules. We alsoprovide the details of some key operations performed atOMMA which are required to support multi-RAT aggregation.

A. Functional Description

A high level architectural view of OMMA is shown in figure3. It includes all functional blocks of OMMA layer includingthe main interfaces for control signaling and data signalling.OMMA layer consists of the following main functional blocks.

1) STA RAT Capability Database: At the AP, STA RATcapability database is used to store RAT capability information(i.e. list of all common RATs) for each of its associated STAs.Moreover, because of poor link quality due to interference ormobility, a subset of RATs can be unavailable for a STA.So this database also stores a list of available RATs at agiven time for each associated STA. This information ofRAT capability and available RATs is updated by OMMAController as described later.

Fig. 3. Block Diagram of the OMMA Layer

2) OMMA Controller: OMMA controller is responsiblefor updating the STA RAT Capability Database either incase of newly associated STAs or change in availability ofRATs for already associated STA. OMMA Controller alsoreceives feedback metrics µj , Vj , V 2

j and X2j from each

RAT j (1 6 j 6 M ) corresponding to AC k from AP toSTA i for that RAT. It then classifies those metrics basedon the STA ID i and QoS class k, and sends them toOMMA Schedulers of corresponding STAs. It also calculatesthe arrival rate λj (corresponding to AC k from AP to STAi) of incoming IP packet and provides this information tothe OMMA Scheduler as one of the parameters required foroptimal multi-RAT aggregation. Moreover, OMMA controlleralso provides system parameters (e.g. number of RATs, typeof RATs, matched set of RATs with STAs to be associated)during discovery and association process as described later.

3) OMMA Scheduler: OMMA layer maintains a separateOMMA Scheduler module corresponding to each associatedSTA and each QoS class supported by the system. OMMAlayer also maintains an IP Packet STA Classifier module andalso a STA QoS Classifier to read the IP packet header andsend it to corresponding OMMA Scheduler module for furtherprocessing. OMMA Scheduler communicates with STA RATCapability Database to extract list of available RATs for a STA.It also selects RATs based on the feedback metrics providedby OMMA Controller and list of available RATs for that STAprovided by STA RAT capability database. On the transmitterside, it distributes packets across selected RATs based on agiven packet assignment scheme. On the receiver side, OMMAscheduler is responsible for aggregating packets received from

RATs and sending them to IP layer.

B. Key operations at OMMA

This section describes some key operations those are im-portant to enable communication between multi-RAT devices.

1) RAT Capability Discovery: Each multi-RAT device canhave a different RAT capability (i.e. set of supported RATs).This generates the need for a discovery and association processin which a device (STA/AP) can advertise its RAT capabilityparameters (i.e. Number of RATs, Type of RATs, etc.) to otherdevices. This way, a STA and an AP can associate with eachother on the set of RATs common to them.

The AP can advertise its RAT capabilities either in the Bea-con (in the passive scanning mode) or in the Probe Response(in the active scanning mode) which is generated in response toa Probe Request from the STA. Beacon is sent on all availableRATs at the AP while Probe Response is sent on the same RATon which Probe Request was received. STA, which receivesAP’s RAT capabilities, selects the set of RATs common toitself and the AP with the help of OMMA Controller. STAsignals the set of common RATs in the Association Requestmessage sent to the AP on every common RAT. AP stores theinformation of RAT capabilities of the STA into its STA RATCapability database.

2) OMMA Mode Selection: This procedure is requiredto decide the mode of operation of the OMMA Schedulerat both sender and receiver. The modes of operation couldeither be based on a pre-defined set of policies for every IPflow, or could be based on feedback parameters received byOMMA from each RAT. Some examples of OMMA modesare described in section VIII (referred to as packet schedulingschemes). At AP, OMMA Controller takes mode selection de-cision and signals this decision to OMMA Scheduler. OMMAScheduler enables/disables sending packets on certain RATsbased on the mode decision. Furthermore, OMMA Controllerat AP also sends mode decision to OMMA receiver at STAusing one of the available RATs for that STA. At STA,mode information received in Beacon is signaled to OMMAController, which in turn configures the OMMA Scheduleraccordingly.

3) RAT Availability Update Management: Since thewireless link on each RAT may have variable link qualityparameters such as packet loss rate, jitter due to factors suchas interference, mobility, some of the RATs common betweenthe UTs may be useable while the others may not be usable.Thus the AP transmitting data to STA may not be aware ofwhich RATs are usable at any given time.

So a procedure for dynamic management of RAT availabil-ity for every STA-AP pair is required. AP sends Beacons on allits RATs periodically. The STA reads the beacon informationon all RATs common to itself and the AP. If the STA is ableto read the beacon information successfully on any RAT, itidentifies that RAT as being available and assigns a value ’1’to that RAT. But if the STA is unable to read the beaconinformation successfully on any RAT, it identifies that RAT asbeing unavailable and assigns a value ’0’ to that RAT. Thus

the OMMA Controller at the STA generates a binary vectorof length equal to the number of RATs common to itself andthe AP.

The STA periodically sends this RAT availability binaryvector to the AP using one of the common RATs. Thisinformation is sent from the RAT to the OMMA Controllerwhich in turn stores it in the STA RAT capability database. Inthis way, the RAT Availability information of any Station-APpair is periodically refreshed.

C. IP Flow Management at OMMA

1) OMMA Sender Operation: OMMA sender takes thedecision of RAT selection to send the incoming IP packets.Procedure for routing of incoming packets to subset of RATsis described below.

1) At OMMA, incoming packet is delivered to IP PacketSTA Classifier and OMMA Controller both,

2) IP Packet STA Classifier sends packet to OMMA Sched-uler corresponding to its destined STA,

3) Scheduler makes the decision on RAT/RATs selectionby using, (1) RAT availability provided by STA RATCapability Database,(2) Feedback metrics (arrival rate,serving rate and average packet delay) provided fromOMMA Controller,

• It selects all the RATs which fulfill the minimumrequirement of QoS class of incoming packet

• In case of starting phase, when there is no feedbackis available, it chooses all the available RATs forthat STA

4) Then Scheduler distributes all the packets on the selectedRATs based on the algorithm described in next section,

5) RAT switch happens when one of the RATs of multipleselected RATs is not able to fulfill the requirement ofgiven QoS class. In this situation, it chooses randomlya RAT from the set of other available RATs those arenot currently chosen for that STA.

2) OMMA Receiver Operation: At OMMA receiver,OMMA maintains separate IP Packet STA Classifier for eachRAT. Each IP Packet STA Classifier reads the packet headerand sends it to OMMA Scheduler corresponding to that STA.OMMA scheduler aggregates the data packets received frommultiple RATs and sends them to IP layer.

VII. IP FLOW MANAGEMENT AT OMMA

This section describes the flow management at both OMMASender and Receiver for incoming IP packets. Procedure ofRAT selection for incoming IP packets at OMMA Sender isdescribed. We also describe the operations at OMMA Receiverrequire to send IP packets (i.e. received from multiple RATs)to IP layer. In this work all IP packets are taken of single QoSclass.

A. OMMA Sender Operation

A high level view of OMMA sender is shown in figure 4.OMMA sender takes the decision of RAT selection to send

Fig. 4. OMMA Sender

the incoming IP packets. Procedure for routing of incomingpackets to subset of RATs is described below.

1) At OMMA, incoming packet is delivered to IP PacketSTA Classifier and OMMA Controller both,

2) IP Packet STA Classifier sends packet to OMMA Sched-uler corresponding to its destined STA,

3) Scheduler makes the decision on RAT/RATs selectionby using, (1) RAT availability provided by STA RATCapability Database,(2) Feedback metrics (arrival rate,serving rate and average packet delay) provided fromOMMA Controller,

• It selects all the RATs which fulfill the minimumrequirement of QoS class of incoming packet

• In case of starting phase, when there is no feedbackis available, it chooses all the available RATs forthat STA

4) Then Scheduler distributes all the packets on the selectedRATs based on the algorithm described in next section,

5) RAT switch happens when one of the RATs of multipleselected RATs is not able to fulfill the requirement ofgiven QoS class. In this situation, it chooses randomlya RAT from the set of other available RATs those arenot currently chosen for that STA.

B. OMMA Receiver Operation

A high level view of OMMA receiver is shown in figure5. At OMMA receiver, OMMA maintains separate IP PacketSTA Classifier for each RAT. Each IP Packet STA Classifierreads the packet header and sends it to OMMA Schedulercorresponding to that STA. OMMA scheduler aggregates thedata packets received from multiple RATs and sends them toIP layer.

Fig. 5. OMMA Receiver

VIII. PERFORMANCE EVALUATION

Fig. 6. Simulation Setup

We simulate a scenario with single server sending datato a single Client over an IP backhaul network and throughthe AP as in figure 6 using OPNET V16.0 simulator. Downlink data sent from the server is set as best effort traffic(all data belongs to a single QoS class). Both AP and STAare capable of supporting two RATs i.e. IEEE 802.11n RAToperating on 2.4GHz ISM band over single 20MHz channel,and, a proprietary RAT which is a modified IEEE 802.11nRAT operating on the TVWS band and aggregates four TVWSchannels(5MHz/channel) at the MAC layer. The OMMA layerimplemented on top of the two RATs and below IP layer of theAP and Client is responsible for splitting traffic at the senderside and aggregating at the receiver. When down link IP trafficreaches the AP, the OMMA layer at the AP either sends alltraffic over one RAT or distributes the traffic across two RATsto send it to the Client using one of the schemes describedbellow. Server sends a new file to the Client by settingup a new TCP connection. Multiple TCP connections aremaintained during the simulation time. When a file downloadis completed, the corresponding TCP connection is terminated.We evaluate the performance of several packet allocationschemes at OMMA by analyzing the simulation results withseveral parameters. The packet scheduling schemes used atOMMA layer are:

• Single ISM band : AP sends all the data to STA over

the RAT operating on ISM band only. RAT operating onTVWS band is disabled.

• Single TVWS band : AP sends all the data to STA overthe RAT operating on TVWS band only. RAT on ISMband is disabled.

• 50:50 traffic split : 50% of the incoming IP packets at APare sent over the RAT operating on ISM band while theother 50% of the packets are sent over RAT operating onTVWS band. Packets are assigned to RATs sequentiallyas they arrive at the AP(regardless of the packet ID).

• Load Balancing : Incoming IP packets are assigned tothe two RATs in the ratio of the serving rates of RATs.Packets are assigned to RATs sequentially as they arriveat the AP(regardless of the packet ID).

• RAT selection per TCP flow : Packets of a single TCPflow assigned to the same RAT. RAT switching followsRound-Robin scheme.

• Minimum delay : Incoming IP packets are assigned tothe two RATs in the ratio of the optimal packet distri-bution scheme(x∗

i ) determined based on minimization ofthe average delay per packet as presented in section IV.Packets are assigned to RATs sequentially as they arriveat the AP(regardless of the packet ID) but still maintainthe optimal packet distribution ratio.

• Leaky bucket : Incoming IP packets are assigned to thetwo RATs in the ratio of the optimal packet distributionscheme(x∗

i ) determined based on minimization of theaverage delay per packet as presented in section IV.However, packets smartly assigned to RATs with OMMALeaky Bucket technique to minimize both per-packetdelay and out-of-order packet reception as presented insection V.

We set the thermal noise of the radio front-end of eachRAT such that average SNR at the Client equals 10dB onISM band and 20dB on TVWS band. Every 30s, server sendsa new file to client, which creates a new TCP connection. Weadjust the file size such that the throughput would be optimalfor each scheduling scheme. The value of file size determinesthe network loads from application layer to transport layer.The network load value for each scheme that we simulated isshowed in table II.

No Scheme Offered Load (Mbps)1 Single ISM band 102 Single TVWS band 17.53 50:50 traffic split 184 RAT selection per TCP flow 165 Load Balancing 276 Minimum delay 277 Leaky bucket 27

TABLE IIOFFER LOAD FOR EACH SCHEME IN SMALL RTT SCENARIO

A. Small RTT

We set the negligible packet latency at IP cloud module,which leads to average end to end round trip time (RTT)

0 100 200 300 400 500 6000

5

10

15

20

25

30

35

40

Simulation Time(Seconds)

Ave

rage

TC

P T

hrou

ghpu

t

ISM Band OnlyTVWS Band Only50% − 50%Load BalancingMinimum DelayLeaky BucketRAT Switching Per TCP Flow

(a) Average TCP Throughput

100 200 300 400 500 6000

0.05

0.1

0.15

0.2

0.25

0.3

0.35

Simulation Time(Seconds)

Inst

anta

neou

s P

acke

t Lat

ency

(S

econ

ds)

ISM Band OnlyTVWS Band Only50% − 50%Load BalancingMinimum DelayLeaky BucketRAT Switching Per TCP Flow

(b) Instantaneous Packet Latencies

Fig. 7. Performance Comparisons with small End to End Round Trip Time

0 100 200 300 400 500 6000

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

Simulation Time (Seconds)

Inst

anta

neou

s N

umbe

r of

Ret

rans

mis

sion

s

ISM Band OnlyTVWS Band Only50% − 50%Load BalancingMinimum DelayLeaky BucketRAT Switching Per TCP Flow

Fig. 8. Instantaneous Number of Retransmissions with small RTT

as small as 50ms with aggregation schemes and 100ms withsingle RAT schemes. The average MAC latency differencebetween two RATs is around 15ms. So the MAC latencydifferent between RATs are equivalent to around 15% RTTto 30% of the aggregation schemes.

Figure 7 presents the performance comparison with the op-timal network load for each scheme. In figure 7(a) presents theaverage throughput comparison between the packet allocationschemes. Throughput on TVWS band only scheme is clearlyoutperform ISM Band only by around 170%. This due to thefact that noise level on TVWS RAT is 20dB while it is 10dBon ISM band. It shows on the packet latency graph on figure

7(b) with both of this RATs suffer high packet latency at thebeginning of the simulation but converged later. ISM bandonly suffers with higher packet latency, takes longer time toconverged and converged at higher latency. TCP flow controlneed more time to absorb the network load on higher noiseISM band.

It is also interesting to see in figure 7(a) that 50% - 50%scheme has almost the same throughput compare to the singleTVWS band scheme. RAT switching per TCP flow schemehas even worse performance compare to the single TVWSband scheme. This due to the fact that 50% - 50% and RATswitching per TCP flow schemes do not adapt with the unequalchannel qualities on RATs. RAT switching per TCP flow sendsevery packet of the same TCP flow on the same RAT, so donot suffer the packet out of out of order at the receiving side.That lead less number of packet retransmission as showed infigure 8. However, since this scheduling scheme change RATfor each TCP flow follow the Round Robin method, that makethe throughput split equally on each RAT and suffer higheraverage delay as showed in figure 7(b).

The aggregation schemes as Load Balancing, MinimumDelay and Leaky Bucket have good performance with LeakyBucket get the highest result. This schemes have the adaptivescheduling schemes with the channel qualities on each RAT,so get lower packet latency compare to the above mentionedschemes. Leaky bucket having slightly lower number of re-transmissions than the other two when the curves convergeddue to per packet smart scheduling helps to reduce the re-ordering delay. That leads to less number of duplicate ACKsand retransmission.

B. High RTT

We then adjust higher packet delay at the IP cloud modulewith the values uniformly distributed in the range from 80ms

0 100 200 300 400 500 6000

5

10

15

20

25

30

35

Simulation Time (Seconds)

Ave

rage

TC

P T

hrou

ghpu

t (M

bps)

ISM Band OnlyTVWS Band Only50% − 50%Load BalancingMinimum DelayLeaky BucketRAT Switching Per TCP Flow

(a) Average TCP Throughput

0 100 200 300 400 500 6000.1

0.15

0.2

0.25

0.3

0.35

Simulation Time (Seconds)

Inst

anta

neou

s P

acke

t Lat

ency

(S

econ

ds)

ISM Band OnlyTVWS Band Only50% − 50%Load BalancingMinimum DelayLeaky BucketRAT Switching Per TCP Flow

(b) Instantaneous Packet Latencies

Fig. 9. Performance Comparisons with long RTT scenario

0 100 200 300 400 500 6000

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

Simulation Time (Seconds)

Inst

anta

neou

s N

umbe

r of

Ret

rans

mis

sion

s

ISM Band OnlyTVWS Band Only50% − 50%Load BalancingMinimum DelayLeaky BucketRAT Switching Per TCP Flow

Fig. 10. Instantaneous Number of Retransmissions with long RTT

to 120ms. This lead to the end to end round trip time of theaggregation scheme increased to around 250ms. This is themost popular end to end round trip time based on the ITUstandard for delay sensitive service like voice or live streamingvideo. The average MAC latency difference between two RATsis reduced to around 3ms. MAC latency different betweenRATs are equivalent to around 1.2% RTT of the aggregationschemes.

The simulation results in this scenario are showed in figure9. The conclusions of the previous case still hold in this casealthough the performances are all reduced because of longerthe end to end RTT. TVWS band only scheme is always

outperform ISM Band only. Both schemes suffer high packetlatency at the beginning of the simulation but converged later.However, ISM band only suffers much longer before it couldconverged due to longer RTT makes bigger cost for retransmitdata on this high noise RAT. TCP flow control need more timeto absorb the network load on this higher noise ISM band.

50% - 50% and RAT switching per TCP flow schemesstill having worse performance compare to the single TVWSband scheme due to they do not adapt with the unequalchannel qualities on RATs. The aggregation schemes as LoadBalancing, Minimum Delay and Leaky Bucket have goodperformance with Leaky Bucket get the highest result sincethis smart scheduling scheme having slightly lower number ofretransmissions.

IX. CONCLUSION

This paper proposed the concept of Opportunistic Multi-MAC Aggregation (OMMA) to enable multi-RAT aggregation.The OMMA layer resides between IP layer and RAT protocolstacks and is responsible for scheduling of IP packets simul-taneously across multiple RATs. We proposed an analyticalframework to derive the optimal packet scheduling strategyover RATs when single AP communicates with multipleSTAs simultaneously with IP traffic belonging to various QoSclasses. This scheme maximizes the aggregated throughputand minimizes the average end-to-end packet latency. We alsoproposed a per-packet scheduling algorithm, called OMMALeaky Bucket, which not only achieves the optimal packetdistribution but also minimizes the packet re-sequencing delayat the receiver. We also provided a description of the OMMAsystem architecture which includes functional design, solutionfor discovery and association process between multi-RATdevices and dynamic RAT update management.

For future work, we will be looking at the problem ofOMMA operation in a wireless multi-cast scenario by lookingat traffic management scheme and packet assignment methodat OMMA to optimize multi-cast throughput and minimizepacket latency.

REFERENCES

[1] G. Koudouridis, R. Agero, E. Alexandri, J. Choque, K. Dimou, H. Arimi,H. Lederer, J. Sachs, and R. Sigle, “Generic link layer functionality formulti-radio access networks,” in IST Mobile and Wireless Communica-tions Summit, 2005.

[2] K. Dimou, R. Agero, M. Bortnik, R. Karimi, G. Koudouridis, S. Kamin-ski, H. Lederer, and J. Sachs, “Generic link layer: a solution for multi-radio transmission diversity in communication networks beyond 3g,” inVehicular Technology Conference, 2005.

[3] X.-D. Trinh, G. Jo, J. Lee, J.-H. Na, W. Park, and H.-S. Cho, “A radio-resource switching scheme in aggregated radio access network,” in TheSeventh International Conference on Digital Telecommunications, 2012.

[4] (2012) Ieee 802.16’s hetnet study group. [Online]. Available:http://www.ieee802.org/16/sg/het/

[5] D. Krishnaswamy, D. Zhang, S. Soliman, B. Mohanty, D. Cavendish,W. Ge, and S. Eravelli, “Concurrent bandwidth aggregation over wirelessnetworks,” in IEEE Intl Conf on Computing, Communications, andNetworking (ICNC), 2012.

[6] H. Han, S. Shakkottai, C. V. Hollot, R. Srikant, and D. Towsley, “Multi-path tcp: a joint congestion control and routing scheme to exploit pathdiversity in the internet,” IEEE/ACM Trans. Netw., vol. 14, no. 6, pp.1260–1271, Dec. 2006.

[7] F. Kelly and T. Voice, “Stability of end-to-end algorithms for jointrouting and rate control,” SIGCOMM Comput. Commun. Rev., vol. 35,no. 2, pp. 5–12, Apr. 2005.

[8] A. Ford, C. Raiciu, M. Handley, and O. Bonaventure, “Tcp extensionsfor multipath operation with multiple addresses,” March 2011.

[9] P. Key, L. Massouli, and D. Towsley, “Multipath routing, congestioncontrol and dynamic load balancing,” 2007, pp. IV–1341–IV–1344.

[10] D. Zhang, P. K. Vitthaladevuni, B. Mohanty, and J. Hou, “Performanceanalysis of dual-carrier hsdpa,” in VTC Spring, 2010, pp. 1–5.

[11] Y. Wu, H. Viswanathan, T. E. Klein, M. Haner, and A. R. Calderbank,“Capacity optimization in networks with heterogeneous radio accesstechnologies,” in GLOBECOM, 2011, pp. 1–5.

[12] Y. Kon, M. Ito, N. Hassel, M. Hasegawa, K. Ishizu, and H. Harada,“Autonomous parameter optimization of a heterogeneous wireless net-work aggregation system using machine learning algorithms.” in CCNC.IEEE, 2012, pp. 894–898.

[13] S. Mao, S. S. Panwar, and Y. T. Hou, “On optimal partitioning ofrealtime traffic over multiple paths,” in INFOCOM, 2005, pp. 2325–2336.

[14] G. P. Koudouridis, A. Yaver, and M. U. Khattak, “Performance evalua-tion of multi-radio transmission diversity for tcp flows,” in VTC Spring,2009.

[15] T. Elkourdi, A. Chincholi, T. Le, and A. Demir, “Cross-layer optimiza-tion for opportunistic multi-mac aggregation,” in Under Submision toVTC, 2013.

[16] P. Eldad and S. Robert, “Throughput, robustness, and reliability in802.11n.” Cambridge University Press, New York: Cambridge Uni-versity Press., 2008.

[17] D. P. Bertsekas and R. Gallager, Data networks (2. ed.). Prentice Hall,1992.

[18] D. A. Pierre, Optimization Theory with Applications. Dover Publica-tions, 1986.