Research Article A Novel Scheduling Index Rule Proposal...

15
Research Article A Novel Scheduling Index Rule Proposal for QoE Maximization in Wireless Networks Ianire Taboada and Fidel Liberal Faculty of Engineering of Bilbao, University of the Basque Country UPV/EHU, Alameda Urquijo s/n, 48013 Bilbao, Spain Correspondence should be addressed to Ianire Taboada; [email protected] Received 9 April 2014; Accepted 9 July 2014; Published 5 August 2014 Academic Editor: Victor Kovtunenko Copyright © 2014 I. Taboada and F. Liberal. is is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. is paper deals with the resource allocation problem aimed at maximizing users’ perception of quality in wireless channels with time-varying capacity. First of all, we model the subjective quality-aware scheduling problem in the framework of Markovian decision processes. en, given that the obtaining of the optimal solution of this model is unachievable, we propose a simple scheduling index rule with closed-form expression by using a methodology based on Whittle approach. Finally, we analyze the performance of the achieved scheduling proposal in several relevant scenarios, concluding that it outperforms the most popular existing resource allocation strategies. 1. Introduction Undoubtedly, the use of mobile Internet applications has notably increased over the last years, which has led to the growth of the demand for wireless bandwidth. Hence, one of the fundamental challenges that network providers nowadays face is how to efficiently share radio resources among users’ traffic flows. In wireless links channel capacity evolves over time due to the intrinsic degradations of this medium, which has motivated the investigation of scheduling problems in time-varying channels. Traditional scheduling strategies for resource allocation are oriented to objective quality parameters such as delay. Nevertheless, considering the importance and the necessity of network resource allocation for maximizing users’ subjective quality of service or quality of experience (QoE) [1], QoE- driven scheduling becomes essential for network providers. us, motivated by the necessity of obtaining an imple- mentable QoE-aware scheduler in wireless channels with time-varying capacity, in this paper we aim at characterizing in closed-form a novel channel-aware scheduler for the problem of maximizing users’ perceived quality. We focus on a scenario where traffic flows arrive and depart upon service completion. 1.1. Background. Channel-aware or opportunistic schedulers give priority to users in good channel conditions. Although several channel-aware strategies exist in the literature, Max Rate and Proportional Fair [2] among the most popular, the achievement of the optimal solution for time-varying scheduling optimization problems is rather difficult and unknown. Without any doubt, most analyzed resource allo- cation problems aim at minimizing the average delay of user traffic flows [37]. e best contribution found in the field of optimizing delay in a time-varying context is found in [6], which pro- poses a simple index-based heuristic scheduler that performs satisfactorily. In this work, the authors consider exponentially distributed flow sizes and formulate the optimal scheduling problem as a Markov decision process (MDP) [8]. Since this model is PSPACE-hard [9], it does not allow obtaining a tractable solution, and the authors use approximations for its resolution. is way, using the theory of restless bandits [10] and Whittle approach [11], the authors develop an index rule called potential improvement (PI), which has the property of being asymptotically optimal and fluid-optimality [12]. Nonetheless, analyzing the problem of minimizing mean delay for generally distributed flow sizes in randomly time- varying channels remains still open. Hindawi Publishing Corporation Abstract and Applied Analysis Volume 2014, Article ID 647157, 14 pages http://dx.doi.org/10.1155/2014/647157

Transcript of Research Article A Novel Scheduling Index Rule Proposal...

Research ArticleA Novel Scheduling Index Rule Proposal for QoE Maximizationin Wireless Networks

Ianire Taboada and Fidel Liberal

Faculty of Engineering of Bilbao University of the Basque Country UPVEHU Alameda Urquijo sn 48013 Bilbao Spain

Correspondence should be addressed to Ianire Taboada ianiretaboadaehues

Received 9 April 2014 Accepted 9 July 2014 Published 5 August 2014

Academic Editor Victor Kovtunenko

Copyright copy 2014 I Taboada and F Liberal This is an open access article distributed under the Creative Commons AttributionLicense which permits unrestricted use distribution and reproduction in any medium provided the original work is properlycited

This paper deals with the resource allocation problem aimed at maximizing usersrsquo perception of quality in wireless channels withtime-varying capacity First of all we model the subjective quality-aware scheduling problem in the framework of Markoviandecision processes Then given that the obtaining of the optimal solution of this model is unachievable we propose a simplescheduling index rule with closed-form expression by using a methodology based on Whittle approach Finally we analyze theperformance of the achieved scheduling proposal in several relevant scenarios concluding that it outperforms the most popularexisting resource allocation strategies

1 Introduction

Undoubtedly the use of mobile Internet applications hasnotably increased over the last years which has led to thegrowth of the demand for wireless bandwidth Hence one ofthe fundamental challenges that network providers nowadaysface is how to efficiently share radio resources among usersrsquotraffic flows In wireless links channel capacity evolves overtime due to the intrinsic degradations of this medium whichhas motivated the investigation of scheduling problems intime-varying channels

Traditional scheduling strategies for resource allocationare oriented to objective quality parameters such as delayNevertheless considering the importance and the necessity ofnetwork resource allocation for maximizing usersrsquo subjectivequality of service or quality of experience (QoE) [1] QoE-driven scheduling becomes essential for network providers

Thus motivated by the necessity of obtaining an imple-mentable QoE-aware scheduler in wireless channels withtime-varying capacity in this paper we aim at characterizingin closed-form a novel channel-aware scheduler for theproblem of maximizing usersrsquo perceived quality We focus ona scenario where traffic flows arrive and depart upon servicecompletion

11 Background Channel-aware or opportunistic schedulersgive priority to users in good channel conditions Althoughseveral channel-aware strategies exist in the literature MaxRate and Proportional Fair [2] among the most popularthe achievement of the optimal solution for time-varyingscheduling optimization problems is rather difficult andunknown Without any doubt most analyzed resource allo-cation problems aim at minimizing the average delay of usertraffic flows [3ndash7]

The best contribution found in the field of optimizingdelay in a time-varying context is found in [6] which pro-poses a simple index-based heuristic scheduler that performssatisfactorily In this work the authors consider exponentiallydistributed flow sizes and formulate the optimal schedulingproblem as a Markov decision process (MDP) [8] Since thismodel is PSPACE-hard [9] it does not allow obtaining atractable solution and the authors use approximations for itsresolution This way using the theory of restless bandits [10]and Whittle approach [11] the authors develop an index rulecalled potential improvement (PI) which has the propertyof being asymptotically optimal and fluid-optimality [12]Nonetheless analyzing the problem of minimizing meandelay for generally distributed flow sizes in randomly time-varying channels remains still open

Hindawi Publishing CorporationAbstract and Applied AnalysisVolume 2014 Article ID 647157 14 pageshttpdxdoiorg1011552014647157

2 Abstract and Applied Analysis

Furthermore even though a few QoE and channel-awarescheduling proposals exist [13ndash15] no significant analyticalframework is provided for these heuristics In fact there isno solution based on relevant mathematical methods such asMDP for modelling and Whittle method for its resolution inorder to obtain a simple scheduling index rule

On the other hand it is worth mentioning the workcarried out in [16] which is pioneer in introducing QoE-awareness in a MDP Moreover since the proposed modelis analytically unsolvable in general they achieve an easytractable and well-performing QoE-aware index rule basedon Gittins approach [4] for constant channel capacity whichthey denominate Gittins MOS (GM) It is known that Gittinsproposed a method to obtain the so-called Gittins indexwhich originally minimized mean delay for constant channelcapacity However as concluded in [17] Gittins approach isunsuitable for solving problems with time-varying channel

12 Scope and Contributions Since the stochastic anddynamic resource allocation problem of subjective qualitymaximization in channels with time-varying capacity isanalytically and computationally unfeasible for finding anoptimal solution the main objective of this work is to designa simple and tractable heuristic priority scheduling rule usinganalytical tools that have had a great contribution in theoptimization area

Thus the main contribution of this paper is threefold

(i) Firstly we propose a QoE-aware MDP model in atime-varying channel context

(ii) Secondly in order to achieve a tractable solution forthe aforementioned QoE and channel-aware modelwe focus on designing a simple heuristic index ruleusing Whittle approach

(iii) Thirdly although for many years exponential flowsize distributions have been considered for trafficmodelling in order to simplify the resolution ofscheduling optimization problems as a step forwardin this work we take into account size distributionsthat better capture the real world patterns

The remainder of the paper is organized as follows Firstof all we characterize the QoE-aware scheduling problemfor the time-varying channel context in Section 2 ThenSection 3 describes the extension of the basic QoE-awareMDP model to time-varying channels In Section 4 weachieve the Whittle index-based solution for the proposedMDP and its performance is evaluated in Section 5 FinallySection 6 gathers the main conclusions of this paper

2 Problem Description

We analyze the problem of maximizing average QoE in time-varying channels Even though this study is applicable to anytime-varying channel context we focus onwireless networksIn particular we centre on a wireless downlink data channelin a single cell system In this way at the beginning of eachtransmission time interval (TTI) the scheduler located in the

base station makes decisions in order to choose a user trafficflow to transmit

21 User Each user 119896 in the system is uniquely associatedwith the flow related to its requested content with its QoEcharacteristic and with its wireless channel We use a genericconcept of flow which refers to an amount of data that canbe displayed to (received by) the end user as a standaloneelement

We consider a system with a fixed number of userswithout arrivals of new users This assumption simplifies themathematical model However being aware of the impact ofarrivals on the performance of scheduling we will analyzethe performance of scheduling strategies in the presence ofarrivals

211 Flow Size Flows are characterized by their random size119883 the total amount of bits to be transferred Sizes are assumedto be independently distributed with E[119883] lt infin where119865(119909) = P(119883 le 119909) 119891(119909) and ℎ(119909) = 119891(119909)(1 minus 119865(119909)) arethe corresponding cumulative distribution function densityfunction and hazard rate respectively

We focus on the important class of size distributionswith a decreasing hazard rate particularly we assume Paretodistributed flow sizes It is known that Internet traffic flowsizes are properly modelled by means of Pareto distributions[18] which better fit Internet flow sizes compared withthe simplistic exponential approach We define the Paretodistribution with shape parameter 120572 gt 1 and scale parameter120574 gt 0 whose density function for all 119909 ge 0 is

119891 (119909) =120574120572

(1 + 120574119909)120572+1

(1)

Note that even though we use Pareto distributed flowsizes the results provided are valid for any size distributionwith a decreasing hazard rate

212 QoE Characteristic The time needed to transmit tochannel all the bits from a user flow has a direct impact onuserrsquos perceived quality We consider that this delay is themain cause of subjective quality distortion Nowadays mostof the Internet traffic is transported over TCP which imple-ments the packet recovery mechanisms to avoid applicationlevel losses In TCP-based services the predominant sourceof losses at the application is packets arriving later than theirplayout time Hence it is generally useful to provide a delay-driven QoE-awareness

This way we quantify QoE by a delay dependent MeanOpinion Score (MOS) [19] utility function User satisfactionis evaluated in MOS scale of 1 (very poor quality) to 5(excellent quality)

As shown in QoE studies [20] perceived quality ismaximum until a low delay value 119889min Then from this delaythreshold QoE degradation happens until a high delay value119889max Above this delay threshold service is unaffordableresulting in a MOS value of 1 The values of these delaythresholds depend on several factors [21] such as the flowsize the service type the network type and the user expertise

Abstract and Applied Analysis 3

0 5 10 15 20 25 301

15

2

25

3

35

4

45

5

d (s)

MO

S

dmaxdmin

Figure 1 MOS versus delay

We illustrate an example of the explained behavior inFigure 1

213 Channel Model The channel quality associated to auser 119896 evolves randomly and independently of its evolutionhistory We assume that the channel can be in N =

1 2 119873119896 conditions Therefore user 119896 is in channel

condition 119899 with probability 119902119896119899 having sum

119899isinN 119902119896119899

= 1In channel state 119899 119903

119896119899bits are transmitted to channel and

transmission rates 119904119896119899

are 1199041198961le 1199041198962le sdot sdot sdot le 119904

119896119873119896

22 Server The objective of the scheduler is finding a policy120587 that solves the following optimization problem

max120587

1

119870

119870

sum

119896=1

MOS119896(119889119896) (2)

According to the already described aspects about the schedul-ing problem we consider a set of admissible policies Πwhich fulfils the next properties

(i) Delay dependency the time that a flow remains in thesystem 119889 is known

(ii) QoE-awareness the delay-dependent MOS functionthat characterizes user subjective quality is known

(iii) Channel-awareness channel quality indicator (CQI)information sent from mobile users is used as userrsquosinstantaneous channel condition information

(iv) Nonanticipation similarly to most current IP sys-tems flow size is unknown However existent nonan-ticipating size-based disciplines make use of flowattained service [4 7] the bits that have been trans-ferred of a flow Due to the memoryless property ofthe exponential distributions for these distributionsthe hazard rate is independent of attained serviceNevertheless the considered Pareto distributionshave a decreasing hazard rate with attained service[22] this is completion probability 120583

(119886119899) is higher

for lower attained service values This way attainedservice 119886 is used as size information

(v) Preemption we assume that the server is preemptivethat is at every decision epoch it is permitted tosuspend the service of a flow whose transmission isunfinished

(vi) Single service only the transmission of a flow isallowed in each TTI

3 MDP Formulation

In this section we formulate the problem described inSection 2 as a MDP First we provide the QoE-aware MDPmodel of each user 119896 Then we formulate the optimizationproblem for the joint MDP model which takes into accountall the users in the system

31 MDP Model of a Job The action space B of user 119896is binary action 0 means not serving and if action 1 ischosen 119903

119896119899bits from this user being in channel state 119899

are transmitted to channel Besides each user 119896 is definedby tuple (S

119896 (R119887119896119904)119887isinB

(W119887119896119904)119887isinB

(P119887119896119904)119887isinB

) These elementsare defined as follows

(i) S119896= (A119896timesD119896times 1 2 119873

119896) cup [D

119896+ 1] cup [lowast]

is the set of all 119904 states for a user 119896 which is classifiedin three groups

(a) Unfinished states all the states in which flowtransmission is uncompleted belong to thisgroup These states have three dimensionsdenoted as 119904 = (119886 119889 119899) with attained servicelevel 119886 isin A

119896 delay level 119889 isin D

119896 and

channel condition 119899 isin 1 2 119873119896 as com-

ponents The attained services that correspondto these attained service levels are multiples ofthe first not null 119903

119896119899transmission rate whereas

the delays that correspond to delay levels aremultiples of TTI

(b) Reward states these 119904 = [119889] states are reachedonce flow transmission has finished and inthese delay-dependent states the QoE-awarereward is given This way aimed at maximizingaverage MOS a delay dependant QoE-awarereward giving only happens once the wholeflow has been transmitted since from sub-jective quality perspective the MOS functiononly makes sense once flow transmission iscompleted (see [16] for more details)

(c) Final state the chain of states ends in theabsorbing state 119904 = [lowast]

(ii) R119887119896= (119877

119887

119896119904)119904isinS119896

where 119877119887119896119904

is the expected one-slotreward received for user 119896 at state 119904 if action 119887 isdecided at the beginning of a TTI

119877119887

119896(119886119889119899)= 0 119877

119887

119896[119889]= MOS (119889 + 1) 119877

119887

119896[lowast]= 0

(3)

where MOS(119889 + 1) is normalized MOS in the range[0 1]

4 Abstract and Applied Analysis

(iii) W119887119896= (119882

119887

119896119904)119904isinS119896

where119882119887119896119904

is the expected one-slotwork done for user 119896 at state 119904 if action 119887 is decided atthe beginning of a TTI

1198820

119896119904= 0 119882

1

119896119904= 1 (4)

(iv) P119887119896= (119901119887

119896(119904 1199041015840

))1199041199041015840isinS119896

where 119901119887119896(119904 1199041015840

) is the probabil-ity of user 119896 of moving from state 119904 to state 1199041015840 if action119887 is decided at the beginning of a TTI

1199010

119896((119886 119889 119899) (119886 119889 + 1119898)) = 119902

119896119898

1199011

119896((119886 119889 119899) (119886 + 119903

119899 119889 + 1119898)) = 119902

119896119898sdot (1 minus 120583

119896(119886119899))

1199011

119896((119886 119889 119899) [119889 + 1]) = 120583

119896(119886119899)

1199010

119896([119889] [lowast]) = 119901

0

119896([lowast] [lowast]) = 1

(5)

where 120583119896(119886119899)

= P(119886 lt 119883119896le 119886 + 119903

119896119899| 119883119896gt

119886) Observe that the probability of changing fromchannel state 119899 to channel state119898 corresponds to thesteady-state probability 119902

119896119898due to the memoryless

property of channel evolution

Thus the dynamics of user 119896 is captured by state process119904119896(119905) isin S

119896and action process 119887

119896(119905) isin B Moreover we

highlight that the proposedMDPmodel has restless propertywhich admits evolution and rewards even if not chosenfor transmission due to the delay increase and the channelstochastic evolution

Figure 2 shows part of the per-user state diagram of thenew QoE-aware MDP model which summarizes and relatesthe aforedescribed components

32 Optimization Problem Besides the optimization prob-lem (2) associated with such MDP can be written as

max120587isinΠ

E120587

0[

infin

sum

119905=0

sum

119896isinK

120573119905

119877119887119896(119905)

119896((119886119896(119905)119889119896(119905)119899119896(119905))cup[119889119896(119905)])]

sum

119896isinK

119887119896(119905) = 1 forall119905

(6)

Note that we consider the undiscounted reward case wherethe discounted factor 120573 is 1 being reward deterioration intime implicit in the delay-dependent MOS function

Nonetheless optimally solving problem (6) is unfeasiblein general On the one hand when channel capacity isconstant the impossibility of obtaining the exact QoE-awaresolution is revealed in [16] and hence since the time-varyingchannel model is more complex the achievement of theoptimal solution is unviable in this case On the other hand itis known that the allocation constraint causes intractability insimilar problems [11] and therefore in our case having three-dimensional states finding the optimal solution is extremelydifficult or even unachievable In this way once seeingthe impossibility of obtaining the optimal solution for theconsidered problem in this work we will focus on achievingan approximate solution For that purpose in the next sectionwe will propose a heuristic based on Whittle method

(a d + 1 1)

(a d + 1N)

q1

qN

(a d n)

120583(an)

q1(1 minus

120583 (an))

qN(1 minus 120583

(an) )

[d + 1]

b = 1

b = 0 R = 0

(a + rn d +

1N)

(a + rn d +

1 1)

lowast

R = MOS(d + 1)

Figure 2 A part of per-user state diagram

4 QoE-Aware Index Rule Proposal Based onWhittle Approach

In this part we provide aWhittle index rule type approximatesolution to problem (6) based on Whittle method Theidea of Whittle [11] consists in achieving a function thatmeasures the dynamic service priority in order to obtain asimple scheduling index rule Whittle indices are computedin isolated way for each user and they measure the expectedefficiency of serving a user in each state

Note that we are only interested in the indices of (119886 119889 119899)states as once flow transmission is completed schedulingmakes no sense Thereby the optimization problem formu-lated in (6) can be relaxed by requiring to serve a job per sloton average as proposed in [11] which is further approachedby Lagrangian methods [23] and can be decomposed into asingle-job price-based parameterized optimization problemFor a price or Lagrangiar parameter V we will therefore studythe user-119896 subproblem

max120587119896isinΠ

infin

sum

119905=0

E120587

0120573119905

sdot [119877119887119896(119905)

119896(119886119896(119905)119889119896(119905)119899119896(119905))minus V119882119887119896(119905)119896(119886119896(119905)119889119896(119905)119899119896(119905))

]

(7)

The Lagrangiar parameter V can be interpreted as the per-slotcost of serving

Let us define serving set F sube S119896 which prescribes to

serve a user 119896 if (119886 119889 119899) isin F while not to serve this user if(119886 119889 119899) notin F We will refer to states (119886 119889 119899) isin F as activeand (119886 119889 119899) notin F as passive This way it is possible to rewriteproblem (7) as

maxFsube119878119896

RF119896(119886119889119899)

minus VWF119896(119886119889119899)

(8)

whereRF(119886119889119899)

andWF(119886119889119899)

are respectively the expected totalreward and expected total work in (119886 119889 119899) state

Abstract and Applied Analysis 5

Conjecture 1 Problem (8) is indexable

Due to the complexity of our model proving the index-ability of Problem (8) is not an easy task In this way weassume Conjecture 1 and therefore we suppose that foreach (119886 119889 119899) state of user 119896 the Whittle index exists Fromnow on we omit user label 119896 In spite of considering theundiscounted case the mathematical analysis is carried outfor the discounted case so as to avoid infinite values causedby 1minus120573 terms in denominators for 120573 = 1 and then we obtainthe undiscounted index values in the limit 120573 rarr 1

Following Whittle index definition in [10] we formallywrite the Whittle index for Problem (8) Vlowast

(119886119889119899) as

Vlowast(119886119889119899)

= VF(119886119889119899)

=R⟨1F⟩

(119886119889119899)minusR⟨0F⟩

(119886119889119899)

W⟨1F⟩

(119886119889119899)minusW⟨0F⟩

(119886119889119899)

(9)

Whittle index VF(119886119889119899)

represents the rate betweenmarginal reward and marginal work where the marginalreward (work) is the difference of the expected reward earned(work required) by serving and not serving at the initial state(119886 119889 119899) and employing policyF afterwards

Lemma 2 For any state (119886 119889 119899) and under any policy F weobtain

VF(119886119889119899)

= (120573[(1 minus 120583(119886119899)

) sum

119898isinN

119902119898R

F

(119886+119903119899 119889+1119898)+ 120583(119886119899)

sdot MOS (119889 + 1) minus sum

119898isinN

119902119898R

F(119886119889+1119898)

])

times (1 + 120573[(1 minus 120583(119886119899)

) sum

119898isinN

119902119898W

F

(119886+119903119899 119889+1119898)

minus sum

119898isinN

119902119898W

F(119886119889+1119898)

])

minus1

(10)

Proof From the definition of reward and work respectivelywe have

RF(119886119889119899)

=

R⟨1F⟩

(119886119889119899)= 120573[(1 minus 120583

(119886119899)) sum

119898isinN

119902119898RF

(119886+119903119899 119889+1119898)

+ 120583(119886119899)

sdot MOS (119889 + 1) ]

(119886 119889 119899) isin F

R⟨0F⟩

(119886119889119899)= 120573 sum

119898isinN

119902119898RF(119886119889+1119898)

(119886 119889 119899) notin F

(11)

(1)F0= 0

(2) for 119894 = 1 rarr number of states do(3) (119886 119889 119899)

119894isin argmax VF119894minus1

(119886119889119899) (119886 119889 119899) notin F

119894minus1

(4) Vlowast(119886119889119899)119894

= VF119894minus1(119886119889119899)119894

F119894= F119894minus1cup (119886 119889 119899)

119894

(5) end for

Algorithm 1AG-algorithm for Problem (8)

WF(119886119889119899)

=

W⟨1F⟩

(119886119889119899)= 1 + 120573 (1 minus 120583

(119886119899)) sum

119898isinN

119902119898WF(119886+119903119899 119889+1119898)

(119886 119889 119899) isin F

W⟨0F⟩

(119886119889119899)= 120573 sum

119898isinN

119902119898WF(119886119889+1119898)

(119886 119889 119899) notin F

(12)

By substituting the expressions (11) and (12) in formula(9) we achieve a more complete VF

(119886119889119899)(10) formulation

Nevertheless in order to obtain an analytically tractableWhittle index expression for (10) in the state (119886 119889 119899) it isnecessary to determine the policyF of the future states thathave influence on the index computation of this state As canbe observed from (10) (11) and (12) expressions in thesefuture states the value of attained service is maintained orincreased whereas delay grows over time Since increasingattained service decreases instant completion probability fora Pareto distribution and increasing delay degrades QoE itwould seem intuitively correct to suppose that future stateswith the same or worse channel conditions are passive Insuch a way we could say that in a state with the same or worsechannel condition being attained service and delay higherthat the Whittle index value is lower However for futurestates with better channel conditions their passive or activeproperty is not trivial because the improvement in channelcondition does not guarantee being a better state due to theaforementioned behavior of the increasing attained serviceand delayTherefore finding the active set of all the influentialstates for Whittle index computation becomes a challengingtask

In such situation aimed at achieving the characteristicsof the optimal policy and the subsequent active set buildingevolution we will use an algorithm called Adaptive-GreedyshortlyAG-algorithm (see [10] for a survey) This algorithmcomputes Whittle indices numerically taking into accountoptimal active sets This way we have implemented AG-algorithm for our case study in order to verify the fundamen-tal properties of the Whittle index for our problem whichwill be useful to determine the active set and consequentlya closed-form index expression We provide the operation ofthe applied algorithm in Algorithm 1

We have performed several numerical experiments for120573 = 1 case in which we have reduced the state space ofboth delay and channel states so as to do the execution of thealgorithm computationally and in time viably We have onlyconsidered two channel conditions and eleven levels of delay

6 Abstract and Applied Analysis

times10minus3

6

5

4

3

2

1

00

0

5 5

10

10times107

ad

Vlowast (ad1)

(a)

0

0

5 5

10

10times107

a d

Vlowast (ad2)

001

0008

0006

0004

0002

0

(b)

Figure 3 Whittle index values obtained from AG-algorithm for two channel conditions and different attained services and delays For aPareto distribution with 120572 = 15 and E[119883] = 5Mb 119902

2= 05 119903

1= 84Kb 119903

2= 168Kb 119889min = 2 s and 119889max = 9 s

0

0

5 5

10

10

times107

ad

Vlowast (ad1)

001

0008

0006

0004

0002

0

q2 = 01

(a)

times10minus3

4

3

2

1

0

0

0

5 5

10

10

times107

a d

Vlowast (ad1)

q2 = 09

(b)

Figure 4 Whittle index values obtained fromAG-algorithm for different channel state probabilities For a Pareto distribution with 120572 = 15and E[119883] = 5Mb 119903

1= 84Kb 119903

2= 168Kb 119889min = 2 s and 119889max = 9 s

in the scenarios In such a way from experimental results weconjecture the following fundamental properties

(1) TheWhittle index value depends on attained servicedelay and channel state Moreover as can be seenin graphs from Figures 3 and 4 we can clearlydistinguish the ranges of MOS function above theupper delay threshold the index value is null

(2) For the same channel condition and the samedelay level Whittle index values are decreasing withattained service Vlowast

(119886119889119898)gt Vlowast(1198861015840119889119898)

forall1198861015840

gt 119886 (illus-trated in Figures 3 and 4)

(3) For the same channel condition and the same attainedservice levelWhittle index values are decreasing withdelay Vlowast

(119886119889119898)gt Vlowast(119886119889+1119898)

(See Figures 3 and 4)

Abstract and Applied Analysis 7

(4) In a better channel condition for the same attainedservice level and the same delay level the Whittleindex value is higher Vlowast

(1198861198891198981)gt Vlowast(1198861198891198982)

forall1198981gt 1198982

(Observe Figure 3)(5) Opposite to the previous channel-aware work [6]

where Whittle indices in the best channel conditionare infinite having absolute priority in this channelstate in our case being in the best channel conditiondoes not guarantee that the Whittle index value ishigher than in the rest of the channel conditions it ispossible to fulfil Vlowast

(119886119889119898lt119873)gt Vlowast(11988610158401198891015840119873)

If we comparethe two graphs in Figure 3 it can be easily seen howfor several attained services andor delayed values theWhittle index values in the worst channel conditionare higher than in the best channel condition (takingalso into account nonnull values)

(6) Dependency on channel condition probabilities hilefor channel condition 119873 the Whittle index valueremains invariant with channel state probability forthe rest of channel conditions its value decreases aslong as channel state probabilities increase in betterchannel conditions (shown in Figure 4)

Once we analyzed the properties of theWhittle index weset out to define the structure of the active set in order toobtain an analytically and computationally tractable Whittleindex expression In such a way to derive a closed-formcharacterization of the Whittle index of (119886 119889 119899) state first ofall it is necessary to specify the activity of (119886 119889 + 1119898) and(119886 + 119903

119899 119889 + 1119898) states As concluded from Whittle index

properties it is known that (119886 119889+1119898 le 119899)cup(119886+119903119899 119889+1119898 le

119899) states are passive However it is not trivial to specify abovewhich attained service level andor delay level if (119886 119889+1119898 gt

119899)cup (119886+ 119903119899 119889 + 1119898 gt 119899) states with better channel condition

are active or passiveNonetheless in the best channel the active set is totally

specified since all the states that compute in the index are pas-sive because of nonexisting channel improvement and beingin the future attained service equal or higher and delay higherAs shown in Proposition 3 for the best channel condition theWhittle index has a closed-form expression where the indexvalue is the multiplication between completion probabilityand normalizedMOS function Note that the obtainedWhit-tle index expression in the best channel state is equivalent tothe GM index achieved for constant channel [16]

Proposition 3 The Whittle index for Problem (8) in the bestchannel condition is given by

Vlowast(119886119889119873)

= 120583(119886119873)

sdot MOS (119889 + 1) (13)

Proof It is known that for Vlowast(119886119889119873)

index computation thatF = 0 In this way using (11) and (12) RF

(119886119889+1119898)=

120573sum119898isinN 119902

119898RF(119886119889+2119898)

RF(119886119889+2119898)

= 120573sum119898isinN 119902

119898RF(119886119889+3119898)

RF(119886119889gt119889max 119898)

= 0 Hence analogolously RF(119886+119903119899 119889+1119898)

= 0WF(119886119889+1119898)

= 0 and WF(119886+119903119899 119889+1119898)

= 0 Substituting thesenull terms in expression (10) and being 120573 = 1 we achieve(13)

Besides as stated before obtaining a computationallytractable Whittle index expression for any channel conditionis not possible In such situation in order to achieve atractable Whittle-based QoE-aware index rule for time-varying channels we propose a Whittle-based approximateheuristic in the next subsection

41 Approximate Whittle Index Proposal In the following wepropose an approximate solution for the QoE-aware Whittleindex in a time-varying channel context (10)

First of all we will determine the structure of the activeset For the Whittle index computation of (119886 119889 119899) state aspreviously concluded it is known that (119886 119889 + 1119898 le 119899) cup (119886+

119903119899 119889 + 1119898 le 119899) states are passive but we need to define the

unknown activity of (119886 119889 + 1119898 gt 119899) cup (119886 + 119903119899 119889 + 1119898 gt 119899)

states Thus so as to determine if these states are active orpassive we apply the following two considerations

(1) 120583 approximation

120583(119886119898)

asymp 120583(119886+119903119897 119898)

(14)

(2) MOS approximation

MOS (119889) asymp MOS (119889 + 1) (15)

On the one hand we assume that for any 119897 = 1 2 119873

value that 120583(119886119898)

asymp 120583(119886+119903119897 119898)

(14) It is known that (119886 119889119898 gt

119899) states are active and being 120583(119886119898)

asymp 120583(119886+119903119899 119898)

we couldconsider that (119886 + 119903

119899 119889 119898 gt 119899) states are active

On the other hand we assume MOS(119889) asymp MOS(119889 + 1)(15) since QoE degradation in a higher delay level is neg-ligible in the millisecond scale that delay evolves Takinginto account the approximation about MOS function being(119886 119889119898 gt 119899) states active we can suppose that (119886 119889 + 1119898 gt

119899) states are also active This way combining approximations(14) and (15) we consider that (119886 + 119903

119899 119889 + 1119898 gt 119899) states are

activeThen suppose that the aforementioned 120583 and

MOS approximations cause 119877119886119889

= sum119898119902119898119877(119886119889119898)

asymp

sum119898119902119898119877(119886+119903119897 119889+1119898)

and 119882119886119889

= sum119898119902119898119882(119886119889119898)

asymp

sum119898119902119898119882(119886+119903119897 119889+1119898)

So using these reward and workapproximations the first simplified expression for theWhittle index formulation (10) is

Vlowast(119886119889119899)

=120573120583(119886119899)

(MOS (119889 + 1) minus 119877119886119889)

1 minus 120573120583(119886119899)

119882119886119889

(16)

In such a way applying the previous simplificationswe obtain the Whittle index approximation presentedin Proposition 4 what we call attained service PI MOS(ASPIM) The methodology to compute the closed-formexpression of this QoE-aware Whittle-based index is pre-sented in the Appendix

Proposition 4 The formulation of the ASPIM index is

119860119878119875119868119872 = Vlowast(119886119889119899)

= lim120573rarr1

120573120583(119886119899)

sdot MOS (119889 + 1) (1 minus 120573)1 minus 120573 + 120573sum

119898gt119899119902119898(120583(119886119898)

minus 120583(119886119899)

)

(17)

8 Abstract and Applied Analysis

Table 1 CQIs and corresponding transmission rates (Mbps)

CQI 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15119904 0 42 672 84 11256 168 2184 252 2688 336 4468 504 5376 672 758 8064

If we analyze expression (17) we observe that this QoE-aware and channel-aware Whittle-based ASPIM index isimplementable This proposal depends on QoE size andchannel properties Moreover the obtained index for thebest channel condition is equal to the original Whittle index(13) since the summation in the denominator becomes nullHowever as shown in (18) when channel state is not the bestthe index value goes to zero

ASPIM119899 =119873

= lim120573rarr1

120583(119886119899)

sdot MOS (119889 + 1) (1 minus 120573)sum119898gt119899

119902119898(120583(119886119898)

minus 120583(119886119899)

)997888rarr 0

(18)

Hence a user in its best channel state with nonnullinstantaneous normalized MOS function has priority overa user which is not in its best channel condition Thus wesummarize the proposed QoE-aware Whittle-based ASPIMindex rule in Definition 5

Definition 5 The ASPIM index rule consists of at everydecision slot

(i) serving the user in its best channel condition withnonnull instantaneous normalized MOS functionwith the highest value of 120583

(119886119873)sdot MOS(119889 + 1)

(ii) if there is no user in its best channel condition withnonnull instantaneous normalized MOS functionserving the user with the highest value of (120583

(119886119899)sdot

MOS(119889 + 1))sum119898gt119899

119902119898(120583(119886119898)

minus 120583(119886119899)

) (using (18)normalized by (1 minus 120573))

So if channel condition is not the best the index valueis the ratio of the multiplication between the actual servicecompletion and the instantaneous normalizedMOS functionrespect to the expected potential improvement of the comple-tion probability

Therefore in the context of time-varying channels aimedat maximizing average MOS we have proposed ASPIMscheduling algorithm which is a simple and tractable QoE-aware channel-aware and size-based index rule Neverthe-less in order to verify the correct behavior of this heuristicwe analyze its performance in the subsequent section

5 Performance Evaluation

In this section we evaluate the performance of the proposedASPIM index rule (see proposal in Definition 5) To that endwe compare the QoE performance of our heuristic with theone achieved by well-known scheduling strategies in severalsimulation scenarios Below we provide a brief description ofthe scheduling priority rules used in our experiments

(i) Max rate (MR) this opportunistic policy consists inserving the user with the best channel capacity

(ii) Proportional fair (PF) this channel-aware disci-pline serves the user with the highest ratio betweeninstantaneous transmission rate and current attainedthroughput

(iii) Cost and attained service dependant 120583 (119888AS120583)although it might seem counter-intuitive minimizingmean delay does not result in maximizing averageMOS [24] This way in order to compare the per-formance with delay-aware solutions we propose toadapt the classical 119888120583 rule aimed at minimizing meandelay [25] to attained serviceThis adaptation consistsof serving the user with the highest value of 119888120583

(119886119899)

being 119888 the waiting cost paid for every slot while flowtransmission is uncompleted

(iv) GM [16] thisQoE-aware pioneering index rule origi-nally proposed for constant channel capacity choosesthe user with the highest value of 120583

(119886119899)sdot MOS(119889 + 1)

In case of ties we use random tie-breaking ruleConcerning traffic flows in the system we use the Pareto

size distribution given in (1) in simulations whose 120572 and 120574parameters determine mean flow size We consider typicalmean sizes used in wireless networks 05Mb (small traffica web page email) 5Mb (medium traffic a PDF documenta picture etc) and 50Mb (MP3 audio a group of picturesfrom a video sequence etc) User traffic flows arrive to thescheduler according to a Poisson process with 120582 rate

Referring to network characteristics we use transmis-sion rates that depend on CQIs used in real wirelessnetworks which are associated with 4G modulation andcoding schemes We show the mapping between CQI valuesand transmission rates in Table 1 The presented values areadapted and extrapolated from [26] for a cell of 20MHz

Moreover we consider different network loads in order toanalyze the behavior of scheduling strategies under differentnetwork conditions Flow arrival rate determines networkload 120588 where 120588 = 120582 sdot (E[119883]119904

119873) We consider seven network

states in this study low load (120588 = 025 0375) medium load(120588 = 05 075) and high load (120588 = 085 09 095)

Regarding QoE aspects as typically utilized in perceivedquality studies [27] we use a log-shaped MOS function forQoE degradation presented in (19) This way employing theMOS curve modelling provided in [21] as starting point andtaking into account the flow size the network type and theuser expertise we have suitably defined 119889min and 119889max delaythresholds used in simulations and consequently 119888

1and 1198882

MOS coefficients Consider

MOS = 1198881minus 1198882sdot log (119889) (19)

We have implemented the whole network environmentfor the scheduling of network resources in MATLAB Inrelation to the performed experiments for each scenario

Abstract and Applied Analysis 9

Table 2 Parameter set in the experimental scenarios

Scenario MOS Size Channel 119888

1 119889min = 05 s 119889max = 10 s 120572 = 15 120574 = 4 sdot 10minus7 CQI = 3 5 11198881= 407 119888

2= 133 E[119883] = 5Mbit 119902

2= 05

2 119889min = 04 s 119889max = 06 s 120572 = 15 120574 = 4 sdot 10minus7 CQI = 3 5 11198881= minus403 119888

2= 986 E[119883] = 5Mbit 119902

2= 05

3 119889min = 05 s 119889max = 10 s 120572 = 15 120574 = 4 sdot 10minus7 CQI = 3 9 11198881= 407 119888

2= 133 E[119883] = 5Mbit 119902

2= 05

4 119889min = 005 s 119889max = 1 s 120572 = 15 120574 = 4 sdot 10minus6 CQI = 3 5 11198881= 1 1198882= 133 E[119883] = 05Mbit 119902

2= 05

5

119889min1 = 05 s 119889max1 = 5 s 1205721= 15 120574

1= 4 sdot 10

minus7

see Table 3 5 111988811= 379 119888

21= 173 E[119883

1] = 5Mbit

119889min2 = 5 s 119889max2 = 30 s 1205722= 15 120574

2= 4 sdot 10

minus8

11988812= 859 119888

22= 223 E[119883

2] = 50Mbit

Table 3 Channel state probabilities in Scenario 5

CQI 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15119902 028 012 009 008 008 008 007 006 005 004 003 001 0009 00005 00003 00002

for each combination of scheduling discipline and networkload we have carried out a set of 10 rounds of 10000 slength simulations These rounds differ in the randomlypregenerated traces (input vector) of sizes and arrivals Thusnot only the overall average MOS values will be provided butwe will also include their 95 confidence intervals

Besides in order to guarantee that the obtained perfor-mance results are generally valid we will analyze differentsimulation scenarios We have chosen five relevant settingswhich differ inQoE size channel orand cost characteristicsThe parameters of these scenarios are summarized in Table 2Note that in the first four scenarios we restrict ourselves tothe case of two channel conditions so as to get fundamentalperformance results

We have carefully selected the scenarios used in simula-tions First of all we define a setting (Scenario 1) in which thevalues of all the parameters are typical or standard the valuesof these parameters introduce low error in the approxima-tions considered for our ASPIM scheduler proposal Thenin order to evaluate worst performance cases for ASPIMwe have chosen three scenarios (Scenario 2 Scenario 3 andScenario 4) in which the error of the approximation used forASPIM is high due to a different factor (gradient ofMOS rateand size resp)

Apart from that it is worthmentioning that for obtainingthe ASPIM index rule that we have considered an isolatedMDP model per user Hence the achieved solution is basedon a unique user type or single class However it would beinteresting to examine the proposal when there are differenttypes of users which is analyzed in Scenario 5 Furthermorethis last setting resembles a real 4G wireless network Notethat even though this technology allows the simultaneoustransmission of multiple flows per TTI we assume that asingle flow transmits in each TTI

03 04 05 06 07 08 09

25

3

35

4

45

5

E[M

OS]

ASPIMMRPF

cAS120583GM

120588

Figure 5 E[MOS] for the basic scenario

Next we describe the results we have achieved in theaforementioned scenarios

51 Scenario 1 Basic Case In this first family of simulationswe consider a basic or typical scenario which takes intoaccount the equiprobable channel case and medium-sizedself-similar flows Figure 5 shows averageMOS results for thissetting employing different scheduling policies and networkloadsThemain conclusion achieved from these results is thatASPIM outperforms the rest of disciplines which is betterthan the QoE-aware GM rule proposed for constant channelcapacity Moreover contrary to the other policies it holds

10 Abstract and Applied Analysis

0 02 04 06 08 10

1

2

3

4

d (s)

PDF(d

)

(a)

1 2 3 4 50

01

02

03

04

05

06

07

MOS

PDF(

MO

S)

ASPIMMRPF

cAS120583GM

(b)

1 2 3 4 50

02

04

06

08

1

MOS

CCD

F(M

OS)

ASPIMMRPF

cAS120583GM

Satisth

(c)

Figure 6 Statistics in Scenario 1 delay PDF (a) MOS PDF (b) and MOS CCDF (c)

averageMOSvalues in the same level for all120588 Referring to thetraditional channel-aware disciplines both PF and MR givebad behavior comparing to the rest of scheduling strategies

In order to justify the previous performance results wenow analyze delay andMOS statistics for the highest networkload considered In such a way as regarded in Figure 6(a) forthe Probability Density Function (PDF) of delay we highlightthe probabilitymass or peak that appears for some schedulingdisciplines before the low delay threshold (05 s) The heightand weight of a peak belonging to a policy have direct impacton its average MOS ASPIM discipline shows the highestand the widest peak among the presented ones followedby GM but with a notable difference Therefore our novelQoE-aware policy seems to try to finish the transmissionof most of the flows before 119889min which is reflected in theaforementioned peaks Note that the cAS120583 rule aimed atminimizing delay behaves in different manner which showsmore fairness among delays in the range shown compared tothe QoE-aware ones

What is previously concluded about delay statistics hasa direct reflection on QoE statistics This way as can beseen in the PDF of MOS presented in Figure 6(b) if theobjective is to maximize average QoE it is important onthe one hand to be the height and the weight of the peaksaround the maximum MOS high and on the other hand tobe the value around the minimum MOS low ASPIM fulfilsthese positive properties and consequently it is the bestin average QoE terms Besides if we analyze the Comple-mentary Cumulative Distribution Function (CCDF) of MOSprovided in Figure 6(c) we observe that ASPIM gives thehighest probability of being MOS value higher Consideringfor example a QoE satisfaction threshold satisth of MOS =4 in this point ASPIM improves GM a 10 whereas thedeterioration of MR and PF is higher than a 50

52 Scenario 2 High Demanding Users Case This scenarioreflects the case of users with low tolerance to delay in

Abstract and Applied Analysis 11

03 04 05 06 07 08 09

05

06

07

08

09

ASPIMMRPF

cAS120583GM

120588

E[M

OS]

2E

[MO

S]1

Figure 7 E[MOS] for abrupt MOS scenario

which theMOS function in theQoE degradation range showsa great slope Thus in this setting the error of the MOSapproximation (15) is high since the gradient of MOS amongconsecutive delay levels becomes significant In Figure 7we compare the results of this second scenario and thebasic scenario From the illustrated graph we could say thatour QoE-aware proposal adapts to any MOS function Thisis whereas for the rest of disciplines (excluding GM) thenegative effects due to the abrupt degradation of the QoEutility function are appreciable ASPIMmanages to avoid thisadapting to any MOS function

53 Scenario 3 Higher Error Case due to Rate During thesimplification process carried out for obtaining the ASPIMindex rule in Section 41 the 120583 approximation used (see (14))introduces a rate dependant error Note that increasing therate in the good channel state makes bigger the error of the 120583approximation used Therefore we will analyze the obtainedperformance for our new QoE-aware proposal under thishigher error due to rate In the previous scenarios we haveassumed that the transmission rate in the good channelcondition is twice the rate in the bad channel state In thissetting we consider that the transmission rate in the goodchannel is four times the transmission rate in the bad channelThis way according to the QoE results presented in Figure 8in spite of a higher error caused by rate ASPIM is superior tothe rest of policies

54 Scenario 4 High Error Case due to Size Besides it isknown that for a distribution with a decreasing hazard ratesuch as Pareto that the gradient of completion probabilityincreases as long as the mean size decreases which makesthe error of the 120583 approximation (14) for the ASPIM ruleachievement bigger In this way in this scenario we verifythat ASPIM gives satisfactory behavior under a more notableerror due to size where we consider a mean size ten times

03 04 05 06 07 08 09

3

35

4

45

5

E[M

OS]

ASPIMMRPF

cAS120583GM

120588

Figure 8 E[MOS] under higher error due to rate

03 04 05 06 07 08 093

35

4

45

5E

[MO

S]

ASPIMMRPF

cAS120583GM

120588

Figure 9 E[MOS] under high error due to size

lower We conclude from Figure 9 that even under a higherror caused by size that ASPIM shows good performancebeing the best alternative

55 Scenario 5 A Real Network Case This last scenarioreflects a real 4G wireless network context but with thesimplification that a single user transmits in each TTI To thatend we use CQI traces obtained from a system-level radioaccess simulator [28] considering users moving at 5 kmhin the cell For this setting with dynamic users we definechannel propagation by the Extended Pedestrian Amodel Asin current 4G networks we consider 16 channel conditionsThe state probabilities of these channel states are collectedin Table 3 which are decreasing with the improvement ofchannel quality In this scenario we consider two classes ofusers which differ in size QoE and cost The mean size

12 Abstract and Applied Analysis

03 04 05 06 07 08 09

1

2

3

4

5

E[M

OS]

120588

(a)

03 04 05 06 07 08 091

2

3

4

5

E[M

OS]

ASPIMMRPF

cAS120583GM

120588

(b)

03 04 05 06 07 08 091

2

3

4

5

E[M

OS]

ASPIMMRPF

cAS120583GM

120588

(c)

Figure 10 E[MOS] for 4G scenario aggregate (a) class 1 (b) class 2 (c)

in class 2 is ten times bigger than in class 1 and we givepriority to the small-sized class by having bigger cost andQoErequirement

Under this mixture of classes and realistic channel mod-els in Figure 10(a) we observe that scheduling strategiesshow the same tendency as in the previous scenarios It isremarkable that the difference between ASPIM and GM isbigger especially for high 120588 values Moreover under thehighest network load ASPIM achieves MOS values aboveQoE satisfaction threshold (MOS = 4) Furthermore averageMOS values forMR and PF dramatically decrease comparingwith the previous settings

Concerning class behaviour we show per-class resultsin Figures 10(b) and 10(c) As can be concluded fromthe presented results QoE-aware policies guarantee fairnessamong classes and ASPIM is the best choice also inside classAside from that for the cost dependant cAS120583 discipline thedeterioration of the class with no priority is significant

6 Conclusions

This paper goes into detail about obtaining a schedulingalgorithmaimed atmaximizing usersrsquo perception of quality inchannels with time-varying capacity Furthermore this workconsiders traffic flowswith nonexponential size distributionscontrary to previous approaches that assume memorylessdistributions

As first contribution we provide a MDP-based modelfor the scheduling problem presented This model combinesQoE-awareness size-awareness and channel-awarenessNev-ertheless since this model can not be solved either analyt-ically or computationally from the proposed MDP modelwe derive a simple tractable and implementable well-performing scheduling index rule by applying amethodologybased on Whittle approach

The proposed scheduling solution still combines QoE-size- and channel-awareness Our scheduling proposal givespriority to users in their best channel condition that are

Abstract and Applied Analysis 13

out of the service unavoidable range As verified fromsimulation results the proposed scheduling strategy showssuitable subjective quality performance in a wide range ofscenarios including the case of a simplified 4G network withheterogeneous users

Therefore the results of this work present a relevantmathematical basis for developing scheduling algorithmsaimed at maximizing QoE in current and future networkswith time-varying channel capacity This way the proposedchannel-aware ASPIM discipline will be useful for networkoperators in order to guarantee to their customers servicesatisfaction in time-varying wireless networks Apart fromthat the approximations and simplifications used in Whittlemethod are applicable to any area when the gradient ofsystem components is considerably small among consecutivedecision slots

As future research we will extend our work to amultiuserapproach considering the simultaneous transmission ofmul-tiple flows per TTI

Appendix

Proof of Proposition 4

LemmaA1 Suppose that states (119886 119889+1119898 le 119899)cup(119886+119903119899 119889+

1119898 le 119899) are passive and that states (119886 119889 + 1119898 gt 119899) cup (119886 +

119903119899 119889 + 1119898 gt 119899) are active and

Vlowast(119886119889119899)

=120573120583(119886119899)

( MOS (119889 + 1) minus 119877119886119889)

1 minus 120573120583(119886119899)

119882119886119889

(A1)

Along with this if we assume that 119877119886119889

= sum119898119902119898119877(119886119889119898)

asymp

sum119898119902119898119877(119886+119903119897 119889+1119898)

and 119882119886119889

= sum119898119902119898119882(119886119889119898)

asymp

sum119898119902119898119882(119886+119903119897 119889+1119898)

for any 119897 = 1 2 119873 value for theundiscounted case

Vlowast(119886119889119899)

= lim120573rarr1

120573120583(119886119899)

sdot MOS (119889 + 1) (1 minus 120573)1 minus 120573 + 120573sum

119898gt119899119902119898(120583(119886119898)

minus 120583(119886119899)

) (A2)

Proof Referring to work elements using (12)

119882(119886119889119898le119899)

= 120573sum

119898

1199021015840

119898119882(119886119889+1119898

1015840)= 120573119882

119886119889

119882(119886119889119898gt119899)

= 1 + 120573 (1 minus 120583(119886119898)

)sum

119898

1199021015840

119898119882(119886+119903119898 119889+1119898

1015840)

= 1 + 120573 (1 minus 120583(119886119898)

)119882119886119889

(A3)

And by (A3) expressions

119882119886119889= (1 minus sum

119898gt119899

119902119898)120573119882

119886119889+ sum

119898gt119899

119902119898(1 + 120573 (1 minus 120583

(119886119898))119882119886119889)

(A4)

This way isolating119882119886119889

119882119886119889=

sum119898gt119899

119902119898

1 minus 120573 + 120573sum119898gt119899

119902119898120583(119886119898)

(A5)

Analogously for reward elements using (11)

119877(119886119889119898le119899)

= 120573sum

119898

1199021015840

119898119877(119886119889+1119898

1015840)= 120573119877119886119889

119877(119886119889119898gt119899)

= 120573[(1 minus 120583(119886119898)

)sum

119898

1199021015840

119898119877(119886+119903119898 119898

1015840)+ 120583(119886119898)

sdot MOS (119889 + 1)]

= 120573 [(1 minus 120583(119886119898)

) 119877119886119889+ 120583(119886119898)

sdot MOS (119889 + 1)] (A6)

And thus from (A6)

119877119886119889= (1 minus sum

119898gt119899

119902119898)120573119877119886119889

+ sum

119898gt119899

119902119898(120573 [(1 minus 120583

(119886119898)) 119877119886119889+ 120583(119886119898)

sdot MOS (119889 + 1)])

(A7)

Isolating 119877119886119889

119877119886119889=120573sum119898gt119899

119902119898120583(119886119898)

MOS (119889 + 1)1 minus 120573 + 120573sum

119898gt119899119902119898120583(119886119898)

(A8)

Substituting (A5) and (A8) in (A1) and by simplifyingwe obtain

Vlowast(119886119889119899)

= (120573120583(119886119899)

(MOS (119889 + 1)

minus120573sum119898gt119899

119902119898120583(119886119898)

MOS (119889 + 1)1 minus 120573 + 120573sum

119898gt119899119902119898120583(119886119898)

))

times (1 minus 120573120583(119886119899)

sum119898gt119899

119902119898

1 minus 120573 + 120573sum119898gt119899

119902119898120583(119886119898)

)

minus1

= (120573120583(119886119899)

(MOS (119889 + 1) (1 minus 120573 + 120573sum119898gt119899

119902119898120583(119886119898)

)

minus120573sum

119898gt119899

119902119898120583(119886119898)

MOS (119889 + 1)))

times (1 minus 120573 + 120573sum

119898gt119899

119902119898120583(119886119898)

minus 120573120583(119886119899)

sum

119898gt119899

119902119898)

minus1

=120573120583(119886119899)

sdot MOS (119889 + 1) (1 minus 120573)1 minus 120573 + 120573sum

119898gt119899119902119898(120583(119886119898)

minus 120583(119886119899)

)

(A9)

And therefore for the undiscounted case the 120573 rarr 1

limit for expression (A9) is (A2)

Conflict of Interests

The authors declare that there is no conflict of interestsregarding the publication of this paper

14 Abstract and Applied Analysis

Acknowledgment

The research leading to these results has received fundingfrom the European Union Seventh Framework Programme(FP72007ndash2013) under Grant agreement 284863 (FP7 SECGERYON)

References

[1] ITU-T P10G100 Amendment 1 (0107) New Appendix I-definition of Quality of Experience (QoE) 2006

[2] S Aalto and P Lassila ldquoFlow-level stability and performance ofchannel-aware priority-based schedulersrdquo in Proceeedings of the6th Euro NF Conference on Next Generation Internet (NGI rsquo10)IEEE June 2010

[3] L KleinrockQueueing Systems vol 2 ofComputer ApplicationsJohn Wiley amp Sons 1976

[4] J Gittins ldquoBandit processes and dynamic allocation indicesrdquoJournal of the Royal Statistical Society B Methodological vol 41no 2 pp 148ndash177 1979

[5] K Avrachenkovt U Ayesta P Brown and E Nyberg ldquoDif-ferentiation between short and long TCP ows predictabilityof the response timerdquo in Proceedings of the 23rd Annual JointConference of the IEEE Computer and Communications Societies(INFOCOM rsquo04) vol 2 p 762 IEEE 2004

[6] U Ayesta M Erausquin and P Jacko ldquoA modeling frameworkfor optimizing the flow-level scheduling with time-varyingchannelsrdquo Performance Evaluation vol 67 no 11 pp 1014ndash10292010

[7] S Aalto U Ayesta and R Righter ldquoProperties of the Gittinsindex with application to optimal schedulingrdquo Probability in theEngineering and Informational Sciences vol 25 no 3 pp 269ndash288 2011

[8] M Puterman ldquoMarkov decision processesrdquo in Handbooks inOperations Research and Management Science vol 2 pp 331ndash434 1990

[9] C H Papadimitriou and J N Tsitsiklis ldquoThe complexityof Markov decision processesrdquo Mathematics of OperationsResearch vol 12 no 3 pp 441ndash450 1987

[10] J Nino-Mora ldquoDynamic priority allocation via restless banditmarginal productivity indicesrdquo TOP vol 15 no 2 pp 161ndash1982007

[11] P Whittle ldquoRestless bandits activity allocation in a changingworldrdquo Journal of Applied Probability vol 25 pp 287ndash298 1988

[12] U Ayesta M Erausquin M Jonckheere and I M VerloopldquoScheduling in a random environment stability and asymptoticoptimalityrdquo IEEEACM Transactions on Networking vol 21 no1 pp 258ndash271 2013

[13] S Khan S Thakolsri E Steinbach and W Kellerer ldquoQoE-based cross-layer optimization for wireless multiuser systemsrdquoin Proceedings of the 18th ITC Specialist Seminar on Quality ofExperience 2008

[14] S Thakolsri S Khan E Steinbach and W Kellerer ldquoQoE-driven cross-layer optimization for high speed downlink packetaccessrdquo Journal of Communications vol 4 no 9 pp 669ndash6802009

[15] P Ameigeiras J J Ramos-Munoz J Navarro-Ortiz PMogensen and J M Lopez-Soler ldquoQoE oriented cross-layer design of a resource allocation algorithm in beyond 3GsystemsrdquoComputer Communications vol 33 no 5 pp 571ndash5822010

[16] I Taboada F Liberal J O Fajardo and U Ayesta ldquoQoE-aware optimization of multimedia flow schedulingrdquo ComputerCommunications vol 36 no 15-16 pp 1629ndash1638 2013

[17] U Ayesta M Erausquin and P Jacko ldquoResource-sharing in asingle server with time-varying capacityrdquo in Proceeding of the49th Annual Allerton Conference on Communication Controland Computing (Allerton rsquo11) pp 377ndash383 Monticello Ill USASeptember 2011

[18] K Thompson G J Miller and R Wilder ldquoWide-area internettraffic patterns and characteristicsrdquo IEEE Network vol 11 no 6pp 10ndash23 1997

[19] M Fiedler T Hossfeld and P Tran-Gia ldquoA generic quantitativerelationship between quality of experience and quality ofservicerdquo IEEE Network vol 24 no 2 pp 36ndash41 2010

[20] A Richards M Antoniades V Witana and G Rogers ldquoMap-ping user level QoS from a single parameterrdquo in Proceedingsof the International Conference on Multimedia Networks andServices Citeseer 1998

[21] E Ibarrola F Liberal I Taboada and R Ortega ldquoWebQoE evaluation in multi-agent networks validation of ITU-TG1030rdquo in Proceedings of the 5th International Conference onAutonomic and Autonomous Systems (ICAS rsquo09) pp 289ndash294Valencia Spain April 2009

[22] S Aalto and U Ayesta ldquoOptimal scheduling of jobs with aDHR tail in the MG1 queuerdquo in Proceedings of the 3rd Inter-national Conference on Performance Evaluation Methodologiesand Tools (ValueTools rsquo08) article 50 Institute for ComputerSciences Social-Informatics and Telecommunications Engi-neering Athens Greece 2008

[23] V Visweswaran ldquoDecomposition techniques for MILPlagrangian relaxationrdquo in Encyclopedia of Optimization pp632ndash638 Springer New York NY USA 2009

[24] I Taboada J O Fajardo and F Liberal ldquoPerformance analysisof scheduling algorithms for web QoE optimization in wirelessnetworksrdquo Network Protocols and Algorithms vol 4 pp 27ndash432012

[25] J A van Mieghem ldquoDynamic scheduling with convex delaycosts the generalized 119888120583 rulerdquoTheAnnals of Applied Probabilityvol 5 no 3 pp 809ndash833 1995

[26] S Sesia I Toufik and M Baker LTE-The UMTS Long TermEvolution FromTheory to Practice Wiley 2011

[27] P Reichl B Tuffin and R Schatz ldquoLogarithmic laws inservice quality perception where microeconomics meets psy-chophysics and quality of experiencerdquo Telecommunication Sys-tems vol 52 no 2 pp 587ndash600 2013

[28] J C Ikuno M Wrulich and M Rupp ldquoSystem level simulationof LTE networksrdquo in Proceedings of the IEEE 71st VehicularTechnology Conference Taipei Taiwan May 2010

Submit your manuscripts athttpwwwhindawicom

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical Problems in Engineering

Hindawi Publishing Corporationhttpwwwhindawicom

Differential EquationsInternational Journal of

Volume 2014

Applied MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical PhysicsAdvances in

Complex AnalysisJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

OptimizationJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Operations ResearchAdvances in

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Function Spaces

Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of Mathematics and Mathematical Sciences

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Algebra

Discrete Dynamics in Nature and Society

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Decision SciencesAdvances in

Discrete MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Stochastic AnalysisInternational Journal of

2 Abstract and Applied Analysis

Furthermore even though a few QoE and channel-awarescheduling proposals exist [13ndash15] no significant analyticalframework is provided for these heuristics In fact there isno solution based on relevant mathematical methods such asMDP for modelling and Whittle method for its resolution inorder to obtain a simple scheduling index rule

On the other hand it is worth mentioning the workcarried out in [16] which is pioneer in introducing QoE-awareness in a MDP Moreover since the proposed modelis analytically unsolvable in general they achieve an easytractable and well-performing QoE-aware index rule basedon Gittins approach [4] for constant channel capacity whichthey denominate Gittins MOS (GM) It is known that Gittinsproposed a method to obtain the so-called Gittins indexwhich originally minimized mean delay for constant channelcapacity However as concluded in [17] Gittins approach isunsuitable for solving problems with time-varying channel

12 Scope and Contributions Since the stochastic anddynamic resource allocation problem of subjective qualitymaximization in channels with time-varying capacity isanalytically and computationally unfeasible for finding anoptimal solution the main objective of this work is to designa simple and tractable heuristic priority scheduling rule usinganalytical tools that have had a great contribution in theoptimization area

Thus the main contribution of this paper is threefold

(i) Firstly we propose a QoE-aware MDP model in atime-varying channel context

(ii) Secondly in order to achieve a tractable solution forthe aforementioned QoE and channel-aware modelwe focus on designing a simple heuristic index ruleusing Whittle approach

(iii) Thirdly although for many years exponential flowsize distributions have been considered for trafficmodelling in order to simplify the resolution ofscheduling optimization problems as a step forwardin this work we take into account size distributionsthat better capture the real world patterns

The remainder of the paper is organized as follows Firstof all we characterize the QoE-aware scheduling problemfor the time-varying channel context in Section 2 ThenSection 3 describes the extension of the basic QoE-awareMDP model to time-varying channels In Section 4 weachieve the Whittle index-based solution for the proposedMDP and its performance is evaluated in Section 5 FinallySection 6 gathers the main conclusions of this paper

2 Problem Description

We analyze the problem of maximizing average QoE in time-varying channels Even though this study is applicable to anytime-varying channel context we focus onwireless networksIn particular we centre on a wireless downlink data channelin a single cell system In this way at the beginning of eachtransmission time interval (TTI) the scheduler located in the

base station makes decisions in order to choose a user trafficflow to transmit

21 User Each user 119896 in the system is uniquely associatedwith the flow related to its requested content with its QoEcharacteristic and with its wireless channel We use a genericconcept of flow which refers to an amount of data that canbe displayed to (received by) the end user as a standaloneelement

We consider a system with a fixed number of userswithout arrivals of new users This assumption simplifies themathematical model However being aware of the impact ofarrivals on the performance of scheduling we will analyzethe performance of scheduling strategies in the presence ofarrivals

211 Flow Size Flows are characterized by their random size119883 the total amount of bits to be transferred Sizes are assumedto be independently distributed with E[119883] lt infin where119865(119909) = P(119883 le 119909) 119891(119909) and ℎ(119909) = 119891(119909)(1 minus 119865(119909)) arethe corresponding cumulative distribution function densityfunction and hazard rate respectively

We focus on the important class of size distributionswith a decreasing hazard rate particularly we assume Paretodistributed flow sizes It is known that Internet traffic flowsizes are properly modelled by means of Pareto distributions[18] which better fit Internet flow sizes compared withthe simplistic exponential approach We define the Paretodistribution with shape parameter 120572 gt 1 and scale parameter120574 gt 0 whose density function for all 119909 ge 0 is

119891 (119909) =120574120572

(1 + 120574119909)120572+1

(1)

Note that even though we use Pareto distributed flowsizes the results provided are valid for any size distributionwith a decreasing hazard rate

212 QoE Characteristic The time needed to transmit tochannel all the bits from a user flow has a direct impact onuserrsquos perceived quality We consider that this delay is themain cause of subjective quality distortion Nowadays mostof the Internet traffic is transported over TCP which imple-ments the packet recovery mechanisms to avoid applicationlevel losses In TCP-based services the predominant sourceof losses at the application is packets arriving later than theirplayout time Hence it is generally useful to provide a delay-driven QoE-awareness

This way we quantify QoE by a delay dependent MeanOpinion Score (MOS) [19] utility function User satisfactionis evaluated in MOS scale of 1 (very poor quality) to 5(excellent quality)

As shown in QoE studies [20] perceived quality ismaximum until a low delay value 119889min Then from this delaythreshold QoE degradation happens until a high delay value119889max Above this delay threshold service is unaffordableresulting in a MOS value of 1 The values of these delaythresholds depend on several factors [21] such as the flowsize the service type the network type and the user expertise

Abstract and Applied Analysis 3

0 5 10 15 20 25 301

15

2

25

3

35

4

45

5

d (s)

MO

S

dmaxdmin

Figure 1 MOS versus delay

We illustrate an example of the explained behavior inFigure 1

213 Channel Model The channel quality associated to auser 119896 evolves randomly and independently of its evolutionhistory We assume that the channel can be in N =

1 2 119873119896 conditions Therefore user 119896 is in channel

condition 119899 with probability 119902119896119899 having sum

119899isinN 119902119896119899

= 1In channel state 119899 119903

119896119899bits are transmitted to channel and

transmission rates 119904119896119899

are 1199041198961le 1199041198962le sdot sdot sdot le 119904

119896119873119896

22 Server The objective of the scheduler is finding a policy120587 that solves the following optimization problem

max120587

1

119870

119870

sum

119896=1

MOS119896(119889119896) (2)

According to the already described aspects about the schedul-ing problem we consider a set of admissible policies Πwhich fulfils the next properties

(i) Delay dependency the time that a flow remains in thesystem 119889 is known

(ii) QoE-awareness the delay-dependent MOS functionthat characterizes user subjective quality is known

(iii) Channel-awareness channel quality indicator (CQI)information sent from mobile users is used as userrsquosinstantaneous channel condition information

(iv) Nonanticipation similarly to most current IP sys-tems flow size is unknown However existent nonan-ticipating size-based disciplines make use of flowattained service [4 7] the bits that have been trans-ferred of a flow Due to the memoryless property ofthe exponential distributions for these distributionsthe hazard rate is independent of attained serviceNevertheless the considered Pareto distributionshave a decreasing hazard rate with attained service[22] this is completion probability 120583

(119886119899) is higher

for lower attained service values This way attainedservice 119886 is used as size information

(v) Preemption we assume that the server is preemptivethat is at every decision epoch it is permitted tosuspend the service of a flow whose transmission isunfinished

(vi) Single service only the transmission of a flow isallowed in each TTI

3 MDP Formulation

In this section we formulate the problem described inSection 2 as a MDP First we provide the QoE-aware MDPmodel of each user 119896 Then we formulate the optimizationproblem for the joint MDP model which takes into accountall the users in the system

31 MDP Model of a Job The action space B of user 119896is binary action 0 means not serving and if action 1 ischosen 119903

119896119899bits from this user being in channel state 119899

are transmitted to channel Besides each user 119896 is definedby tuple (S

119896 (R119887119896119904)119887isinB

(W119887119896119904)119887isinB

(P119887119896119904)119887isinB

) These elementsare defined as follows

(i) S119896= (A119896timesD119896times 1 2 119873

119896) cup [D

119896+ 1] cup [lowast]

is the set of all 119904 states for a user 119896 which is classifiedin three groups

(a) Unfinished states all the states in which flowtransmission is uncompleted belong to thisgroup These states have three dimensionsdenoted as 119904 = (119886 119889 119899) with attained servicelevel 119886 isin A

119896 delay level 119889 isin D

119896 and

channel condition 119899 isin 1 2 119873119896 as com-

ponents The attained services that correspondto these attained service levels are multiples ofthe first not null 119903

119896119899transmission rate whereas

the delays that correspond to delay levels aremultiples of TTI

(b) Reward states these 119904 = [119889] states are reachedonce flow transmission has finished and inthese delay-dependent states the QoE-awarereward is given This way aimed at maximizingaverage MOS a delay dependant QoE-awarereward giving only happens once the wholeflow has been transmitted since from sub-jective quality perspective the MOS functiononly makes sense once flow transmission iscompleted (see [16] for more details)

(c) Final state the chain of states ends in theabsorbing state 119904 = [lowast]

(ii) R119887119896= (119877

119887

119896119904)119904isinS119896

where 119877119887119896119904

is the expected one-slotreward received for user 119896 at state 119904 if action 119887 isdecided at the beginning of a TTI

119877119887

119896(119886119889119899)= 0 119877

119887

119896[119889]= MOS (119889 + 1) 119877

119887

119896[lowast]= 0

(3)

where MOS(119889 + 1) is normalized MOS in the range[0 1]

4 Abstract and Applied Analysis

(iii) W119887119896= (119882

119887

119896119904)119904isinS119896

where119882119887119896119904

is the expected one-slotwork done for user 119896 at state 119904 if action 119887 is decided atthe beginning of a TTI

1198820

119896119904= 0 119882

1

119896119904= 1 (4)

(iv) P119887119896= (119901119887

119896(119904 1199041015840

))1199041199041015840isinS119896

where 119901119887119896(119904 1199041015840

) is the probabil-ity of user 119896 of moving from state 119904 to state 1199041015840 if action119887 is decided at the beginning of a TTI

1199010

119896((119886 119889 119899) (119886 119889 + 1119898)) = 119902

119896119898

1199011

119896((119886 119889 119899) (119886 + 119903

119899 119889 + 1119898)) = 119902

119896119898sdot (1 minus 120583

119896(119886119899))

1199011

119896((119886 119889 119899) [119889 + 1]) = 120583

119896(119886119899)

1199010

119896([119889] [lowast]) = 119901

0

119896([lowast] [lowast]) = 1

(5)

where 120583119896(119886119899)

= P(119886 lt 119883119896le 119886 + 119903

119896119899| 119883119896gt

119886) Observe that the probability of changing fromchannel state 119899 to channel state119898 corresponds to thesteady-state probability 119902

119896119898due to the memoryless

property of channel evolution

Thus the dynamics of user 119896 is captured by state process119904119896(119905) isin S

119896and action process 119887

119896(119905) isin B Moreover we

highlight that the proposedMDPmodel has restless propertywhich admits evolution and rewards even if not chosenfor transmission due to the delay increase and the channelstochastic evolution

Figure 2 shows part of the per-user state diagram of thenew QoE-aware MDP model which summarizes and relatesthe aforedescribed components

32 Optimization Problem Besides the optimization prob-lem (2) associated with such MDP can be written as

max120587isinΠ

E120587

0[

infin

sum

119905=0

sum

119896isinK

120573119905

119877119887119896(119905)

119896((119886119896(119905)119889119896(119905)119899119896(119905))cup[119889119896(119905)])]

sum

119896isinK

119887119896(119905) = 1 forall119905

(6)

Note that we consider the undiscounted reward case wherethe discounted factor 120573 is 1 being reward deterioration intime implicit in the delay-dependent MOS function

Nonetheless optimally solving problem (6) is unfeasiblein general On the one hand when channel capacity isconstant the impossibility of obtaining the exact QoE-awaresolution is revealed in [16] and hence since the time-varyingchannel model is more complex the achievement of theoptimal solution is unviable in this case On the other hand itis known that the allocation constraint causes intractability insimilar problems [11] and therefore in our case having three-dimensional states finding the optimal solution is extremelydifficult or even unachievable In this way once seeingthe impossibility of obtaining the optimal solution for theconsidered problem in this work we will focus on achievingan approximate solution For that purpose in the next sectionwe will propose a heuristic based on Whittle method

(a d + 1 1)

(a d + 1N)

q1

qN

(a d n)

120583(an)

q1(1 minus

120583 (an))

qN(1 minus 120583

(an) )

[d + 1]

b = 1

b = 0 R = 0

(a + rn d +

1N)

(a + rn d +

1 1)

lowast

R = MOS(d + 1)

Figure 2 A part of per-user state diagram

4 QoE-Aware Index Rule Proposal Based onWhittle Approach

In this part we provide aWhittle index rule type approximatesolution to problem (6) based on Whittle method Theidea of Whittle [11] consists in achieving a function thatmeasures the dynamic service priority in order to obtain asimple scheduling index rule Whittle indices are computedin isolated way for each user and they measure the expectedefficiency of serving a user in each state

Note that we are only interested in the indices of (119886 119889 119899)states as once flow transmission is completed schedulingmakes no sense Thereby the optimization problem formu-lated in (6) can be relaxed by requiring to serve a job per sloton average as proposed in [11] which is further approachedby Lagrangian methods [23] and can be decomposed into asingle-job price-based parameterized optimization problemFor a price or Lagrangiar parameter V we will therefore studythe user-119896 subproblem

max120587119896isinΠ

infin

sum

119905=0

E120587

0120573119905

sdot [119877119887119896(119905)

119896(119886119896(119905)119889119896(119905)119899119896(119905))minus V119882119887119896(119905)119896(119886119896(119905)119889119896(119905)119899119896(119905))

]

(7)

The Lagrangiar parameter V can be interpreted as the per-slotcost of serving

Let us define serving set F sube S119896 which prescribes to

serve a user 119896 if (119886 119889 119899) isin F while not to serve this user if(119886 119889 119899) notin F We will refer to states (119886 119889 119899) isin F as activeand (119886 119889 119899) notin F as passive This way it is possible to rewriteproblem (7) as

maxFsube119878119896

RF119896(119886119889119899)

minus VWF119896(119886119889119899)

(8)

whereRF(119886119889119899)

andWF(119886119889119899)

are respectively the expected totalreward and expected total work in (119886 119889 119899) state

Abstract and Applied Analysis 5

Conjecture 1 Problem (8) is indexable

Due to the complexity of our model proving the index-ability of Problem (8) is not an easy task In this way weassume Conjecture 1 and therefore we suppose that foreach (119886 119889 119899) state of user 119896 the Whittle index exists Fromnow on we omit user label 119896 In spite of considering theundiscounted case the mathematical analysis is carried outfor the discounted case so as to avoid infinite values causedby 1minus120573 terms in denominators for 120573 = 1 and then we obtainthe undiscounted index values in the limit 120573 rarr 1

Following Whittle index definition in [10] we formallywrite the Whittle index for Problem (8) Vlowast

(119886119889119899) as

Vlowast(119886119889119899)

= VF(119886119889119899)

=R⟨1F⟩

(119886119889119899)minusR⟨0F⟩

(119886119889119899)

W⟨1F⟩

(119886119889119899)minusW⟨0F⟩

(119886119889119899)

(9)

Whittle index VF(119886119889119899)

represents the rate betweenmarginal reward and marginal work where the marginalreward (work) is the difference of the expected reward earned(work required) by serving and not serving at the initial state(119886 119889 119899) and employing policyF afterwards

Lemma 2 For any state (119886 119889 119899) and under any policy F weobtain

VF(119886119889119899)

= (120573[(1 minus 120583(119886119899)

) sum

119898isinN

119902119898R

F

(119886+119903119899 119889+1119898)+ 120583(119886119899)

sdot MOS (119889 + 1) minus sum

119898isinN

119902119898R

F(119886119889+1119898)

])

times (1 + 120573[(1 minus 120583(119886119899)

) sum

119898isinN

119902119898W

F

(119886+119903119899 119889+1119898)

minus sum

119898isinN

119902119898W

F(119886119889+1119898)

])

minus1

(10)

Proof From the definition of reward and work respectivelywe have

RF(119886119889119899)

=

R⟨1F⟩

(119886119889119899)= 120573[(1 minus 120583

(119886119899)) sum

119898isinN

119902119898RF

(119886+119903119899 119889+1119898)

+ 120583(119886119899)

sdot MOS (119889 + 1) ]

(119886 119889 119899) isin F

R⟨0F⟩

(119886119889119899)= 120573 sum

119898isinN

119902119898RF(119886119889+1119898)

(119886 119889 119899) notin F

(11)

(1)F0= 0

(2) for 119894 = 1 rarr number of states do(3) (119886 119889 119899)

119894isin argmax VF119894minus1

(119886119889119899) (119886 119889 119899) notin F

119894minus1

(4) Vlowast(119886119889119899)119894

= VF119894minus1(119886119889119899)119894

F119894= F119894minus1cup (119886 119889 119899)

119894

(5) end for

Algorithm 1AG-algorithm for Problem (8)

WF(119886119889119899)

=

W⟨1F⟩

(119886119889119899)= 1 + 120573 (1 minus 120583

(119886119899)) sum

119898isinN

119902119898WF(119886+119903119899 119889+1119898)

(119886 119889 119899) isin F

W⟨0F⟩

(119886119889119899)= 120573 sum

119898isinN

119902119898WF(119886119889+1119898)

(119886 119889 119899) notin F

(12)

By substituting the expressions (11) and (12) in formula(9) we achieve a more complete VF

(119886119889119899)(10) formulation

Nevertheless in order to obtain an analytically tractableWhittle index expression for (10) in the state (119886 119889 119899) it isnecessary to determine the policyF of the future states thathave influence on the index computation of this state As canbe observed from (10) (11) and (12) expressions in thesefuture states the value of attained service is maintained orincreased whereas delay grows over time Since increasingattained service decreases instant completion probability fora Pareto distribution and increasing delay degrades QoE itwould seem intuitively correct to suppose that future stateswith the same or worse channel conditions are passive Insuch a way we could say that in a state with the same or worsechannel condition being attained service and delay higherthat the Whittle index value is lower However for futurestates with better channel conditions their passive or activeproperty is not trivial because the improvement in channelcondition does not guarantee being a better state due to theaforementioned behavior of the increasing attained serviceand delayTherefore finding the active set of all the influentialstates for Whittle index computation becomes a challengingtask

In such situation aimed at achieving the characteristicsof the optimal policy and the subsequent active set buildingevolution we will use an algorithm called Adaptive-GreedyshortlyAG-algorithm (see [10] for a survey) This algorithmcomputes Whittle indices numerically taking into accountoptimal active sets This way we have implemented AG-algorithm for our case study in order to verify the fundamen-tal properties of the Whittle index for our problem whichwill be useful to determine the active set and consequentlya closed-form index expression We provide the operation ofthe applied algorithm in Algorithm 1

We have performed several numerical experiments for120573 = 1 case in which we have reduced the state space ofboth delay and channel states so as to do the execution of thealgorithm computationally and in time viably We have onlyconsidered two channel conditions and eleven levels of delay

6 Abstract and Applied Analysis

times10minus3

6

5

4

3

2

1

00

0

5 5

10

10times107

ad

Vlowast (ad1)

(a)

0

0

5 5

10

10times107

a d

Vlowast (ad2)

001

0008

0006

0004

0002

0

(b)

Figure 3 Whittle index values obtained from AG-algorithm for two channel conditions and different attained services and delays For aPareto distribution with 120572 = 15 and E[119883] = 5Mb 119902

2= 05 119903

1= 84Kb 119903

2= 168Kb 119889min = 2 s and 119889max = 9 s

0

0

5 5

10

10

times107

ad

Vlowast (ad1)

001

0008

0006

0004

0002

0

q2 = 01

(a)

times10minus3

4

3

2

1

0

0

0

5 5

10

10

times107

a d

Vlowast (ad1)

q2 = 09

(b)

Figure 4 Whittle index values obtained fromAG-algorithm for different channel state probabilities For a Pareto distribution with 120572 = 15and E[119883] = 5Mb 119903

1= 84Kb 119903

2= 168Kb 119889min = 2 s and 119889max = 9 s

in the scenarios In such a way from experimental results weconjecture the following fundamental properties

(1) TheWhittle index value depends on attained servicedelay and channel state Moreover as can be seenin graphs from Figures 3 and 4 we can clearlydistinguish the ranges of MOS function above theupper delay threshold the index value is null

(2) For the same channel condition and the samedelay level Whittle index values are decreasing withattained service Vlowast

(119886119889119898)gt Vlowast(1198861015840119889119898)

forall1198861015840

gt 119886 (illus-trated in Figures 3 and 4)

(3) For the same channel condition and the same attainedservice levelWhittle index values are decreasing withdelay Vlowast

(119886119889119898)gt Vlowast(119886119889+1119898)

(See Figures 3 and 4)

Abstract and Applied Analysis 7

(4) In a better channel condition for the same attainedservice level and the same delay level the Whittleindex value is higher Vlowast

(1198861198891198981)gt Vlowast(1198861198891198982)

forall1198981gt 1198982

(Observe Figure 3)(5) Opposite to the previous channel-aware work [6]

where Whittle indices in the best channel conditionare infinite having absolute priority in this channelstate in our case being in the best channel conditiondoes not guarantee that the Whittle index value ishigher than in the rest of the channel conditions it ispossible to fulfil Vlowast

(119886119889119898lt119873)gt Vlowast(11988610158401198891015840119873)

If we comparethe two graphs in Figure 3 it can be easily seen howfor several attained services andor delayed values theWhittle index values in the worst channel conditionare higher than in the best channel condition (takingalso into account nonnull values)

(6) Dependency on channel condition probabilities hilefor channel condition 119873 the Whittle index valueremains invariant with channel state probability forthe rest of channel conditions its value decreases aslong as channel state probabilities increase in betterchannel conditions (shown in Figure 4)

Once we analyzed the properties of theWhittle index weset out to define the structure of the active set in order toobtain an analytically and computationally tractable Whittleindex expression In such a way to derive a closed-formcharacterization of the Whittle index of (119886 119889 119899) state first ofall it is necessary to specify the activity of (119886 119889 + 1119898) and(119886 + 119903

119899 119889 + 1119898) states As concluded from Whittle index

properties it is known that (119886 119889+1119898 le 119899)cup(119886+119903119899 119889+1119898 le

119899) states are passive However it is not trivial to specify abovewhich attained service level andor delay level if (119886 119889+1119898 gt

119899)cup (119886+ 119903119899 119889 + 1119898 gt 119899) states with better channel condition

are active or passiveNonetheless in the best channel the active set is totally

specified since all the states that compute in the index are pas-sive because of nonexisting channel improvement and beingin the future attained service equal or higher and delay higherAs shown in Proposition 3 for the best channel condition theWhittle index has a closed-form expression where the indexvalue is the multiplication between completion probabilityand normalizedMOS function Note that the obtainedWhit-tle index expression in the best channel state is equivalent tothe GM index achieved for constant channel [16]

Proposition 3 The Whittle index for Problem (8) in the bestchannel condition is given by

Vlowast(119886119889119873)

= 120583(119886119873)

sdot MOS (119889 + 1) (13)

Proof It is known that for Vlowast(119886119889119873)

index computation thatF = 0 In this way using (11) and (12) RF

(119886119889+1119898)=

120573sum119898isinN 119902

119898RF(119886119889+2119898)

RF(119886119889+2119898)

= 120573sum119898isinN 119902

119898RF(119886119889+3119898)

RF(119886119889gt119889max 119898)

= 0 Hence analogolously RF(119886+119903119899 119889+1119898)

= 0WF(119886119889+1119898)

= 0 and WF(119886+119903119899 119889+1119898)

= 0 Substituting thesenull terms in expression (10) and being 120573 = 1 we achieve(13)

Besides as stated before obtaining a computationallytractable Whittle index expression for any channel conditionis not possible In such situation in order to achieve atractable Whittle-based QoE-aware index rule for time-varying channels we propose a Whittle-based approximateheuristic in the next subsection

41 Approximate Whittle Index Proposal In the following wepropose an approximate solution for the QoE-aware Whittleindex in a time-varying channel context (10)

First of all we will determine the structure of the activeset For the Whittle index computation of (119886 119889 119899) state aspreviously concluded it is known that (119886 119889 + 1119898 le 119899) cup (119886+

119903119899 119889 + 1119898 le 119899) states are passive but we need to define the

unknown activity of (119886 119889 + 1119898 gt 119899) cup (119886 + 119903119899 119889 + 1119898 gt 119899)

states Thus so as to determine if these states are active orpassive we apply the following two considerations

(1) 120583 approximation

120583(119886119898)

asymp 120583(119886+119903119897 119898)

(14)

(2) MOS approximation

MOS (119889) asymp MOS (119889 + 1) (15)

On the one hand we assume that for any 119897 = 1 2 119873

value that 120583(119886119898)

asymp 120583(119886+119903119897 119898)

(14) It is known that (119886 119889119898 gt

119899) states are active and being 120583(119886119898)

asymp 120583(119886+119903119899 119898)

we couldconsider that (119886 + 119903

119899 119889 119898 gt 119899) states are active

On the other hand we assume MOS(119889) asymp MOS(119889 + 1)(15) since QoE degradation in a higher delay level is neg-ligible in the millisecond scale that delay evolves Takinginto account the approximation about MOS function being(119886 119889119898 gt 119899) states active we can suppose that (119886 119889 + 1119898 gt

119899) states are also active This way combining approximations(14) and (15) we consider that (119886 + 119903

119899 119889 + 1119898 gt 119899) states are

activeThen suppose that the aforementioned 120583 and

MOS approximations cause 119877119886119889

= sum119898119902119898119877(119886119889119898)

asymp

sum119898119902119898119877(119886+119903119897 119889+1119898)

and 119882119886119889

= sum119898119902119898119882(119886119889119898)

asymp

sum119898119902119898119882(119886+119903119897 119889+1119898)

So using these reward and workapproximations the first simplified expression for theWhittle index formulation (10) is

Vlowast(119886119889119899)

=120573120583(119886119899)

(MOS (119889 + 1) minus 119877119886119889)

1 minus 120573120583(119886119899)

119882119886119889

(16)

In such a way applying the previous simplificationswe obtain the Whittle index approximation presentedin Proposition 4 what we call attained service PI MOS(ASPIM) The methodology to compute the closed-formexpression of this QoE-aware Whittle-based index is pre-sented in the Appendix

Proposition 4 The formulation of the ASPIM index is

119860119878119875119868119872 = Vlowast(119886119889119899)

= lim120573rarr1

120573120583(119886119899)

sdot MOS (119889 + 1) (1 minus 120573)1 minus 120573 + 120573sum

119898gt119899119902119898(120583(119886119898)

minus 120583(119886119899)

)

(17)

8 Abstract and Applied Analysis

Table 1 CQIs and corresponding transmission rates (Mbps)

CQI 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15119904 0 42 672 84 11256 168 2184 252 2688 336 4468 504 5376 672 758 8064

If we analyze expression (17) we observe that this QoE-aware and channel-aware Whittle-based ASPIM index isimplementable This proposal depends on QoE size andchannel properties Moreover the obtained index for thebest channel condition is equal to the original Whittle index(13) since the summation in the denominator becomes nullHowever as shown in (18) when channel state is not the bestthe index value goes to zero

ASPIM119899 =119873

= lim120573rarr1

120583(119886119899)

sdot MOS (119889 + 1) (1 minus 120573)sum119898gt119899

119902119898(120583(119886119898)

minus 120583(119886119899)

)997888rarr 0

(18)

Hence a user in its best channel state with nonnullinstantaneous normalized MOS function has priority overa user which is not in its best channel condition Thus wesummarize the proposed QoE-aware Whittle-based ASPIMindex rule in Definition 5

Definition 5 The ASPIM index rule consists of at everydecision slot

(i) serving the user in its best channel condition withnonnull instantaneous normalized MOS functionwith the highest value of 120583

(119886119873)sdot MOS(119889 + 1)

(ii) if there is no user in its best channel condition withnonnull instantaneous normalized MOS functionserving the user with the highest value of (120583

(119886119899)sdot

MOS(119889 + 1))sum119898gt119899

119902119898(120583(119886119898)

minus 120583(119886119899)

) (using (18)normalized by (1 minus 120573))

So if channel condition is not the best the index valueis the ratio of the multiplication between the actual servicecompletion and the instantaneous normalizedMOS functionrespect to the expected potential improvement of the comple-tion probability

Therefore in the context of time-varying channels aimedat maximizing average MOS we have proposed ASPIMscheduling algorithm which is a simple and tractable QoE-aware channel-aware and size-based index rule Neverthe-less in order to verify the correct behavior of this heuristicwe analyze its performance in the subsequent section

5 Performance Evaluation

In this section we evaluate the performance of the proposedASPIM index rule (see proposal in Definition 5) To that endwe compare the QoE performance of our heuristic with theone achieved by well-known scheduling strategies in severalsimulation scenarios Below we provide a brief description ofthe scheduling priority rules used in our experiments

(i) Max rate (MR) this opportunistic policy consists inserving the user with the best channel capacity

(ii) Proportional fair (PF) this channel-aware disci-pline serves the user with the highest ratio betweeninstantaneous transmission rate and current attainedthroughput

(iii) Cost and attained service dependant 120583 (119888AS120583)although it might seem counter-intuitive minimizingmean delay does not result in maximizing averageMOS [24] This way in order to compare the per-formance with delay-aware solutions we propose toadapt the classical 119888120583 rule aimed at minimizing meandelay [25] to attained serviceThis adaptation consistsof serving the user with the highest value of 119888120583

(119886119899)

being 119888 the waiting cost paid for every slot while flowtransmission is uncompleted

(iv) GM [16] thisQoE-aware pioneering index rule origi-nally proposed for constant channel capacity choosesthe user with the highest value of 120583

(119886119899)sdot MOS(119889 + 1)

In case of ties we use random tie-breaking ruleConcerning traffic flows in the system we use the Pareto

size distribution given in (1) in simulations whose 120572 and 120574parameters determine mean flow size We consider typicalmean sizes used in wireless networks 05Mb (small traffica web page email) 5Mb (medium traffic a PDF documenta picture etc) and 50Mb (MP3 audio a group of picturesfrom a video sequence etc) User traffic flows arrive to thescheduler according to a Poisson process with 120582 rate

Referring to network characteristics we use transmis-sion rates that depend on CQIs used in real wirelessnetworks which are associated with 4G modulation andcoding schemes We show the mapping between CQI valuesand transmission rates in Table 1 The presented values areadapted and extrapolated from [26] for a cell of 20MHz

Moreover we consider different network loads in order toanalyze the behavior of scheduling strategies under differentnetwork conditions Flow arrival rate determines networkload 120588 where 120588 = 120582 sdot (E[119883]119904

119873) We consider seven network

states in this study low load (120588 = 025 0375) medium load(120588 = 05 075) and high load (120588 = 085 09 095)

Regarding QoE aspects as typically utilized in perceivedquality studies [27] we use a log-shaped MOS function forQoE degradation presented in (19) This way employing theMOS curve modelling provided in [21] as starting point andtaking into account the flow size the network type and theuser expertise we have suitably defined 119889min and 119889max delaythresholds used in simulations and consequently 119888

1and 1198882

MOS coefficients Consider

MOS = 1198881minus 1198882sdot log (119889) (19)

We have implemented the whole network environmentfor the scheduling of network resources in MATLAB Inrelation to the performed experiments for each scenario

Abstract and Applied Analysis 9

Table 2 Parameter set in the experimental scenarios

Scenario MOS Size Channel 119888

1 119889min = 05 s 119889max = 10 s 120572 = 15 120574 = 4 sdot 10minus7 CQI = 3 5 11198881= 407 119888

2= 133 E[119883] = 5Mbit 119902

2= 05

2 119889min = 04 s 119889max = 06 s 120572 = 15 120574 = 4 sdot 10minus7 CQI = 3 5 11198881= minus403 119888

2= 986 E[119883] = 5Mbit 119902

2= 05

3 119889min = 05 s 119889max = 10 s 120572 = 15 120574 = 4 sdot 10minus7 CQI = 3 9 11198881= 407 119888

2= 133 E[119883] = 5Mbit 119902

2= 05

4 119889min = 005 s 119889max = 1 s 120572 = 15 120574 = 4 sdot 10minus6 CQI = 3 5 11198881= 1 1198882= 133 E[119883] = 05Mbit 119902

2= 05

5

119889min1 = 05 s 119889max1 = 5 s 1205721= 15 120574

1= 4 sdot 10

minus7

see Table 3 5 111988811= 379 119888

21= 173 E[119883

1] = 5Mbit

119889min2 = 5 s 119889max2 = 30 s 1205722= 15 120574

2= 4 sdot 10

minus8

11988812= 859 119888

22= 223 E[119883

2] = 50Mbit

Table 3 Channel state probabilities in Scenario 5

CQI 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15119902 028 012 009 008 008 008 007 006 005 004 003 001 0009 00005 00003 00002

for each combination of scheduling discipline and networkload we have carried out a set of 10 rounds of 10000 slength simulations These rounds differ in the randomlypregenerated traces (input vector) of sizes and arrivals Thusnot only the overall average MOS values will be provided butwe will also include their 95 confidence intervals

Besides in order to guarantee that the obtained perfor-mance results are generally valid we will analyze differentsimulation scenarios We have chosen five relevant settingswhich differ inQoE size channel orand cost characteristicsThe parameters of these scenarios are summarized in Table 2Note that in the first four scenarios we restrict ourselves tothe case of two channel conditions so as to get fundamentalperformance results

We have carefully selected the scenarios used in simula-tions First of all we define a setting (Scenario 1) in which thevalues of all the parameters are typical or standard the valuesof these parameters introduce low error in the approxima-tions considered for our ASPIM scheduler proposal Thenin order to evaluate worst performance cases for ASPIMwe have chosen three scenarios (Scenario 2 Scenario 3 andScenario 4) in which the error of the approximation used forASPIM is high due to a different factor (gradient ofMOS rateand size resp)

Apart from that it is worthmentioning that for obtainingthe ASPIM index rule that we have considered an isolatedMDP model per user Hence the achieved solution is basedon a unique user type or single class However it would beinteresting to examine the proposal when there are differenttypes of users which is analyzed in Scenario 5 Furthermorethis last setting resembles a real 4G wireless network Notethat even though this technology allows the simultaneoustransmission of multiple flows per TTI we assume that asingle flow transmits in each TTI

03 04 05 06 07 08 09

25

3

35

4

45

5

E[M

OS]

ASPIMMRPF

cAS120583GM

120588

Figure 5 E[MOS] for the basic scenario

Next we describe the results we have achieved in theaforementioned scenarios

51 Scenario 1 Basic Case In this first family of simulationswe consider a basic or typical scenario which takes intoaccount the equiprobable channel case and medium-sizedself-similar flows Figure 5 shows averageMOS results for thissetting employing different scheduling policies and networkloadsThemain conclusion achieved from these results is thatASPIM outperforms the rest of disciplines which is betterthan the QoE-aware GM rule proposed for constant channelcapacity Moreover contrary to the other policies it holds

10 Abstract and Applied Analysis

0 02 04 06 08 10

1

2

3

4

d (s)

PDF(d

)

(a)

1 2 3 4 50

01

02

03

04

05

06

07

MOS

PDF(

MO

S)

ASPIMMRPF

cAS120583GM

(b)

1 2 3 4 50

02

04

06

08

1

MOS

CCD

F(M

OS)

ASPIMMRPF

cAS120583GM

Satisth

(c)

Figure 6 Statistics in Scenario 1 delay PDF (a) MOS PDF (b) and MOS CCDF (c)

averageMOSvalues in the same level for all120588 Referring to thetraditional channel-aware disciplines both PF and MR givebad behavior comparing to the rest of scheduling strategies

In order to justify the previous performance results wenow analyze delay andMOS statistics for the highest networkload considered In such a way as regarded in Figure 6(a) forthe Probability Density Function (PDF) of delay we highlightthe probabilitymass or peak that appears for some schedulingdisciplines before the low delay threshold (05 s) The heightand weight of a peak belonging to a policy have direct impacton its average MOS ASPIM discipline shows the highestand the widest peak among the presented ones followedby GM but with a notable difference Therefore our novelQoE-aware policy seems to try to finish the transmissionof most of the flows before 119889min which is reflected in theaforementioned peaks Note that the cAS120583 rule aimed atminimizing delay behaves in different manner which showsmore fairness among delays in the range shown compared tothe QoE-aware ones

What is previously concluded about delay statistics hasa direct reflection on QoE statistics This way as can beseen in the PDF of MOS presented in Figure 6(b) if theobjective is to maximize average QoE it is important onthe one hand to be the height and the weight of the peaksaround the maximum MOS high and on the other hand tobe the value around the minimum MOS low ASPIM fulfilsthese positive properties and consequently it is the bestin average QoE terms Besides if we analyze the Comple-mentary Cumulative Distribution Function (CCDF) of MOSprovided in Figure 6(c) we observe that ASPIM gives thehighest probability of being MOS value higher Consideringfor example a QoE satisfaction threshold satisth of MOS =4 in this point ASPIM improves GM a 10 whereas thedeterioration of MR and PF is higher than a 50

52 Scenario 2 High Demanding Users Case This scenarioreflects the case of users with low tolerance to delay in

Abstract and Applied Analysis 11

03 04 05 06 07 08 09

05

06

07

08

09

ASPIMMRPF

cAS120583GM

120588

E[M

OS]

2E

[MO

S]1

Figure 7 E[MOS] for abrupt MOS scenario

which theMOS function in theQoE degradation range showsa great slope Thus in this setting the error of the MOSapproximation (15) is high since the gradient of MOS amongconsecutive delay levels becomes significant In Figure 7we compare the results of this second scenario and thebasic scenario From the illustrated graph we could say thatour QoE-aware proposal adapts to any MOS function Thisis whereas for the rest of disciplines (excluding GM) thenegative effects due to the abrupt degradation of the QoEutility function are appreciable ASPIMmanages to avoid thisadapting to any MOS function

53 Scenario 3 Higher Error Case due to Rate During thesimplification process carried out for obtaining the ASPIMindex rule in Section 41 the 120583 approximation used (see (14))introduces a rate dependant error Note that increasing therate in the good channel state makes bigger the error of the 120583approximation used Therefore we will analyze the obtainedperformance for our new QoE-aware proposal under thishigher error due to rate In the previous scenarios we haveassumed that the transmission rate in the good channelcondition is twice the rate in the bad channel state In thissetting we consider that the transmission rate in the goodchannel is four times the transmission rate in the bad channelThis way according to the QoE results presented in Figure 8in spite of a higher error caused by rate ASPIM is superior tothe rest of policies

54 Scenario 4 High Error Case due to Size Besides it isknown that for a distribution with a decreasing hazard ratesuch as Pareto that the gradient of completion probabilityincreases as long as the mean size decreases which makesthe error of the 120583 approximation (14) for the ASPIM ruleachievement bigger In this way in this scenario we verifythat ASPIM gives satisfactory behavior under a more notableerror due to size where we consider a mean size ten times

03 04 05 06 07 08 09

3

35

4

45

5

E[M

OS]

ASPIMMRPF

cAS120583GM

120588

Figure 8 E[MOS] under higher error due to rate

03 04 05 06 07 08 093

35

4

45

5E

[MO

S]

ASPIMMRPF

cAS120583GM

120588

Figure 9 E[MOS] under high error due to size

lower We conclude from Figure 9 that even under a higherror caused by size that ASPIM shows good performancebeing the best alternative

55 Scenario 5 A Real Network Case This last scenarioreflects a real 4G wireless network context but with thesimplification that a single user transmits in each TTI To thatend we use CQI traces obtained from a system-level radioaccess simulator [28] considering users moving at 5 kmhin the cell For this setting with dynamic users we definechannel propagation by the Extended Pedestrian Amodel Asin current 4G networks we consider 16 channel conditionsThe state probabilities of these channel states are collectedin Table 3 which are decreasing with the improvement ofchannel quality In this scenario we consider two classes ofusers which differ in size QoE and cost The mean size

12 Abstract and Applied Analysis

03 04 05 06 07 08 09

1

2

3

4

5

E[M

OS]

120588

(a)

03 04 05 06 07 08 091

2

3

4

5

E[M

OS]

ASPIMMRPF

cAS120583GM

120588

(b)

03 04 05 06 07 08 091

2

3

4

5

E[M

OS]

ASPIMMRPF

cAS120583GM

120588

(c)

Figure 10 E[MOS] for 4G scenario aggregate (a) class 1 (b) class 2 (c)

in class 2 is ten times bigger than in class 1 and we givepriority to the small-sized class by having bigger cost andQoErequirement

Under this mixture of classes and realistic channel mod-els in Figure 10(a) we observe that scheduling strategiesshow the same tendency as in the previous scenarios It isremarkable that the difference between ASPIM and GM isbigger especially for high 120588 values Moreover under thehighest network load ASPIM achieves MOS values aboveQoE satisfaction threshold (MOS = 4) Furthermore averageMOS values forMR and PF dramatically decrease comparingwith the previous settings

Concerning class behaviour we show per-class resultsin Figures 10(b) and 10(c) As can be concluded fromthe presented results QoE-aware policies guarantee fairnessamong classes and ASPIM is the best choice also inside classAside from that for the cost dependant cAS120583 discipline thedeterioration of the class with no priority is significant

6 Conclusions

This paper goes into detail about obtaining a schedulingalgorithmaimed atmaximizing usersrsquo perception of quality inchannels with time-varying capacity Furthermore this workconsiders traffic flowswith nonexponential size distributionscontrary to previous approaches that assume memorylessdistributions

As first contribution we provide a MDP-based modelfor the scheduling problem presented This model combinesQoE-awareness size-awareness and channel-awarenessNev-ertheless since this model can not be solved either analyt-ically or computationally from the proposed MDP modelwe derive a simple tractable and implementable well-performing scheduling index rule by applying amethodologybased on Whittle approach

The proposed scheduling solution still combines QoE-size- and channel-awareness Our scheduling proposal givespriority to users in their best channel condition that are

Abstract and Applied Analysis 13

out of the service unavoidable range As verified fromsimulation results the proposed scheduling strategy showssuitable subjective quality performance in a wide range ofscenarios including the case of a simplified 4G network withheterogeneous users

Therefore the results of this work present a relevantmathematical basis for developing scheduling algorithmsaimed at maximizing QoE in current and future networkswith time-varying channel capacity This way the proposedchannel-aware ASPIM discipline will be useful for networkoperators in order to guarantee to their customers servicesatisfaction in time-varying wireless networks Apart fromthat the approximations and simplifications used in Whittlemethod are applicable to any area when the gradient ofsystem components is considerably small among consecutivedecision slots

As future research we will extend our work to amultiuserapproach considering the simultaneous transmission ofmul-tiple flows per TTI

Appendix

Proof of Proposition 4

LemmaA1 Suppose that states (119886 119889+1119898 le 119899)cup(119886+119903119899 119889+

1119898 le 119899) are passive and that states (119886 119889 + 1119898 gt 119899) cup (119886 +

119903119899 119889 + 1119898 gt 119899) are active and

Vlowast(119886119889119899)

=120573120583(119886119899)

( MOS (119889 + 1) minus 119877119886119889)

1 minus 120573120583(119886119899)

119882119886119889

(A1)

Along with this if we assume that 119877119886119889

= sum119898119902119898119877(119886119889119898)

asymp

sum119898119902119898119877(119886+119903119897 119889+1119898)

and 119882119886119889

= sum119898119902119898119882(119886119889119898)

asymp

sum119898119902119898119882(119886+119903119897 119889+1119898)

for any 119897 = 1 2 119873 value for theundiscounted case

Vlowast(119886119889119899)

= lim120573rarr1

120573120583(119886119899)

sdot MOS (119889 + 1) (1 minus 120573)1 minus 120573 + 120573sum

119898gt119899119902119898(120583(119886119898)

minus 120583(119886119899)

) (A2)

Proof Referring to work elements using (12)

119882(119886119889119898le119899)

= 120573sum

119898

1199021015840

119898119882(119886119889+1119898

1015840)= 120573119882

119886119889

119882(119886119889119898gt119899)

= 1 + 120573 (1 minus 120583(119886119898)

)sum

119898

1199021015840

119898119882(119886+119903119898 119889+1119898

1015840)

= 1 + 120573 (1 minus 120583(119886119898)

)119882119886119889

(A3)

And by (A3) expressions

119882119886119889= (1 minus sum

119898gt119899

119902119898)120573119882

119886119889+ sum

119898gt119899

119902119898(1 + 120573 (1 minus 120583

(119886119898))119882119886119889)

(A4)

This way isolating119882119886119889

119882119886119889=

sum119898gt119899

119902119898

1 minus 120573 + 120573sum119898gt119899

119902119898120583(119886119898)

(A5)

Analogously for reward elements using (11)

119877(119886119889119898le119899)

= 120573sum

119898

1199021015840

119898119877(119886119889+1119898

1015840)= 120573119877119886119889

119877(119886119889119898gt119899)

= 120573[(1 minus 120583(119886119898)

)sum

119898

1199021015840

119898119877(119886+119903119898 119898

1015840)+ 120583(119886119898)

sdot MOS (119889 + 1)]

= 120573 [(1 minus 120583(119886119898)

) 119877119886119889+ 120583(119886119898)

sdot MOS (119889 + 1)] (A6)

And thus from (A6)

119877119886119889= (1 minus sum

119898gt119899

119902119898)120573119877119886119889

+ sum

119898gt119899

119902119898(120573 [(1 minus 120583

(119886119898)) 119877119886119889+ 120583(119886119898)

sdot MOS (119889 + 1)])

(A7)

Isolating 119877119886119889

119877119886119889=120573sum119898gt119899

119902119898120583(119886119898)

MOS (119889 + 1)1 minus 120573 + 120573sum

119898gt119899119902119898120583(119886119898)

(A8)

Substituting (A5) and (A8) in (A1) and by simplifyingwe obtain

Vlowast(119886119889119899)

= (120573120583(119886119899)

(MOS (119889 + 1)

minus120573sum119898gt119899

119902119898120583(119886119898)

MOS (119889 + 1)1 minus 120573 + 120573sum

119898gt119899119902119898120583(119886119898)

))

times (1 minus 120573120583(119886119899)

sum119898gt119899

119902119898

1 minus 120573 + 120573sum119898gt119899

119902119898120583(119886119898)

)

minus1

= (120573120583(119886119899)

(MOS (119889 + 1) (1 minus 120573 + 120573sum119898gt119899

119902119898120583(119886119898)

)

minus120573sum

119898gt119899

119902119898120583(119886119898)

MOS (119889 + 1)))

times (1 minus 120573 + 120573sum

119898gt119899

119902119898120583(119886119898)

minus 120573120583(119886119899)

sum

119898gt119899

119902119898)

minus1

=120573120583(119886119899)

sdot MOS (119889 + 1) (1 minus 120573)1 minus 120573 + 120573sum

119898gt119899119902119898(120583(119886119898)

minus 120583(119886119899)

)

(A9)

And therefore for the undiscounted case the 120573 rarr 1

limit for expression (A9) is (A2)

Conflict of Interests

The authors declare that there is no conflict of interestsregarding the publication of this paper

14 Abstract and Applied Analysis

Acknowledgment

The research leading to these results has received fundingfrom the European Union Seventh Framework Programme(FP72007ndash2013) under Grant agreement 284863 (FP7 SECGERYON)

References

[1] ITU-T P10G100 Amendment 1 (0107) New Appendix I-definition of Quality of Experience (QoE) 2006

[2] S Aalto and P Lassila ldquoFlow-level stability and performance ofchannel-aware priority-based schedulersrdquo in Proceeedings of the6th Euro NF Conference on Next Generation Internet (NGI rsquo10)IEEE June 2010

[3] L KleinrockQueueing Systems vol 2 ofComputer ApplicationsJohn Wiley amp Sons 1976

[4] J Gittins ldquoBandit processes and dynamic allocation indicesrdquoJournal of the Royal Statistical Society B Methodological vol 41no 2 pp 148ndash177 1979

[5] K Avrachenkovt U Ayesta P Brown and E Nyberg ldquoDif-ferentiation between short and long TCP ows predictabilityof the response timerdquo in Proceedings of the 23rd Annual JointConference of the IEEE Computer and Communications Societies(INFOCOM rsquo04) vol 2 p 762 IEEE 2004

[6] U Ayesta M Erausquin and P Jacko ldquoA modeling frameworkfor optimizing the flow-level scheduling with time-varyingchannelsrdquo Performance Evaluation vol 67 no 11 pp 1014ndash10292010

[7] S Aalto U Ayesta and R Righter ldquoProperties of the Gittinsindex with application to optimal schedulingrdquo Probability in theEngineering and Informational Sciences vol 25 no 3 pp 269ndash288 2011

[8] M Puterman ldquoMarkov decision processesrdquo in Handbooks inOperations Research and Management Science vol 2 pp 331ndash434 1990

[9] C H Papadimitriou and J N Tsitsiklis ldquoThe complexityof Markov decision processesrdquo Mathematics of OperationsResearch vol 12 no 3 pp 441ndash450 1987

[10] J Nino-Mora ldquoDynamic priority allocation via restless banditmarginal productivity indicesrdquo TOP vol 15 no 2 pp 161ndash1982007

[11] P Whittle ldquoRestless bandits activity allocation in a changingworldrdquo Journal of Applied Probability vol 25 pp 287ndash298 1988

[12] U Ayesta M Erausquin M Jonckheere and I M VerloopldquoScheduling in a random environment stability and asymptoticoptimalityrdquo IEEEACM Transactions on Networking vol 21 no1 pp 258ndash271 2013

[13] S Khan S Thakolsri E Steinbach and W Kellerer ldquoQoE-based cross-layer optimization for wireless multiuser systemsrdquoin Proceedings of the 18th ITC Specialist Seminar on Quality ofExperience 2008

[14] S Thakolsri S Khan E Steinbach and W Kellerer ldquoQoE-driven cross-layer optimization for high speed downlink packetaccessrdquo Journal of Communications vol 4 no 9 pp 669ndash6802009

[15] P Ameigeiras J J Ramos-Munoz J Navarro-Ortiz PMogensen and J M Lopez-Soler ldquoQoE oriented cross-layer design of a resource allocation algorithm in beyond 3GsystemsrdquoComputer Communications vol 33 no 5 pp 571ndash5822010

[16] I Taboada F Liberal J O Fajardo and U Ayesta ldquoQoE-aware optimization of multimedia flow schedulingrdquo ComputerCommunications vol 36 no 15-16 pp 1629ndash1638 2013

[17] U Ayesta M Erausquin and P Jacko ldquoResource-sharing in asingle server with time-varying capacityrdquo in Proceeding of the49th Annual Allerton Conference on Communication Controland Computing (Allerton rsquo11) pp 377ndash383 Monticello Ill USASeptember 2011

[18] K Thompson G J Miller and R Wilder ldquoWide-area internettraffic patterns and characteristicsrdquo IEEE Network vol 11 no 6pp 10ndash23 1997

[19] M Fiedler T Hossfeld and P Tran-Gia ldquoA generic quantitativerelationship between quality of experience and quality ofservicerdquo IEEE Network vol 24 no 2 pp 36ndash41 2010

[20] A Richards M Antoniades V Witana and G Rogers ldquoMap-ping user level QoS from a single parameterrdquo in Proceedingsof the International Conference on Multimedia Networks andServices Citeseer 1998

[21] E Ibarrola F Liberal I Taboada and R Ortega ldquoWebQoE evaluation in multi-agent networks validation of ITU-TG1030rdquo in Proceedings of the 5th International Conference onAutonomic and Autonomous Systems (ICAS rsquo09) pp 289ndash294Valencia Spain April 2009

[22] S Aalto and U Ayesta ldquoOptimal scheduling of jobs with aDHR tail in the MG1 queuerdquo in Proceedings of the 3rd Inter-national Conference on Performance Evaluation Methodologiesand Tools (ValueTools rsquo08) article 50 Institute for ComputerSciences Social-Informatics and Telecommunications Engi-neering Athens Greece 2008

[23] V Visweswaran ldquoDecomposition techniques for MILPlagrangian relaxationrdquo in Encyclopedia of Optimization pp632ndash638 Springer New York NY USA 2009

[24] I Taboada J O Fajardo and F Liberal ldquoPerformance analysisof scheduling algorithms for web QoE optimization in wirelessnetworksrdquo Network Protocols and Algorithms vol 4 pp 27ndash432012

[25] J A van Mieghem ldquoDynamic scheduling with convex delaycosts the generalized 119888120583 rulerdquoTheAnnals of Applied Probabilityvol 5 no 3 pp 809ndash833 1995

[26] S Sesia I Toufik and M Baker LTE-The UMTS Long TermEvolution FromTheory to Practice Wiley 2011

[27] P Reichl B Tuffin and R Schatz ldquoLogarithmic laws inservice quality perception where microeconomics meets psy-chophysics and quality of experiencerdquo Telecommunication Sys-tems vol 52 no 2 pp 587ndash600 2013

[28] J C Ikuno M Wrulich and M Rupp ldquoSystem level simulationof LTE networksrdquo in Proceedings of the IEEE 71st VehicularTechnology Conference Taipei Taiwan May 2010

Submit your manuscripts athttpwwwhindawicom

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical Problems in Engineering

Hindawi Publishing Corporationhttpwwwhindawicom

Differential EquationsInternational Journal of

Volume 2014

Applied MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical PhysicsAdvances in

Complex AnalysisJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

OptimizationJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Operations ResearchAdvances in

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Function Spaces

Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of Mathematics and Mathematical Sciences

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Algebra

Discrete Dynamics in Nature and Society

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Decision SciencesAdvances in

Discrete MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Stochastic AnalysisInternational Journal of

Abstract and Applied Analysis 3

0 5 10 15 20 25 301

15

2

25

3

35

4

45

5

d (s)

MO

S

dmaxdmin

Figure 1 MOS versus delay

We illustrate an example of the explained behavior inFigure 1

213 Channel Model The channel quality associated to auser 119896 evolves randomly and independently of its evolutionhistory We assume that the channel can be in N =

1 2 119873119896 conditions Therefore user 119896 is in channel

condition 119899 with probability 119902119896119899 having sum

119899isinN 119902119896119899

= 1In channel state 119899 119903

119896119899bits are transmitted to channel and

transmission rates 119904119896119899

are 1199041198961le 1199041198962le sdot sdot sdot le 119904

119896119873119896

22 Server The objective of the scheduler is finding a policy120587 that solves the following optimization problem

max120587

1

119870

119870

sum

119896=1

MOS119896(119889119896) (2)

According to the already described aspects about the schedul-ing problem we consider a set of admissible policies Πwhich fulfils the next properties

(i) Delay dependency the time that a flow remains in thesystem 119889 is known

(ii) QoE-awareness the delay-dependent MOS functionthat characterizes user subjective quality is known

(iii) Channel-awareness channel quality indicator (CQI)information sent from mobile users is used as userrsquosinstantaneous channel condition information

(iv) Nonanticipation similarly to most current IP sys-tems flow size is unknown However existent nonan-ticipating size-based disciplines make use of flowattained service [4 7] the bits that have been trans-ferred of a flow Due to the memoryless property ofthe exponential distributions for these distributionsthe hazard rate is independent of attained serviceNevertheless the considered Pareto distributionshave a decreasing hazard rate with attained service[22] this is completion probability 120583

(119886119899) is higher

for lower attained service values This way attainedservice 119886 is used as size information

(v) Preemption we assume that the server is preemptivethat is at every decision epoch it is permitted tosuspend the service of a flow whose transmission isunfinished

(vi) Single service only the transmission of a flow isallowed in each TTI

3 MDP Formulation

In this section we formulate the problem described inSection 2 as a MDP First we provide the QoE-aware MDPmodel of each user 119896 Then we formulate the optimizationproblem for the joint MDP model which takes into accountall the users in the system

31 MDP Model of a Job The action space B of user 119896is binary action 0 means not serving and if action 1 ischosen 119903

119896119899bits from this user being in channel state 119899

are transmitted to channel Besides each user 119896 is definedby tuple (S

119896 (R119887119896119904)119887isinB

(W119887119896119904)119887isinB

(P119887119896119904)119887isinB

) These elementsare defined as follows

(i) S119896= (A119896timesD119896times 1 2 119873

119896) cup [D

119896+ 1] cup [lowast]

is the set of all 119904 states for a user 119896 which is classifiedin three groups

(a) Unfinished states all the states in which flowtransmission is uncompleted belong to thisgroup These states have three dimensionsdenoted as 119904 = (119886 119889 119899) with attained servicelevel 119886 isin A

119896 delay level 119889 isin D

119896 and

channel condition 119899 isin 1 2 119873119896 as com-

ponents The attained services that correspondto these attained service levels are multiples ofthe first not null 119903

119896119899transmission rate whereas

the delays that correspond to delay levels aremultiples of TTI

(b) Reward states these 119904 = [119889] states are reachedonce flow transmission has finished and inthese delay-dependent states the QoE-awarereward is given This way aimed at maximizingaverage MOS a delay dependant QoE-awarereward giving only happens once the wholeflow has been transmitted since from sub-jective quality perspective the MOS functiononly makes sense once flow transmission iscompleted (see [16] for more details)

(c) Final state the chain of states ends in theabsorbing state 119904 = [lowast]

(ii) R119887119896= (119877

119887

119896119904)119904isinS119896

where 119877119887119896119904

is the expected one-slotreward received for user 119896 at state 119904 if action 119887 isdecided at the beginning of a TTI

119877119887

119896(119886119889119899)= 0 119877

119887

119896[119889]= MOS (119889 + 1) 119877

119887

119896[lowast]= 0

(3)

where MOS(119889 + 1) is normalized MOS in the range[0 1]

4 Abstract and Applied Analysis

(iii) W119887119896= (119882

119887

119896119904)119904isinS119896

where119882119887119896119904

is the expected one-slotwork done for user 119896 at state 119904 if action 119887 is decided atthe beginning of a TTI

1198820

119896119904= 0 119882

1

119896119904= 1 (4)

(iv) P119887119896= (119901119887

119896(119904 1199041015840

))1199041199041015840isinS119896

where 119901119887119896(119904 1199041015840

) is the probabil-ity of user 119896 of moving from state 119904 to state 1199041015840 if action119887 is decided at the beginning of a TTI

1199010

119896((119886 119889 119899) (119886 119889 + 1119898)) = 119902

119896119898

1199011

119896((119886 119889 119899) (119886 + 119903

119899 119889 + 1119898)) = 119902

119896119898sdot (1 minus 120583

119896(119886119899))

1199011

119896((119886 119889 119899) [119889 + 1]) = 120583

119896(119886119899)

1199010

119896([119889] [lowast]) = 119901

0

119896([lowast] [lowast]) = 1

(5)

where 120583119896(119886119899)

= P(119886 lt 119883119896le 119886 + 119903

119896119899| 119883119896gt

119886) Observe that the probability of changing fromchannel state 119899 to channel state119898 corresponds to thesteady-state probability 119902

119896119898due to the memoryless

property of channel evolution

Thus the dynamics of user 119896 is captured by state process119904119896(119905) isin S

119896and action process 119887

119896(119905) isin B Moreover we

highlight that the proposedMDPmodel has restless propertywhich admits evolution and rewards even if not chosenfor transmission due to the delay increase and the channelstochastic evolution

Figure 2 shows part of the per-user state diagram of thenew QoE-aware MDP model which summarizes and relatesthe aforedescribed components

32 Optimization Problem Besides the optimization prob-lem (2) associated with such MDP can be written as

max120587isinΠ

E120587

0[

infin

sum

119905=0

sum

119896isinK

120573119905

119877119887119896(119905)

119896((119886119896(119905)119889119896(119905)119899119896(119905))cup[119889119896(119905)])]

sum

119896isinK

119887119896(119905) = 1 forall119905

(6)

Note that we consider the undiscounted reward case wherethe discounted factor 120573 is 1 being reward deterioration intime implicit in the delay-dependent MOS function

Nonetheless optimally solving problem (6) is unfeasiblein general On the one hand when channel capacity isconstant the impossibility of obtaining the exact QoE-awaresolution is revealed in [16] and hence since the time-varyingchannel model is more complex the achievement of theoptimal solution is unviable in this case On the other hand itis known that the allocation constraint causes intractability insimilar problems [11] and therefore in our case having three-dimensional states finding the optimal solution is extremelydifficult or even unachievable In this way once seeingthe impossibility of obtaining the optimal solution for theconsidered problem in this work we will focus on achievingan approximate solution For that purpose in the next sectionwe will propose a heuristic based on Whittle method

(a d + 1 1)

(a d + 1N)

q1

qN

(a d n)

120583(an)

q1(1 minus

120583 (an))

qN(1 minus 120583

(an) )

[d + 1]

b = 1

b = 0 R = 0

(a + rn d +

1N)

(a + rn d +

1 1)

lowast

R = MOS(d + 1)

Figure 2 A part of per-user state diagram

4 QoE-Aware Index Rule Proposal Based onWhittle Approach

In this part we provide aWhittle index rule type approximatesolution to problem (6) based on Whittle method Theidea of Whittle [11] consists in achieving a function thatmeasures the dynamic service priority in order to obtain asimple scheduling index rule Whittle indices are computedin isolated way for each user and they measure the expectedefficiency of serving a user in each state

Note that we are only interested in the indices of (119886 119889 119899)states as once flow transmission is completed schedulingmakes no sense Thereby the optimization problem formu-lated in (6) can be relaxed by requiring to serve a job per sloton average as proposed in [11] which is further approachedby Lagrangian methods [23] and can be decomposed into asingle-job price-based parameterized optimization problemFor a price or Lagrangiar parameter V we will therefore studythe user-119896 subproblem

max120587119896isinΠ

infin

sum

119905=0

E120587

0120573119905

sdot [119877119887119896(119905)

119896(119886119896(119905)119889119896(119905)119899119896(119905))minus V119882119887119896(119905)119896(119886119896(119905)119889119896(119905)119899119896(119905))

]

(7)

The Lagrangiar parameter V can be interpreted as the per-slotcost of serving

Let us define serving set F sube S119896 which prescribes to

serve a user 119896 if (119886 119889 119899) isin F while not to serve this user if(119886 119889 119899) notin F We will refer to states (119886 119889 119899) isin F as activeand (119886 119889 119899) notin F as passive This way it is possible to rewriteproblem (7) as

maxFsube119878119896

RF119896(119886119889119899)

minus VWF119896(119886119889119899)

(8)

whereRF(119886119889119899)

andWF(119886119889119899)

are respectively the expected totalreward and expected total work in (119886 119889 119899) state

Abstract and Applied Analysis 5

Conjecture 1 Problem (8) is indexable

Due to the complexity of our model proving the index-ability of Problem (8) is not an easy task In this way weassume Conjecture 1 and therefore we suppose that foreach (119886 119889 119899) state of user 119896 the Whittle index exists Fromnow on we omit user label 119896 In spite of considering theundiscounted case the mathematical analysis is carried outfor the discounted case so as to avoid infinite values causedby 1minus120573 terms in denominators for 120573 = 1 and then we obtainthe undiscounted index values in the limit 120573 rarr 1

Following Whittle index definition in [10] we formallywrite the Whittle index for Problem (8) Vlowast

(119886119889119899) as

Vlowast(119886119889119899)

= VF(119886119889119899)

=R⟨1F⟩

(119886119889119899)minusR⟨0F⟩

(119886119889119899)

W⟨1F⟩

(119886119889119899)minusW⟨0F⟩

(119886119889119899)

(9)

Whittle index VF(119886119889119899)

represents the rate betweenmarginal reward and marginal work where the marginalreward (work) is the difference of the expected reward earned(work required) by serving and not serving at the initial state(119886 119889 119899) and employing policyF afterwards

Lemma 2 For any state (119886 119889 119899) and under any policy F weobtain

VF(119886119889119899)

= (120573[(1 minus 120583(119886119899)

) sum

119898isinN

119902119898R

F

(119886+119903119899 119889+1119898)+ 120583(119886119899)

sdot MOS (119889 + 1) minus sum

119898isinN

119902119898R

F(119886119889+1119898)

])

times (1 + 120573[(1 minus 120583(119886119899)

) sum

119898isinN

119902119898W

F

(119886+119903119899 119889+1119898)

minus sum

119898isinN

119902119898W

F(119886119889+1119898)

])

minus1

(10)

Proof From the definition of reward and work respectivelywe have

RF(119886119889119899)

=

R⟨1F⟩

(119886119889119899)= 120573[(1 minus 120583

(119886119899)) sum

119898isinN

119902119898RF

(119886+119903119899 119889+1119898)

+ 120583(119886119899)

sdot MOS (119889 + 1) ]

(119886 119889 119899) isin F

R⟨0F⟩

(119886119889119899)= 120573 sum

119898isinN

119902119898RF(119886119889+1119898)

(119886 119889 119899) notin F

(11)

(1)F0= 0

(2) for 119894 = 1 rarr number of states do(3) (119886 119889 119899)

119894isin argmax VF119894minus1

(119886119889119899) (119886 119889 119899) notin F

119894minus1

(4) Vlowast(119886119889119899)119894

= VF119894minus1(119886119889119899)119894

F119894= F119894minus1cup (119886 119889 119899)

119894

(5) end for

Algorithm 1AG-algorithm for Problem (8)

WF(119886119889119899)

=

W⟨1F⟩

(119886119889119899)= 1 + 120573 (1 minus 120583

(119886119899)) sum

119898isinN

119902119898WF(119886+119903119899 119889+1119898)

(119886 119889 119899) isin F

W⟨0F⟩

(119886119889119899)= 120573 sum

119898isinN

119902119898WF(119886119889+1119898)

(119886 119889 119899) notin F

(12)

By substituting the expressions (11) and (12) in formula(9) we achieve a more complete VF

(119886119889119899)(10) formulation

Nevertheless in order to obtain an analytically tractableWhittle index expression for (10) in the state (119886 119889 119899) it isnecessary to determine the policyF of the future states thathave influence on the index computation of this state As canbe observed from (10) (11) and (12) expressions in thesefuture states the value of attained service is maintained orincreased whereas delay grows over time Since increasingattained service decreases instant completion probability fora Pareto distribution and increasing delay degrades QoE itwould seem intuitively correct to suppose that future stateswith the same or worse channel conditions are passive Insuch a way we could say that in a state with the same or worsechannel condition being attained service and delay higherthat the Whittle index value is lower However for futurestates with better channel conditions their passive or activeproperty is not trivial because the improvement in channelcondition does not guarantee being a better state due to theaforementioned behavior of the increasing attained serviceand delayTherefore finding the active set of all the influentialstates for Whittle index computation becomes a challengingtask

In such situation aimed at achieving the characteristicsof the optimal policy and the subsequent active set buildingevolution we will use an algorithm called Adaptive-GreedyshortlyAG-algorithm (see [10] for a survey) This algorithmcomputes Whittle indices numerically taking into accountoptimal active sets This way we have implemented AG-algorithm for our case study in order to verify the fundamen-tal properties of the Whittle index for our problem whichwill be useful to determine the active set and consequentlya closed-form index expression We provide the operation ofthe applied algorithm in Algorithm 1

We have performed several numerical experiments for120573 = 1 case in which we have reduced the state space ofboth delay and channel states so as to do the execution of thealgorithm computationally and in time viably We have onlyconsidered two channel conditions and eleven levels of delay

6 Abstract and Applied Analysis

times10minus3

6

5

4

3

2

1

00

0

5 5

10

10times107

ad

Vlowast (ad1)

(a)

0

0

5 5

10

10times107

a d

Vlowast (ad2)

001

0008

0006

0004

0002

0

(b)

Figure 3 Whittle index values obtained from AG-algorithm for two channel conditions and different attained services and delays For aPareto distribution with 120572 = 15 and E[119883] = 5Mb 119902

2= 05 119903

1= 84Kb 119903

2= 168Kb 119889min = 2 s and 119889max = 9 s

0

0

5 5

10

10

times107

ad

Vlowast (ad1)

001

0008

0006

0004

0002

0

q2 = 01

(a)

times10minus3

4

3

2

1

0

0

0

5 5

10

10

times107

a d

Vlowast (ad1)

q2 = 09

(b)

Figure 4 Whittle index values obtained fromAG-algorithm for different channel state probabilities For a Pareto distribution with 120572 = 15and E[119883] = 5Mb 119903

1= 84Kb 119903

2= 168Kb 119889min = 2 s and 119889max = 9 s

in the scenarios In such a way from experimental results weconjecture the following fundamental properties

(1) TheWhittle index value depends on attained servicedelay and channel state Moreover as can be seenin graphs from Figures 3 and 4 we can clearlydistinguish the ranges of MOS function above theupper delay threshold the index value is null

(2) For the same channel condition and the samedelay level Whittle index values are decreasing withattained service Vlowast

(119886119889119898)gt Vlowast(1198861015840119889119898)

forall1198861015840

gt 119886 (illus-trated in Figures 3 and 4)

(3) For the same channel condition and the same attainedservice levelWhittle index values are decreasing withdelay Vlowast

(119886119889119898)gt Vlowast(119886119889+1119898)

(See Figures 3 and 4)

Abstract and Applied Analysis 7

(4) In a better channel condition for the same attainedservice level and the same delay level the Whittleindex value is higher Vlowast

(1198861198891198981)gt Vlowast(1198861198891198982)

forall1198981gt 1198982

(Observe Figure 3)(5) Opposite to the previous channel-aware work [6]

where Whittle indices in the best channel conditionare infinite having absolute priority in this channelstate in our case being in the best channel conditiondoes not guarantee that the Whittle index value ishigher than in the rest of the channel conditions it ispossible to fulfil Vlowast

(119886119889119898lt119873)gt Vlowast(11988610158401198891015840119873)

If we comparethe two graphs in Figure 3 it can be easily seen howfor several attained services andor delayed values theWhittle index values in the worst channel conditionare higher than in the best channel condition (takingalso into account nonnull values)

(6) Dependency on channel condition probabilities hilefor channel condition 119873 the Whittle index valueremains invariant with channel state probability forthe rest of channel conditions its value decreases aslong as channel state probabilities increase in betterchannel conditions (shown in Figure 4)

Once we analyzed the properties of theWhittle index weset out to define the structure of the active set in order toobtain an analytically and computationally tractable Whittleindex expression In such a way to derive a closed-formcharacterization of the Whittle index of (119886 119889 119899) state first ofall it is necessary to specify the activity of (119886 119889 + 1119898) and(119886 + 119903

119899 119889 + 1119898) states As concluded from Whittle index

properties it is known that (119886 119889+1119898 le 119899)cup(119886+119903119899 119889+1119898 le

119899) states are passive However it is not trivial to specify abovewhich attained service level andor delay level if (119886 119889+1119898 gt

119899)cup (119886+ 119903119899 119889 + 1119898 gt 119899) states with better channel condition

are active or passiveNonetheless in the best channel the active set is totally

specified since all the states that compute in the index are pas-sive because of nonexisting channel improvement and beingin the future attained service equal or higher and delay higherAs shown in Proposition 3 for the best channel condition theWhittle index has a closed-form expression where the indexvalue is the multiplication between completion probabilityand normalizedMOS function Note that the obtainedWhit-tle index expression in the best channel state is equivalent tothe GM index achieved for constant channel [16]

Proposition 3 The Whittle index for Problem (8) in the bestchannel condition is given by

Vlowast(119886119889119873)

= 120583(119886119873)

sdot MOS (119889 + 1) (13)

Proof It is known that for Vlowast(119886119889119873)

index computation thatF = 0 In this way using (11) and (12) RF

(119886119889+1119898)=

120573sum119898isinN 119902

119898RF(119886119889+2119898)

RF(119886119889+2119898)

= 120573sum119898isinN 119902

119898RF(119886119889+3119898)

RF(119886119889gt119889max 119898)

= 0 Hence analogolously RF(119886+119903119899 119889+1119898)

= 0WF(119886119889+1119898)

= 0 and WF(119886+119903119899 119889+1119898)

= 0 Substituting thesenull terms in expression (10) and being 120573 = 1 we achieve(13)

Besides as stated before obtaining a computationallytractable Whittle index expression for any channel conditionis not possible In such situation in order to achieve atractable Whittle-based QoE-aware index rule for time-varying channels we propose a Whittle-based approximateheuristic in the next subsection

41 Approximate Whittle Index Proposal In the following wepropose an approximate solution for the QoE-aware Whittleindex in a time-varying channel context (10)

First of all we will determine the structure of the activeset For the Whittle index computation of (119886 119889 119899) state aspreviously concluded it is known that (119886 119889 + 1119898 le 119899) cup (119886+

119903119899 119889 + 1119898 le 119899) states are passive but we need to define the

unknown activity of (119886 119889 + 1119898 gt 119899) cup (119886 + 119903119899 119889 + 1119898 gt 119899)

states Thus so as to determine if these states are active orpassive we apply the following two considerations

(1) 120583 approximation

120583(119886119898)

asymp 120583(119886+119903119897 119898)

(14)

(2) MOS approximation

MOS (119889) asymp MOS (119889 + 1) (15)

On the one hand we assume that for any 119897 = 1 2 119873

value that 120583(119886119898)

asymp 120583(119886+119903119897 119898)

(14) It is known that (119886 119889119898 gt

119899) states are active and being 120583(119886119898)

asymp 120583(119886+119903119899 119898)

we couldconsider that (119886 + 119903

119899 119889 119898 gt 119899) states are active

On the other hand we assume MOS(119889) asymp MOS(119889 + 1)(15) since QoE degradation in a higher delay level is neg-ligible in the millisecond scale that delay evolves Takinginto account the approximation about MOS function being(119886 119889119898 gt 119899) states active we can suppose that (119886 119889 + 1119898 gt

119899) states are also active This way combining approximations(14) and (15) we consider that (119886 + 119903

119899 119889 + 1119898 gt 119899) states are

activeThen suppose that the aforementioned 120583 and

MOS approximations cause 119877119886119889

= sum119898119902119898119877(119886119889119898)

asymp

sum119898119902119898119877(119886+119903119897 119889+1119898)

and 119882119886119889

= sum119898119902119898119882(119886119889119898)

asymp

sum119898119902119898119882(119886+119903119897 119889+1119898)

So using these reward and workapproximations the first simplified expression for theWhittle index formulation (10) is

Vlowast(119886119889119899)

=120573120583(119886119899)

(MOS (119889 + 1) minus 119877119886119889)

1 minus 120573120583(119886119899)

119882119886119889

(16)

In such a way applying the previous simplificationswe obtain the Whittle index approximation presentedin Proposition 4 what we call attained service PI MOS(ASPIM) The methodology to compute the closed-formexpression of this QoE-aware Whittle-based index is pre-sented in the Appendix

Proposition 4 The formulation of the ASPIM index is

119860119878119875119868119872 = Vlowast(119886119889119899)

= lim120573rarr1

120573120583(119886119899)

sdot MOS (119889 + 1) (1 minus 120573)1 minus 120573 + 120573sum

119898gt119899119902119898(120583(119886119898)

minus 120583(119886119899)

)

(17)

8 Abstract and Applied Analysis

Table 1 CQIs and corresponding transmission rates (Mbps)

CQI 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15119904 0 42 672 84 11256 168 2184 252 2688 336 4468 504 5376 672 758 8064

If we analyze expression (17) we observe that this QoE-aware and channel-aware Whittle-based ASPIM index isimplementable This proposal depends on QoE size andchannel properties Moreover the obtained index for thebest channel condition is equal to the original Whittle index(13) since the summation in the denominator becomes nullHowever as shown in (18) when channel state is not the bestthe index value goes to zero

ASPIM119899 =119873

= lim120573rarr1

120583(119886119899)

sdot MOS (119889 + 1) (1 minus 120573)sum119898gt119899

119902119898(120583(119886119898)

minus 120583(119886119899)

)997888rarr 0

(18)

Hence a user in its best channel state with nonnullinstantaneous normalized MOS function has priority overa user which is not in its best channel condition Thus wesummarize the proposed QoE-aware Whittle-based ASPIMindex rule in Definition 5

Definition 5 The ASPIM index rule consists of at everydecision slot

(i) serving the user in its best channel condition withnonnull instantaneous normalized MOS functionwith the highest value of 120583

(119886119873)sdot MOS(119889 + 1)

(ii) if there is no user in its best channel condition withnonnull instantaneous normalized MOS functionserving the user with the highest value of (120583

(119886119899)sdot

MOS(119889 + 1))sum119898gt119899

119902119898(120583(119886119898)

minus 120583(119886119899)

) (using (18)normalized by (1 minus 120573))

So if channel condition is not the best the index valueis the ratio of the multiplication between the actual servicecompletion and the instantaneous normalizedMOS functionrespect to the expected potential improvement of the comple-tion probability

Therefore in the context of time-varying channels aimedat maximizing average MOS we have proposed ASPIMscheduling algorithm which is a simple and tractable QoE-aware channel-aware and size-based index rule Neverthe-less in order to verify the correct behavior of this heuristicwe analyze its performance in the subsequent section

5 Performance Evaluation

In this section we evaluate the performance of the proposedASPIM index rule (see proposal in Definition 5) To that endwe compare the QoE performance of our heuristic with theone achieved by well-known scheduling strategies in severalsimulation scenarios Below we provide a brief description ofthe scheduling priority rules used in our experiments

(i) Max rate (MR) this opportunistic policy consists inserving the user with the best channel capacity

(ii) Proportional fair (PF) this channel-aware disci-pline serves the user with the highest ratio betweeninstantaneous transmission rate and current attainedthroughput

(iii) Cost and attained service dependant 120583 (119888AS120583)although it might seem counter-intuitive minimizingmean delay does not result in maximizing averageMOS [24] This way in order to compare the per-formance with delay-aware solutions we propose toadapt the classical 119888120583 rule aimed at minimizing meandelay [25] to attained serviceThis adaptation consistsof serving the user with the highest value of 119888120583

(119886119899)

being 119888 the waiting cost paid for every slot while flowtransmission is uncompleted

(iv) GM [16] thisQoE-aware pioneering index rule origi-nally proposed for constant channel capacity choosesthe user with the highest value of 120583

(119886119899)sdot MOS(119889 + 1)

In case of ties we use random tie-breaking ruleConcerning traffic flows in the system we use the Pareto

size distribution given in (1) in simulations whose 120572 and 120574parameters determine mean flow size We consider typicalmean sizes used in wireless networks 05Mb (small traffica web page email) 5Mb (medium traffic a PDF documenta picture etc) and 50Mb (MP3 audio a group of picturesfrom a video sequence etc) User traffic flows arrive to thescheduler according to a Poisson process with 120582 rate

Referring to network characteristics we use transmis-sion rates that depend on CQIs used in real wirelessnetworks which are associated with 4G modulation andcoding schemes We show the mapping between CQI valuesand transmission rates in Table 1 The presented values areadapted and extrapolated from [26] for a cell of 20MHz

Moreover we consider different network loads in order toanalyze the behavior of scheduling strategies under differentnetwork conditions Flow arrival rate determines networkload 120588 where 120588 = 120582 sdot (E[119883]119904

119873) We consider seven network

states in this study low load (120588 = 025 0375) medium load(120588 = 05 075) and high load (120588 = 085 09 095)

Regarding QoE aspects as typically utilized in perceivedquality studies [27] we use a log-shaped MOS function forQoE degradation presented in (19) This way employing theMOS curve modelling provided in [21] as starting point andtaking into account the flow size the network type and theuser expertise we have suitably defined 119889min and 119889max delaythresholds used in simulations and consequently 119888

1and 1198882

MOS coefficients Consider

MOS = 1198881minus 1198882sdot log (119889) (19)

We have implemented the whole network environmentfor the scheduling of network resources in MATLAB Inrelation to the performed experiments for each scenario

Abstract and Applied Analysis 9

Table 2 Parameter set in the experimental scenarios

Scenario MOS Size Channel 119888

1 119889min = 05 s 119889max = 10 s 120572 = 15 120574 = 4 sdot 10minus7 CQI = 3 5 11198881= 407 119888

2= 133 E[119883] = 5Mbit 119902

2= 05

2 119889min = 04 s 119889max = 06 s 120572 = 15 120574 = 4 sdot 10minus7 CQI = 3 5 11198881= minus403 119888

2= 986 E[119883] = 5Mbit 119902

2= 05

3 119889min = 05 s 119889max = 10 s 120572 = 15 120574 = 4 sdot 10minus7 CQI = 3 9 11198881= 407 119888

2= 133 E[119883] = 5Mbit 119902

2= 05

4 119889min = 005 s 119889max = 1 s 120572 = 15 120574 = 4 sdot 10minus6 CQI = 3 5 11198881= 1 1198882= 133 E[119883] = 05Mbit 119902

2= 05

5

119889min1 = 05 s 119889max1 = 5 s 1205721= 15 120574

1= 4 sdot 10

minus7

see Table 3 5 111988811= 379 119888

21= 173 E[119883

1] = 5Mbit

119889min2 = 5 s 119889max2 = 30 s 1205722= 15 120574

2= 4 sdot 10

minus8

11988812= 859 119888

22= 223 E[119883

2] = 50Mbit

Table 3 Channel state probabilities in Scenario 5

CQI 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15119902 028 012 009 008 008 008 007 006 005 004 003 001 0009 00005 00003 00002

for each combination of scheduling discipline and networkload we have carried out a set of 10 rounds of 10000 slength simulations These rounds differ in the randomlypregenerated traces (input vector) of sizes and arrivals Thusnot only the overall average MOS values will be provided butwe will also include their 95 confidence intervals

Besides in order to guarantee that the obtained perfor-mance results are generally valid we will analyze differentsimulation scenarios We have chosen five relevant settingswhich differ inQoE size channel orand cost characteristicsThe parameters of these scenarios are summarized in Table 2Note that in the first four scenarios we restrict ourselves tothe case of two channel conditions so as to get fundamentalperformance results

We have carefully selected the scenarios used in simula-tions First of all we define a setting (Scenario 1) in which thevalues of all the parameters are typical or standard the valuesof these parameters introduce low error in the approxima-tions considered for our ASPIM scheduler proposal Thenin order to evaluate worst performance cases for ASPIMwe have chosen three scenarios (Scenario 2 Scenario 3 andScenario 4) in which the error of the approximation used forASPIM is high due to a different factor (gradient ofMOS rateand size resp)

Apart from that it is worthmentioning that for obtainingthe ASPIM index rule that we have considered an isolatedMDP model per user Hence the achieved solution is basedon a unique user type or single class However it would beinteresting to examine the proposal when there are differenttypes of users which is analyzed in Scenario 5 Furthermorethis last setting resembles a real 4G wireless network Notethat even though this technology allows the simultaneoustransmission of multiple flows per TTI we assume that asingle flow transmits in each TTI

03 04 05 06 07 08 09

25

3

35

4

45

5

E[M

OS]

ASPIMMRPF

cAS120583GM

120588

Figure 5 E[MOS] for the basic scenario

Next we describe the results we have achieved in theaforementioned scenarios

51 Scenario 1 Basic Case In this first family of simulationswe consider a basic or typical scenario which takes intoaccount the equiprobable channel case and medium-sizedself-similar flows Figure 5 shows averageMOS results for thissetting employing different scheduling policies and networkloadsThemain conclusion achieved from these results is thatASPIM outperforms the rest of disciplines which is betterthan the QoE-aware GM rule proposed for constant channelcapacity Moreover contrary to the other policies it holds

10 Abstract and Applied Analysis

0 02 04 06 08 10

1

2

3

4

d (s)

PDF(d

)

(a)

1 2 3 4 50

01

02

03

04

05

06

07

MOS

PDF(

MO

S)

ASPIMMRPF

cAS120583GM

(b)

1 2 3 4 50

02

04

06

08

1

MOS

CCD

F(M

OS)

ASPIMMRPF

cAS120583GM

Satisth

(c)

Figure 6 Statistics in Scenario 1 delay PDF (a) MOS PDF (b) and MOS CCDF (c)

averageMOSvalues in the same level for all120588 Referring to thetraditional channel-aware disciplines both PF and MR givebad behavior comparing to the rest of scheduling strategies

In order to justify the previous performance results wenow analyze delay andMOS statistics for the highest networkload considered In such a way as regarded in Figure 6(a) forthe Probability Density Function (PDF) of delay we highlightthe probabilitymass or peak that appears for some schedulingdisciplines before the low delay threshold (05 s) The heightand weight of a peak belonging to a policy have direct impacton its average MOS ASPIM discipline shows the highestand the widest peak among the presented ones followedby GM but with a notable difference Therefore our novelQoE-aware policy seems to try to finish the transmissionof most of the flows before 119889min which is reflected in theaforementioned peaks Note that the cAS120583 rule aimed atminimizing delay behaves in different manner which showsmore fairness among delays in the range shown compared tothe QoE-aware ones

What is previously concluded about delay statistics hasa direct reflection on QoE statistics This way as can beseen in the PDF of MOS presented in Figure 6(b) if theobjective is to maximize average QoE it is important onthe one hand to be the height and the weight of the peaksaround the maximum MOS high and on the other hand tobe the value around the minimum MOS low ASPIM fulfilsthese positive properties and consequently it is the bestin average QoE terms Besides if we analyze the Comple-mentary Cumulative Distribution Function (CCDF) of MOSprovided in Figure 6(c) we observe that ASPIM gives thehighest probability of being MOS value higher Consideringfor example a QoE satisfaction threshold satisth of MOS =4 in this point ASPIM improves GM a 10 whereas thedeterioration of MR and PF is higher than a 50

52 Scenario 2 High Demanding Users Case This scenarioreflects the case of users with low tolerance to delay in

Abstract and Applied Analysis 11

03 04 05 06 07 08 09

05

06

07

08

09

ASPIMMRPF

cAS120583GM

120588

E[M

OS]

2E

[MO

S]1

Figure 7 E[MOS] for abrupt MOS scenario

which theMOS function in theQoE degradation range showsa great slope Thus in this setting the error of the MOSapproximation (15) is high since the gradient of MOS amongconsecutive delay levels becomes significant In Figure 7we compare the results of this second scenario and thebasic scenario From the illustrated graph we could say thatour QoE-aware proposal adapts to any MOS function Thisis whereas for the rest of disciplines (excluding GM) thenegative effects due to the abrupt degradation of the QoEutility function are appreciable ASPIMmanages to avoid thisadapting to any MOS function

53 Scenario 3 Higher Error Case due to Rate During thesimplification process carried out for obtaining the ASPIMindex rule in Section 41 the 120583 approximation used (see (14))introduces a rate dependant error Note that increasing therate in the good channel state makes bigger the error of the 120583approximation used Therefore we will analyze the obtainedperformance for our new QoE-aware proposal under thishigher error due to rate In the previous scenarios we haveassumed that the transmission rate in the good channelcondition is twice the rate in the bad channel state In thissetting we consider that the transmission rate in the goodchannel is four times the transmission rate in the bad channelThis way according to the QoE results presented in Figure 8in spite of a higher error caused by rate ASPIM is superior tothe rest of policies

54 Scenario 4 High Error Case due to Size Besides it isknown that for a distribution with a decreasing hazard ratesuch as Pareto that the gradient of completion probabilityincreases as long as the mean size decreases which makesthe error of the 120583 approximation (14) for the ASPIM ruleachievement bigger In this way in this scenario we verifythat ASPIM gives satisfactory behavior under a more notableerror due to size where we consider a mean size ten times

03 04 05 06 07 08 09

3

35

4

45

5

E[M

OS]

ASPIMMRPF

cAS120583GM

120588

Figure 8 E[MOS] under higher error due to rate

03 04 05 06 07 08 093

35

4

45

5E

[MO

S]

ASPIMMRPF

cAS120583GM

120588

Figure 9 E[MOS] under high error due to size

lower We conclude from Figure 9 that even under a higherror caused by size that ASPIM shows good performancebeing the best alternative

55 Scenario 5 A Real Network Case This last scenarioreflects a real 4G wireless network context but with thesimplification that a single user transmits in each TTI To thatend we use CQI traces obtained from a system-level radioaccess simulator [28] considering users moving at 5 kmhin the cell For this setting with dynamic users we definechannel propagation by the Extended Pedestrian Amodel Asin current 4G networks we consider 16 channel conditionsThe state probabilities of these channel states are collectedin Table 3 which are decreasing with the improvement ofchannel quality In this scenario we consider two classes ofusers which differ in size QoE and cost The mean size

12 Abstract and Applied Analysis

03 04 05 06 07 08 09

1

2

3

4

5

E[M

OS]

120588

(a)

03 04 05 06 07 08 091

2

3

4

5

E[M

OS]

ASPIMMRPF

cAS120583GM

120588

(b)

03 04 05 06 07 08 091

2

3

4

5

E[M

OS]

ASPIMMRPF

cAS120583GM

120588

(c)

Figure 10 E[MOS] for 4G scenario aggregate (a) class 1 (b) class 2 (c)

in class 2 is ten times bigger than in class 1 and we givepriority to the small-sized class by having bigger cost andQoErequirement

Under this mixture of classes and realistic channel mod-els in Figure 10(a) we observe that scheduling strategiesshow the same tendency as in the previous scenarios It isremarkable that the difference between ASPIM and GM isbigger especially for high 120588 values Moreover under thehighest network load ASPIM achieves MOS values aboveQoE satisfaction threshold (MOS = 4) Furthermore averageMOS values forMR and PF dramatically decrease comparingwith the previous settings

Concerning class behaviour we show per-class resultsin Figures 10(b) and 10(c) As can be concluded fromthe presented results QoE-aware policies guarantee fairnessamong classes and ASPIM is the best choice also inside classAside from that for the cost dependant cAS120583 discipline thedeterioration of the class with no priority is significant

6 Conclusions

This paper goes into detail about obtaining a schedulingalgorithmaimed atmaximizing usersrsquo perception of quality inchannels with time-varying capacity Furthermore this workconsiders traffic flowswith nonexponential size distributionscontrary to previous approaches that assume memorylessdistributions

As first contribution we provide a MDP-based modelfor the scheduling problem presented This model combinesQoE-awareness size-awareness and channel-awarenessNev-ertheless since this model can not be solved either analyt-ically or computationally from the proposed MDP modelwe derive a simple tractable and implementable well-performing scheduling index rule by applying amethodologybased on Whittle approach

The proposed scheduling solution still combines QoE-size- and channel-awareness Our scheduling proposal givespriority to users in their best channel condition that are

Abstract and Applied Analysis 13

out of the service unavoidable range As verified fromsimulation results the proposed scheduling strategy showssuitable subjective quality performance in a wide range ofscenarios including the case of a simplified 4G network withheterogeneous users

Therefore the results of this work present a relevantmathematical basis for developing scheduling algorithmsaimed at maximizing QoE in current and future networkswith time-varying channel capacity This way the proposedchannel-aware ASPIM discipline will be useful for networkoperators in order to guarantee to their customers servicesatisfaction in time-varying wireless networks Apart fromthat the approximations and simplifications used in Whittlemethod are applicable to any area when the gradient ofsystem components is considerably small among consecutivedecision slots

As future research we will extend our work to amultiuserapproach considering the simultaneous transmission ofmul-tiple flows per TTI

Appendix

Proof of Proposition 4

LemmaA1 Suppose that states (119886 119889+1119898 le 119899)cup(119886+119903119899 119889+

1119898 le 119899) are passive and that states (119886 119889 + 1119898 gt 119899) cup (119886 +

119903119899 119889 + 1119898 gt 119899) are active and

Vlowast(119886119889119899)

=120573120583(119886119899)

( MOS (119889 + 1) minus 119877119886119889)

1 minus 120573120583(119886119899)

119882119886119889

(A1)

Along with this if we assume that 119877119886119889

= sum119898119902119898119877(119886119889119898)

asymp

sum119898119902119898119877(119886+119903119897 119889+1119898)

and 119882119886119889

= sum119898119902119898119882(119886119889119898)

asymp

sum119898119902119898119882(119886+119903119897 119889+1119898)

for any 119897 = 1 2 119873 value for theundiscounted case

Vlowast(119886119889119899)

= lim120573rarr1

120573120583(119886119899)

sdot MOS (119889 + 1) (1 minus 120573)1 minus 120573 + 120573sum

119898gt119899119902119898(120583(119886119898)

minus 120583(119886119899)

) (A2)

Proof Referring to work elements using (12)

119882(119886119889119898le119899)

= 120573sum

119898

1199021015840

119898119882(119886119889+1119898

1015840)= 120573119882

119886119889

119882(119886119889119898gt119899)

= 1 + 120573 (1 minus 120583(119886119898)

)sum

119898

1199021015840

119898119882(119886+119903119898 119889+1119898

1015840)

= 1 + 120573 (1 minus 120583(119886119898)

)119882119886119889

(A3)

And by (A3) expressions

119882119886119889= (1 minus sum

119898gt119899

119902119898)120573119882

119886119889+ sum

119898gt119899

119902119898(1 + 120573 (1 minus 120583

(119886119898))119882119886119889)

(A4)

This way isolating119882119886119889

119882119886119889=

sum119898gt119899

119902119898

1 minus 120573 + 120573sum119898gt119899

119902119898120583(119886119898)

(A5)

Analogously for reward elements using (11)

119877(119886119889119898le119899)

= 120573sum

119898

1199021015840

119898119877(119886119889+1119898

1015840)= 120573119877119886119889

119877(119886119889119898gt119899)

= 120573[(1 minus 120583(119886119898)

)sum

119898

1199021015840

119898119877(119886+119903119898 119898

1015840)+ 120583(119886119898)

sdot MOS (119889 + 1)]

= 120573 [(1 minus 120583(119886119898)

) 119877119886119889+ 120583(119886119898)

sdot MOS (119889 + 1)] (A6)

And thus from (A6)

119877119886119889= (1 minus sum

119898gt119899

119902119898)120573119877119886119889

+ sum

119898gt119899

119902119898(120573 [(1 minus 120583

(119886119898)) 119877119886119889+ 120583(119886119898)

sdot MOS (119889 + 1)])

(A7)

Isolating 119877119886119889

119877119886119889=120573sum119898gt119899

119902119898120583(119886119898)

MOS (119889 + 1)1 minus 120573 + 120573sum

119898gt119899119902119898120583(119886119898)

(A8)

Substituting (A5) and (A8) in (A1) and by simplifyingwe obtain

Vlowast(119886119889119899)

= (120573120583(119886119899)

(MOS (119889 + 1)

minus120573sum119898gt119899

119902119898120583(119886119898)

MOS (119889 + 1)1 minus 120573 + 120573sum

119898gt119899119902119898120583(119886119898)

))

times (1 minus 120573120583(119886119899)

sum119898gt119899

119902119898

1 minus 120573 + 120573sum119898gt119899

119902119898120583(119886119898)

)

minus1

= (120573120583(119886119899)

(MOS (119889 + 1) (1 minus 120573 + 120573sum119898gt119899

119902119898120583(119886119898)

)

minus120573sum

119898gt119899

119902119898120583(119886119898)

MOS (119889 + 1)))

times (1 minus 120573 + 120573sum

119898gt119899

119902119898120583(119886119898)

minus 120573120583(119886119899)

sum

119898gt119899

119902119898)

minus1

=120573120583(119886119899)

sdot MOS (119889 + 1) (1 minus 120573)1 minus 120573 + 120573sum

119898gt119899119902119898(120583(119886119898)

minus 120583(119886119899)

)

(A9)

And therefore for the undiscounted case the 120573 rarr 1

limit for expression (A9) is (A2)

Conflict of Interests

The authors declare that there is no conflict of interestsregarding the publication of this paper

14 Abstract and Applied Analysis

Acknowledgment

The research leading to these results has received fundingfrom the European Union Seventh Framework Programme(FP72007ndash2013) under Grant agreement 284863 (FP7 SECGERYON)

References

[1] ITU-T P10G100 Amendment 1 (0107) New Appendix I-definition of Quality of Experience (QoE) 2006

[2] S Aalto and P Lassila ldquoFlow-level stability and performance ofchannel-aware priority-based schedulersrdquo in Proceeedings of the6th Euro NF Conference on Next Generation Internet (NGI rsquo10)IEEE June 2010

[3] L KleinrockQueueing Systems vol 2 ofComputer ApplicationsJohn Wiley amp Sons 1976

[4] J Gittins ldquoBandit processes and dynamic allocation indicesrdquoJournal of the Royal Statistical Society B Methodological vol 41no 2 pp 148ndash177 1979

[5] K Avrachenkovt U Ayesta P Brown and E Nyberg ldquoDif-ferentiation between short and long TCP ows predictabilityof the response timerdquo in Proceedings of the 23rd Annual JointConference of the IEEE Computer and Communications Societies(INFOCOM rsquo04) vol 2 p 762 IEEE 2004

[6] U Ayesta M Erausquin and P Jacko ldquoA modeling frameworkfor optimizing the flow-level scheduling with time-varyingchannelsrdquo Performance Evaluation vol 67 no 11 pp 1014ndash10292010

[7] S Aalto U Ayesta and R Righter ldquoProperties of the Gittinsindex with application to optimal schedulingrdquo Probability in theEngineering and Informational Sciences vol 25 no 3 pp 269ndash288 2011

[8] M Puterman ldquoMarkov decision processesrdquo in Handbooks inOperations Research and Management Science vol 2 pp 331ndash434 1990

[9] C H Papadimitriou and J N Tsitsiklis ldquoThe complexityof Markov decision processesrdquo Mathematics of OperationsResearch vol 12 no 3 pp 441ndash450 1987

[10] J Nino-Mora ldquoDynamic priority allocation via restless banditmarginal productivity indicesrdquo TOP vol 15 no 2 pp 161ndash1982007

[11] P Whittle ldquoRestless bandits activity allocation in a changingworldrdquo Journal of Applied Probability vol 25 pp 287ndash298 1988

[12] U Ayesta M Erausquin M Jonckheere and I M VerloopldquoScheduling in a random environment stability and asymptoticoptimalityrdquo IEEEACM Transactions on Networking vol 21 no1 pp 258ndash271 2013

[13] S Khan S Thakolsri E Steinbach and W Kellerer ldquoQoE-based cross-layer optimization for wireless multiuser systemsrdquoin Proceedings of the 18th ITC Specialist Seminar on Quality ofExperience 2008

[14] S Thakolsri S Khan E Steinbach and W Kellerer ldquoQoE-driven cross-layer optimization for high speed downlink packetaccessrdquo Journal of Communications vol 4 no 9 pp 669ndash6802009

[15] P Ameigeiras J J Ramos-Munoz J Navarro-Ortiz PMogensen and J M Lopez-Soler ldquoQoE oriented cross-layer design of a resource allocation algorithm in beyond 3GsystemsrdquoComputer Communications vol 33 no 5 pp 571ndash5822010

[16] I Taboada F Liberal J O Fajardo and U Ayesta ldquoQoE-aware optimization of multimedia flow schedulingrdquo ComputerCommunications vol 36 no 15-16 pp 1629ndash1638 2013

[17] U Ayesta M Erausquin and P Jacko ldquoResource-sharing in asingle server with time-varying capacityrdquo in Proceeding of the49th Annual Allerton Conference on Communication Controland Computing (Allerton rsquo11) pp 377ndash383 Monticello Ill USASeptember 2011

[18] K Thompson G J Miller and R Wilder ldquoWide-area internettraffic patterns and characteristicsrdquo IEEE Network vol 11 no 6pp 10ndash23 1997

[19] M Fiedler T Hossfeld and P Tran-Gia ldquoA generic quantitativerelationship between quality of experience and quality ofservicerdquo IEEE Network vol 24 no 2 pp 36ndash41 2010

[20] A Richards M Antoniades V Witana and G Rogers ldquoMap-ping user level QoS from a single parameterrdquo in Proceedingsof the International Conference on Multimedia Networks andServices Citeseer 1998

[21] E Ibarrola F Liberal I Taboada and R Ortega ldquoWebQoE evaluation in multi-agent networks validation of ITU-TG1030rdquo in Proceedings of the 5th International Conference onAutonomic and Autonomous Systems (ICAS rsquo09) pp 289ndash294Valencia Spain April 2009

[22] S Aalto and U Ayesta ldquoOptimal scheduling of jobs with aDHR tail in the MG1 queuerdquo in Proceedings of the 3rd Inter-national Conference on Performance Evaluation Methodologiesand Tools (ValueTools rsquo08) article 50 Institute for ComputerSciences Social-Informatics and Telecommunications Engi-neering Athens Greece 2008

[23] V Visweswaran ldquoDecomposition techniques for MILPlagrangian relaxationrdquo in Encyclopedia of Optimization pp632ndash638 Springer New York NY USA 2009

[24] I Taboada J O Fajardo and F Liberal ldquoPerformance analysisof scheduling algorithms for web QoE optimization in wirelessnetworksrdquo Network Protocols and Algorithms vol 4 pp 27ndash432012

[25] J A van Mieghem ldquoDynamic scheduling with convex delaycosts the generalized 119888120583 rulerdquoTheAnnals of Applied Probabilityvol 5 no 3 pp 809ndash833 1995

[26] S Sesia I Toufik and M Baker LTE-The UMTS Long TermEvolution FromTheory to Practice Wiley 2011

[27] P Reichl B Tuffin and R Schatz ldquoLogarithmic laws inservice quality perception where microeconomics meets psy-chophysics and quality of experiencerdquo Telecommunication Sys-tems vol 52 no 2 pp 587ndash600 2013

[28] J C Ikuno M Wrulich and M Rupp ldquoSystem level simulationof LTE networksrdquo in Proceedings of the IEEE 71st VehicularTechnology Conference Taipei Taiwan May 2010

Submit your manuscripts athttpwwwhindawicom

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical Problems in Engineering

Hindawi Publishing Corporationhttpwwwhindawicom

Differential EquationsInternational Journal of

Volume 2014

Applied MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical PhysicsAdvances in

Complex AnalysisJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

OptimizationJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Operations ResearchAdvances in

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Function Spaces

Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of Mathematics and Mathematical Sciences

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Algebra

Discrete Dynamics in Nature and Society

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Decision SciencesAdvances in

Discrete MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Stochastic AnalysisInternational Journal of

4 Abstract and Applied Analysis

(iii) W119887119896= (119882

119887

119896119904)119904isinS119896

where119882119887119896119904

is the expected one-slotwork done for user 119896 at state 119904 if action 119887 is decided atthe beginning of a TTI

1198820

119896119904= 0 119882

1

119896119904= 1 (4)

(iv) P119887119896= (119901119887

119896(119904 1199041015840

))1199041199041015840isinS119896

where 119901119887119896(119904 1199041015840

) is the probabil-ity of user 119896 of moving from state 119904 to state 1199041015840 if action119887 is decided at the beginning of a TTI

1199010

119896((119886 119889 119899) (119886 119889 + 1119898)) = 119902

119896119898

1199011

119896((119886 119889 119899) (119886 + 119903

119899 119889 + 1119898)) = 119902

119896119898sdot (1 minus 120583

119896(119886119899))

1199011

119896((119886 119889 119899) [119889 + 1]) = 120583

119896(119886119899)

1199010

119896([119889] [lowast]) = 119901

0

119896([lowast] [lowast]) = 1

(5)

where 120583119896(119886119899)

= P(119886 lt 119883119896le 119886 + 119903

119896119899| 119883119896gt

119886) Observe that the probability of changing fromchannel state 119899 to channel state119898 corresponds to thesteady-state probability 119902

119896119898due to the memoryless

property of channel evolution

Thus the dynamics of user 119896 is captured by state process119904119896(119905) isin S

119896and action process 119887

119896(119905) isin B Moreover we

highlight that the proposedMDPmodel has restless propertywhich admits evolution and rewards even if not chosenfor transmission due to the delay increase and the channelstochastic evolution

Figure 2 shows part of the per-user state diagram of thenew QoE-aware MDP model which summarizes and relatesthe aforedescribed components

32 Optimization Problem Besides the optimization prob-lem (2) associated with such MDP can be written as

max120587isinΠ

E120587

0[

infin

sum

119905=0

sum

119896isinK

120573119905

119877119887119896(119905)

119896((119886119896(119905)119889119896(119905)119899119896(119905))cup[119889119896(119905)])]

sum

119896isinK

119887119896(119905) = 1 forall119905

(6)

Note that we consider the undiscounted reward case wherethe discounted factor 120573 is 1 being reward deterioration intime implicit in the delay-dependent MOS function

Nonetheless optimally solving problem (6) is unfeasiblein general On the one hand when channel capacity isconstant the impossibility of obtaining the exact QoE-awaresolution is revealed in [16] and hence since the time-varyingchannel model is more complex the achievement of theoptimal solution is unviable in this case On the other hand itis known that the allocation constraint causes intractability insimilar problems [11] and therefore in our case having three-dimensional states finding the optimal solution is extremelydifficult or even unachievable In this way once seeingthe impossibility of obtaining the optimal solution for theconsidered problem in this work we will focus on achievingan approximate solution For that purpose in the next sectionwe will propose a heuristic based on Whittle method

(a d + 1 1)

(a d + 1N)

q1

qN

(a d n)

120583(an)

q1(1 minus

120583 (an))

qN(1 minus 120583

(an) )

[d + 1]

b = 1

b = 0 R = 0

(a + rn d +

1N)

(a + rn d +

1 1)

lowast

R = MOS(d + 1)

Figure 2 A part of per-user state diagram

4 QoE-Aware Index Rule Proposal Based onWhittle Approach

In this part we provide aWhittle index rule type approximatesolution to problem (6) based on Whittle method Theidea of Whittle [11] consists in achieving a function thatmeasures the dynamic service priority in order to obtain asimple scheduling index rule Whittle indices are computedin isolated way for each user and they measure the expectedefficiency of serving a user in each state

Note that we are only interested in the indices of (119886 119889 119899)states as once flow transmission is completed schedulingmakes no sense Thereby the optimization problem formu-lated in (6) can be relaxed by requiring to serve a job per sloton average as proposed in [11] which is further approachedby Lagrangian methods [23] and can be decomposed into asingle-job price-based parameterized optimization problemFor a price or Lagrangiar parameter V we will therefore studythe user-119896 subproblem

max120587119896isinΠ

infin

sum

119905=0

E120587

0120573119905

sdot [119877119887119896(119905)

119896(119886119896(119905)119889119896(119905)119899119896(119905))minus V119882119887119896(119905)119896(119886119896(119905)119889119896(119905)119899119896(119905))

]

(7)

The Lagrangiar parameter V can be interpreted as the per-slotcost of serving

Let us define serving set F sube S119896 which prescribes to

serve a user 119896 if (119886 119889 119899) isin F while not to serve this user if(119886 119889 119899) notin F We will refer to states (119886 119889 119899) isin F as activeand (119886 119889 119899) notin F as passive This way it is possible to rewriteproblem (7) as

maxFsube119878119896

RF119896(119886119889119899)

minus VWF119896(119886119889119899)

(8)

whereRF(119886119889119899)

andWF(119886119889119899)

are respectively the expected totalreward and expected total work in (119886 119889 119899) state

Abstract and Applied Analysis 5

Conjecture 1 Problem (8) is indexable

Due to the complexity of our model proving the index-ability of Problem (8) is not an easy task In this way weassume Conjecture 1 and therefore we suppose that foreach (119886 119889 119899) state of user 119896 the Whittle index exists Fromnow on we omit user label 119896 In spite of considering theundiscounted case the mathematical analysis is carried outfor the discounted case so as to avoid infinite values causedby 1minus120573 terms in denominators for 120573 = 1 and then we obtainthe undiscounted index values in the limit 120573 rarr 1

Following Whittle index definition in [10] we formallywrite the Whittle index for Problem (8) Vlowast

(119886119889119899) as

Vlowast(119886119889119899)

= VF(119886119889119899)

=R⟨1F⟩

(119886119889119899)minusR⟨0F⟩

(119886119889119899)

W⟨1F⟩

(119886119889119899)minusW⟨0F⟩

(119886119889119899)

(9)

Whittle index VF(119886119889119899)

represents the rate betweenmarginal reward and marginal work where the marginalreward (work) is the difference of the expected reward earned(work required) by serving and not serving at the initial state(119886 119889 119899) and employing policyF afterwards

Lemma 2 For any state (119886 119889 119899) and under any policy F weobtain

VF(119886119889119899)

= (120573[(1 minus 120583(119886119899)

) sum

119898isinN

119902119898R

F

(119886+119903119899 119889+1119898)+ 120583(119886119899)

sdot MOS (119889 + 1) minus sum

119898isinN

119902119898R

F(119886119889+1119898)

])

times (1 + 120573[(1 minus 120583(119886119899)

) sum

119898isinN

119902119898W

F

(119886+119903119899 119889+1119898)

minus sum

119898isinN

119902119898W

F(119886119889+1119898)

])

minus1

(10)

Proof From the definition of reward and work respectivelywe have

RF(119886119889119899)

=

R⟨1F⟩

(119886119889119899)= 120573[(1 minus 120583

(119886119899)) sum

119898isinN

119902119898RF

(119886+119903119899 119889+1119898)

+ 120583(119886119899)

sdot MOS (119889 + 1) ]

(119886 119889 119899) isin F

R⟨0F⟩

(119886119889119899)= 120573 sum

119898isinN

119902119898RF(119886119889+1119898)

(119886 119889 119899) notin F

(11)

(1)F0= 0

(2) for 119894 = 1 rarr number of states do(3) (119886 119889 119899)

119894isin argmax VF119894minus1

(119886119889119899) (119886 119889 119899) notin F

119894minus1

(4) Vlowast(119886119889119899)119894

= VF119894minus1(119886119889119899)119894

F119894= F119894minus1cup (119886 119889 119899)

119894

(5) end for

Algorithm 1AG-algorithm for Problem (8)

WF(119886119889119899)

=

W⟨1F⟩

(119886119889119899)= 1 + 120573 (1 minus 120583

(119886119899)) sum

119898isinN

119902119898WF(119886+119903119899 119889+1119898)

(119886 119889 119899) isin F

W⟨0F⟩

(119886119889119899)= 120573 sum

119898isinN

119902119898WF(119886119889+1119898)

(119886 119889 119899) notin F

(12)

By substituting the expressions (11) and (12) in formula(9) we achieve a more complete VF

(119886119889119899)(10) formulation

Nevertheless in order to obtain an analytically tractableWhittle index expression for (10) in the state (119886 119889 119899) it isnecessary to determine the policyF of the future states thathave influence on the index computation of this state As canbe observed from (10) (11) and (12) expressions in thesefuture states the value of attained service is maintained orincreased whereas delay grows over time Since increasingattained service decreases instant completion probability fora Pareto distribution and increasing delay degrades QoE itwould seem intuitively correct to suppose that future stateswith the same or worse channel conditions are passive Insuch a way we could say that in a state with the same or worsechannel condition being attained service and delay higherthat the Whittle index value is lower However for futurestates with better channel conditions their passive or activeproperty is not trivial because the improvement in channelcondition does not guarantee being a better state due to theaforementioned behavior of the increasing attained serviceand delayTherefore finding the active set of all the influentialstates for Whittle index computation becomes a challengingtask

In such situation aimed at achieving the characteristicsof the optimal policy and the subsequent active set buildingevolution we will use an algorithm called Adaptive-GreedyshortlyAG-algorithm (see [10] for a survey) This algorithmcomputes Whittle indices numerically taking into accountoptimal active sets This way we have implemented AG-algorithm for our case study in order to verify the fundamen-tal properties of the Whittle index for our problem whichwill be useful to determine the active set and consequentlya closed-form index expression We provide the operation ofthe applied algorithm in Algorithm 1

We have performed several numerical experiments for120573 = 1 case in which we have reduced the state space ofboth delay and channel states so as to do the execution of thealgorithm computationally and in time viably We have onlyconsidered two channel conditions and eleven levels of delay

6 Abstract and Applied Analysis

times10minus3

6

5

4

3

2

1

00

0

5 5

10

10times107

ad

Vlowast (ad1)

(a)

0

0

5 5

10

10times107

a d

Vlowast (ad2)

001

0008

0006

0004

0002

0

(b)

Figure 3 Whittle index values obtained from AG-algorithm for two channel conditions and different attained services and delays For aPareto distribution with 120572 = 15 and E[119883] = 5Mb 119902

2= 05 119903

1= 84Kb 119903

2= 168Kb 119889min = 2 s and 119889max = 9 s

0

0

5 5

10

10

times107

ad

Vlowast (ad1)

001

0008

0006

0004

0002

0

q2 = 01

(a)

times10minus3

4

3

2

1

0

0

0

5 5

10

10

times107

a d

Vlowast (ad1)

q2 = 09

(b)

Figure 4 Whittle index values obtained fromAG-algorithm for different channel state probabilities For a Pareto distribution with 120572 = 15and E[119883] = 5Mb 119903

1= 84Kb 119903

2= 168Kb 119889min = 2 s and 119889max = 9 s

in the scenarios In such a way from experimental results weconjecture the following fundamental properties

(1) TheWhittle index value depends on attained servicedelay and channel state Moreover as can be seenin graphs from Figures 3 and 4 we can clearlydistinguish the ranges of MOS function above theupper delay threshold the index value is null

(2) For the same channel condition and the samedelay level Whittle index values are decreasing withattained service Vlowast

(119886119889119898)gt Vlowast(1198861015840119889119898)

forall1198861015840

gt 119886 (illus-trated in Figures 3 and 4)

(3) For the same channel condition and the same attainedservice levelWhittle index values are decreasing withdelay Vlowast

(119886119889119898)gt Vlowast(119886119889+1119898)

(See Figures 3 and 4)

Abstract and Applied Analysis 7

(4) In a better channel condition for the same attainedservice level and the same delay level the Whittleindex value is higher Vlowast

(1198861198891198981)gt Vlowast(1198861198891198982)

forall1198981gt 1198982

(Observe Figure 3)(5) Opposite to the previous channel-aware work [6]

where Whittle indices in the best channel conditionare infinite having absolute priority in this channelstate in our case being in the best channel conditiondoes not guarantee that the Whittle index value ishigher than in the rest of the channel conditions it ispossible to fulfil Vlowast

(119886119889119898lt119873)gt Vlowast(11988610158401198891015840119873)

If we comparethe two graphs in Figure 3 it can be easily seen howfor several attained services andor delayed values theWhittle index values in the worst channel conditionare higher than in the best channel condition (takingalso into account nonnull values)

(6) Dependency on channel condition probabilities hilefor channel condition 119873 the Whittle index valueremains invariant with channel state probability forthe rest of channel conditions its value decreases aslong as channel state probabilities increase in betterchannel conditions (shown in Figure 4)

Once we analyzed the properties of theWhittle index weset out to define the structure of the active set in order toobtain an analytically and computationally tractable Whittleindex expression In such a way to derive a closed-formcharacterization of the Whittle index of (119886 119889 119899) state first ofall it is necessary to specify the activity of (119886 119889 + 1119898) and(119886 + 119903

119899 119889 + 1119898) states As concluded from Whittle index

properties it is known that (119886 119889+1119898 le 119899)cup(119886+119903119899 119889+1119898 le

119899) states are passive However it is not trivial to specify abovewhich attained service level andor delay level if (119886 119889+1119898 gt

119899)cup (119886+ 119903119899 119889 + 1119898 gt 119899) states with better channel condition

are active or passiveNonetheless in the best channel the active set is totally

specified since all the states that compute in the index are pas-sive because of nonexisting channel improvement and beingin the future attained service equal or higher and delay higherAs shown in Proposition 3 for the best channel condition theWhittle index has a closed-form expression where the indexvalue is the multiplication between completion probabilityand normalizedMOS function Note that the obtainedWhit-tle index expression in the best channel state is equivalent tothe GM index achieved for constant channel [16]

Proposition 3 The Whittle index for Problem (8) in the bestchannel condition is given by

Vlowast(119886119889119873)

= 120583(119886119873)

sdot MOS (119889 + 1) (13)

Proof It is known that for Vlowast(119886119889119873)

index computation thatF = 0 In this way using (11) and (12) RF

(119886119889+1119898)=

120573sum119898isinN 119902

119898RF(119886119889+2119898)

RF(119886119889+2119898)

= 120573sum119898isinN 119902

119898RF(119886119889+3119898)

RF(119886119889gt119889max 119898)

= 0 Hence analogolously RF(119886+119903119899 119889+1119898)

= 0WF(119886119889+1119898)

= 0 and WF(119886+119903119899 119889+1119898)

= 0 Substituting thesenull terms in expression (10) and being 120573 = 1 we achieve(13)

Besides as stated before obtaining a computationallytractable Whittle index expression for any channel conditionis not possible In such situation in order to achieve atractable Whittle-based QoE-aware index rule for time-varying channels we propose a Whittle-based approximateheuristic in the next subsection

41 Approximate Whittle Index Proposal In the following wepropose an approximate solution for the QoE-aware Whittleindex in a time-varying channel context (10)

First of all we will determine the structure of the activeset For the Whittle index computation of (119886 119889 119899) state aspreviously concluded it is known that (119886 119889 + 1119898 le 119899) cup (119886+

119903119899 119889 + 1119898 le 119899) states are passive but we need to define the

unknown activity of (119886 119889 + 1119898 gt 119899) cup (119886 + 119903119899 119889 + 1119898 gt 119899)

states Thus so as to determine if these states are active orpassive we apply the following two considerations

(1) 120583 approximation

120583(119886119898)

asymp 120583(119886+119903119897 119898)

(14)

(2) MOS approximation

MOS (119889) asymp MOS (119889 + 1) (15)

On the one hand we assume that for any 119897 = 1 2 119873

value that 120583(119886119898)

asymp 120583(119886+119903119897 119898)

(14) It is known that (119886 119889119898 gt

119899) states are active and being 120583(119886119898)

asymp 120583(119886+119903119899 119898)

we couldconsider that (119886 + 119903

119899 119889 119898 gt 119899) states are active

On the other hand we assume MOS(119889) asymp MOS(119889 + 1)(15) since QoE degradation in a higher delay level is neg-ligible in the millisecond scale that delay evolves Takinginto account the approximation about MOS function being(119886 119889119898 gt 119899) states active we can suppose that (119886 119889 + 1119898 gt

119899) states are also active This way combining approximations(14) and (15) we consider that (119886 + 119903

119899 119889 + 1119898 gt 119899) states are

activeThen suppose that the aforementioned 120583 and

MOS approximations cause 119877119886119889

= sum119898119902119898119877(119886119889119898)

asymp

sum119898119902119898119877(119886+119903119897 119889+1119898)

and 119882119886119889

= sum119898119902119898119882(119886119889119898)

asymp

sum119898119902119898119882(119886+119903119897 119889+1119898)

So using these reward and workapproximations the first simplified expression for theWhittle index formulation (10) is

Vlowast(119886119889119899)

=120573120583(119886119899)

(MOS (119889 + 1) minus 119877119886119889)

1 minus 120573120583(119886119899)

119882119886119889

(16)

In such a way applying the previous simplificationswe obtain the Whittle index approximation presentedin Proposition 4 what we call attained service PI MOS(ASPIM) The methodology to compute the closed-formexpression of this QoE-aware Whittle-based index is pre-sented in the Appendix

Proposition 4 The formulation of the ASPIM index is

119860119878119875119868119872 = Vlowast(119886119889119899)

= lim120573rarr1

120573120583(119886119899)

sdot MOS (119889 + 1) (1 minus 120573)1 minus 120573 + 120573sum

119898gt119899119902119898(120583(119886119898)

minus 120583(119886119899)

)

(17)

8 Abstract and Applied Analysis

Table 1 CQIs and corresponding transmission rates (Mbps)

CQI 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15119904 0 42 672 84 11256 168 2184 252 2688 336 4468 504 5376 672 758 8064

If we analyze expression (17) we observe that this QoE-aware and channel-aware Whittle-based ASPIM index isimplementable This proposal depends on QoE size andchannel properties Moreover the obtained index for thebest channel condition is equal to the original Whittle index(13) since the summation in the denominator becomes nullHowever as shown in (18) when channel state is not the bestthe index value goes to zero

ASPIM119899 =119873

= lim120573rarr1

120583(119886119899)

sdot MOS (119889 + 1) (1 minus 120573)sum119898gt119899

119902119898(120583(119886119898)

minus 120583(119886119899)

)997888rarr 0

(18)

Hence a user in its best channel state with nonnullinstantaneous normalized MOS function has priority overa user which is not in its best channel condition Thus wesummarize the proposed QoE-aware Whittle-based ASPIMindex rule in Definition 5

Definition 5 The ASPIM index rule consists of at everydecision slot

(i) serving the user in its best channel condition withnonnull instantaneous normalized MOS functionwith the highest value of 120583

(119886119873)sdot MOS(119889 + 1)

(ii) if there is no user in its best channel condition withnonnull instantaneous normalized MOS functionserving the user with the highest value of (120583

(119886119899)sdot

MOS(119889 + 1))sum119898gt119899

119902119898(120583(119886119898)

minus 120583(119886119899)

) (using (18)normalized by (1 minus 120573))

So if channel condition is not the best the index valueis the ratio of the multiplication between the actual servicecompletion and the instantaneous normalizedMOS functionrespect to the expected potential improvement of the comple-tion probability

Therefore in the context of time-varying channels aimedat maximizing average MOS we have proposed ASPIMscheduling algorithm which is a simple and tractable QoE-aware channel-aware and size-based index rule Neverthe-less in order to verify the correct behavior of this heuristicwe analyze its performance in the subsequent section

5 Performance Evaluation

In this section we evaluate the performance of the proposedASPIM index rule (see proposal in Definition 5) To that endwe compare the QoE performance of our heuristic with theone achieved by well-known scheduling strategies in severalsimulation scenarios Below we provide a brief description ofthe scheduling priority rules used in our experiments

(i) Max rate (MR) this opportunistic policy consists inserving the user with the best channel capacity

(ii) Proportional fair (PF) this channel-aware disci-pline serves the user with the highest ratio betweeninstantaneous transmission rate and current attainedthroughput

(iii) Cost and attained service dependant 120583 (119888AS120583)although it might seem counter-intuitive minimizingmean delay does not result in maximizing averageMOS [24] This way in order to compare the per-formance with delay-aware solutions we propose toadapt the classical 119888120583 rule aimed at minimizing meandelay [25] to attained serviceThis adaptation consistsof serving the user with the highest value of 119888120583

(119886119899)

being 119888 the waiting cost paid for every slot while flowtransmission is uncompleted

(iv) GM [16] thisQoE-aware pioneering index rule origi-nally proposed for constant channel capacity choosesthe user with the highest value of 120583

(119886119899)sdot MOS(119889 + 1)

In case of ties we use random tie-breaking ruleConcerning traffic flows in the system we use the Pareto

size distribution given in (1) in simulations whose 120572 and 120574parameters determine mean flow size We consider typicalmean sizes used in wireless networks 05Mb (small traffica web page email) 5Mb (medium traffic a PDF documenta picture etc) and 50Mb (MP3 audio a group of picturesfrom a video sequence etc) User traffic flows arrive to thescheduler according to a Poisson process with 120582 rate

Referring to network characteristics we use transmis-sion rates that depend on CQIs used in real wirelessnetworks which are associated with 4G modulation andcoding schemes We show the mapping between CQI valuesand transmission rates in Table 1 The presented values areadapted and extrapolated from [26] for a cell of 20MHz

Moreover we consider different network loads in order toanalyze the behavior of scheduling strategies under differentnetwork conditions Flow arrival rate determines networkload 120588 where 120588 = 120582 sdot (E[119883]119904

119873) We consider seven network

states in this study low load (120588 = 025 0375) medium load(120588 = 05 075) and high load (120588 = 085 09 095)

Regarding QoE aspects as typically utilized in perceivedquality studies [27] we use a log-shaped MOS function forQoE degradation presented in (19) This way employing theMOS curve modelling provided in [21] as starting point andtaking into account the flow size the network type and theuser expertise we have suitably defined 119889min and 119889max delaythresholds used in simulations and consequently 119888

1and 1198882

MOS coefficients Consider

MOS = 1198881minus 1198882sdot log (119889) (19)

We have implemented the whole network environmentfor the scheduling of network resources in MATLAB Inrelation to the performed experiments for each scenario

Abstract and Applied Analysis 9

Table 2 Parameter set in the experimental scenarios

Scenario MOS Size Channel 119888

1 119889min = 05 s 119889max = 10 s 120572 = 15 120574 = 4 sdot 10minus7 CQI = 3 5 11198881= 407 119888

2= 133 E[119883] = 5Mbit 119902

2= 05

2 119889min = 04 s 119889max = 06 s 120572 = 15 120574 = 4 sdot 10minus7 CQI = 3 5 11198881= minus403 119888

2= 986 E[119883] = 5Mbit 119902

2= 05

3 119889min = 05 s 119889max = 10 s 120572 = 15 120574 = 4 sdot 10minus7 CQI = 3 9 11198881= 407 119888

2= 133 E[119883] = 5Mbit 119902

2= 05

4 119889min = 005 s 119889max = 1 s 120572 = 15 120574 = 4 sdot 10minus6 CQI = 3 5 11198881= 1 1198882= 133 E[119883] = 05Mbit 119902

2= 05

5

119889min1 = 05 s 119889max1 = 5 s 1205721= 15 120574

1= 4 sdot 10

minus7

see Table 3 5 111988811= 379 119888

21= 173 E[119883

1] = 5Mbit

119889min2 = 5 s 119889max2 = 30 s 1205722= 15 120574

2= 4 sdot 10

minus8

11988812= 859 119888

22= 223 E[119883

2] = 50Mbit

Table 3 Channel state probabilities in Scenario 5

CQI 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15119902 028 012 009 008 008 008 007 006 005 004 003 001 0009 00005 00003 00002

for each combination of scheduling discipline and networkload we have carried out a set of 10 rounds of 10000 slength simulations These rounds differ in the randomlypregenerated traces (input vector) of sizes and arrivals Thusnot only the overall average MOS values will be provided butwe will also include their 95 confidence intervals

Besides in order to guarantee that the obtained perfor-mance results are generally valid we will analyze differentsimulation scenarios We have chosen five relevant settingswhich differ inQoE size channel orand cost characteristicsThe parameters of these scenarios are summarized in Table 2Note that in the first four scenarios we restrict ourselves tothe case of two channel conditions so as to get fundamentalperformance results

We have carefully selected the scenarios used in simula-tions First of all we define a setting (Scenario 1) in which thevalues of all the parameters are typical or standard the valuesof these parameters introduce low error in the approxima-tions considered for our ASPIM scheduler proposal Thenin order to evaluate worst performance cases for ASPIMwe have chosen three scenarios (Scenario 2 Scenario 3 andScenario 4) in which the error of the approximation used forASPIM is high due to a different factor (gradient ofMOS rateand size resp)

Apart from that it is worthmentioning that for obtainingthe ASPIM index rule that we have considered an isolatedMDP model per user Hence the achieved solution is basedon a unique user type or single class However it would beinteresting to examine the proposal when there are differenttypes of users which is analyzed in Scenario 5 Furthermorethis last setting resembles a real 4G wireless network Notethat even though this technology allows the simultaneoustransmission of multiple flows per TTI we assume that asingle flow transmits in each TTI

03 04 05 06 07 08 09

25

3

35

4

45

5

E[M

OS]

ASPIMMRPF

cAS120583GM

120588

Figure 5 E[MOS] for the basic scenario

Next we describe the results we have achieved in theaforementioned scenarios

51 Scenario 1 Basic Case In this first family of simulationswe consider a basic or typical scenario which takes intoaccount the equiprobable channel case and medium-sizedself-similar flows Figure 5 shows averageMOS results for thissetting employing different scheduling policies and networkloadsThemain conclusion achieved from these results is thatASPIM outperforms the rest of disciplines which is betterthan the QoE-aware GM rule proposed for constant channelcapacity Moreover contrary to the other policies it holds

10 Abstract and Applied Analysis

0 02 04 06 08 10

1

2

3

4

d (s)

PDF(d

)

(a)

1 2 3 4 50

01

02

03

04

05

06

07

MOS

PDF(

MO

S)

ASPIMMRPF

cAS120583GM

(b)

1 2 3 4 50

02

04

06

08

1

MOS

CCD

F(M

OS)

ASPIMMRPF

cAS120583GM

Satisth

(c)

Figure 6 Statistics in Scenario 1 delay PDF (a) MOS PDF (b) and MOS CCDF (c)

averageMOSvalues in the same level for all120588 Referring to thetraditional channel-aware disciplines both PF and MR givebad behavior comparing to the rest of scheduling strategies

In order to justify the previous performance results wenow analyze delay andMOS statistics for the highest networkload considered In such a way as regarded in Figure 6(a) forthe Probability Density Function (PDF) of delay we highlightthe probabilitymass or peak that appears for some schedulingdisciplines before the low delay threshold (05 s) The heightand weight of a peak belonging to a policy have direct impacton its average MOS ASPIM discipline shows the highestand the widest peak among the presented ones followedby GM but with a notable difference Therefore our novelQoE-aware policy seems to try to finish the transmissionof most of the flows before 119889min which is reflected in theaforementioned peaks Note that the cAS120583 rule aimed atminimizing delay behaves in different manner which showsmore fairness among delays in the range shown compared tothe QoE-aware ones

What is previously concluded about delay statistics hasa direct reflection on QoE statistics This way as can beseen in the PDF of MOS presented in Figure 6(b) if theobjective is to maximize average QoE it is important onthe one hand to be the height and the weight of the peaksaround the maximum MOS high and on the other hand tobe the value around the minimum MOS low ASPIM fulfilsthese positive properties and consequently it is the bestin average QoE terms Besides if we analyze the Comple-mentary Cumulative Distribution Function (CCDF) of MOSprovided in Figure 6(c) we observe that ASPIM gives thehighest probability of being MOS value higher Consideringfor example a QoE satisfaction threshold satisth of MOS =4 in this point ASPIM improves GM a 10 whereas thedeterioration of MR and PF is higher than a 50

52 Scenario 2 High Demanding Users Case This scenarioreflects the case of users with low tolerance to delay in

Abstract and Applied Analysis 11

03 04 05 06 07 08 09

05

06

07

08

09

ASPIMMRPF

cAS120583GM

120588

E[M

OS]

2E

[MO

S]1

Figure 7 E[MOS] for abrupt MOS scenario

which theMOS function in theQoE degradation range showsa great slope Thus in this setting the error of the MOSapproximation (15) is high since the gradient of MOS amongconsecutive delay levels becomes significant In Figure 7we compare the results of this second scenario and thebasic scenario From the illustrated graph we could say thatour QoE-aware proposal adapts to any MOS function Thisis whereas for the rest of disciplines (excluding GM) thenegative effects due to the abrupt degradation of the QoEutility function are appreciable ASPIMmanages to avoid thisadapting to any MOS function

53 Scenario 3 Higher Error Case due to Rate During thesimplification process carried out for obtaining the ASPIMindex rule in Section 41 the 120583 approximation used (see (14))introduces a rate dependant error Note that increasing therate in the good channel state makes bigger the error of the 120583approximation used Therefore we will analyze the obtainedperformance for our new QoE-aware proposal under thishigher error due to rate In the previous scenarios we haveassumed that the transmission rate in the good channelcondition is twice the rate in the bad channel state In thissetting we consider that the transmission rate in the goodchannel is four times the transmission rate in the bad channelThis way according to the QoE results presented in Figure 8in spite of a higher error caused by rate ASPIM is superior tothe rest of policies

54 Scenario 4 High Error Case due to Size Besides it isknown that for a distribution with a decreasing hazard ratesuch as Pareto that the gradient of completion probabilityincreases as long as the mean size decreases which makesthe error of the 120583 approximation (14) for the ASPIM ruleachievement bigger In this way in this scenario we verifythat ASPIM gives satisfactory behavior under a more notableerror due to size where we consider a mean size ten times

03 04 05 06 07 08 09

3

35

4

45

5

E[M

OS]

ASPIMMRPF

cAS120583GM

120588

Figure 8 E[MOS] under higher error due to rate

03 04 05 06 07 08 093

35

4

45

5E

[MO

S]

ASPIMMRPF

cAS120583GM

120588

Figure 9 E[MOS] under high error due to size

lower We conclude from Figure 9 that even under a higherror caused by size that ASPIM shows good performancebeing the best alternative

55 Scenario 5 A Real Network Case This last scenarioreflects a real 4G wireless network context but with thesimplification that a single user transmits in each TTI To thatend we use CQI traces obtained from a system-level radioaccess simulator [28] considering users moving at 5 kmhin the cell For this setting with dynamic users we definechannel propagation by the Extended Pedestrian Amodel Asin current 4G networks we consider 16 channel conditionsThe state probabilities of these channel states are collectedin Table 3 which are decreasing with the improvement ofchannel quality In this scenario we consider two classes ofusers which differ in size QoE and cost The mean size

12 Abstract and Applied Analysis

03 04 05 06 07 08 09

1

2

3

4

5

E[M

OS]

120588

(a)

03 04 05 06 07 08 091

2

3

4

5

E[M

OS]

ASPIMMRPF

cAS120583GM

120588

(b)

03 04 05 06 07 08 091

2

3

4

5

E[M

OS]

ASPIMMRPF

cAS120583GM

120588

(c)

Figure 10 E[MOS] for 4G scenario aggregate (a) class 1 (b) class 2 (c)

in class 2 is ten times bigger than in class 1 and we givepriority to the small-sized class by having bigger cost andQoErequirement

Under this mixture of classes and realistic channel mod-els in Figure 10(a) we observe that scheduling strategiesshow the same tendency as in the previous scenarios It isremarkable that the difference between ASPIM and GM isbigger especially for high 120588 values Moreover under thehighest network load ASPIM achieves MOS values aboveQoE satisfaction threshold (MOS = 4) Furthermore averageMOS values forMR and PF dramatically decrease comparingwith the previous settings

Concerning class behaviour we show per-class resultsin Figures 10(b) and 10(c) As can be concluded fromthe presented results QoE-aware policies guarantee fairnessamong classes and ASPIM is the best choice also inside classAside from that for the cost dependant cAS120583 discipline thedeterioration of the class with no priority is significant

6 Conclusions

This paper goes into detail about obtaining a schedulingalgorithmaimed atmaximizing usersrsquo perception of quality inchannels with time-varying capacity Furthermore this workconsiders traffic flowswith nonexponential size distributionscontrary to previous approaches that assume memorylessdistributions

As first contribution we provide a MDP-based modelfor the scheduling problem presented This model combinesQoE-awareness size-awareness and channel-awarenessNev-ertheless since this model can not be solved either analyt-ically or computationally from the proposed MDP modelwe derive a simple tractable and implementable well-performing scheduling index rule by applying amethodologybased on Whittle approach

The proposed scheduling solution still combines QoE-size- and channel-awareness Our scheduling proposal givespriority to users in their best channel condition that are

Abstract and Applied Analysis 13

out of the service unavoidable range As verified fromsimulation results the proposed scheduling strategy showssuitable subjective quality performance in a wide range ofscenarios including the case of a simplified 4G network withheterogeneous users

Therefore the results of this work present a relevantmathematical basis for developing scheduling algorithmsaimed at maximizing QoE in current and future networkswith time-varying channel capacity This way the proposedchannel-aware ASPIM discipline will be useful for networkoperators in order to guarantee to their customers servicesatisfaction in time-varying wireless networks Apart fromthat the approximations and simplifications used in Whittlemethod are applicable to any area when the gradient ofsystem components is considerably small among consecutivedecision slots

As future research we will extend our work to amultiuserapproach considering the simultaneous transmission ofmul-tiple flows per TTI

Appendix

Proof of Proposition 4

LemmaA1 Suppose that states (119886 119889+1119898 le 119899)cup(119886+119903119899 119889+

1119898 le 119899) are passive and that states (119886 119889 + 1119898 gt 119899) cup (119886 +

119903119899 119889 + 1119898 gt 119899) are active and

Vlowast(119886119889119899)

=120573120583(119886119899)

( MOS (119889 + 1) minus 119877119886119889)

1 minus 120573120583(119886119899)

119882119886119889

(A1)

Along with this if we assume that 119877119886119889

= sum119898119902119898119877(119886119889119898)

asymp

sum119898119902119898119877(119886+119903119897 119889+1119898)

and 119882119886119889

= sum119898119902119898119882(119886119889119898)

asymp

sum119898119902119898119882(119886+119903119897 119889+1119898)

for any 119897 = 1 2 119873 value for theundiscounted case

Vlowast(119886119889119899)

= lim120573rarr1

120573120583(119886119899)

sdot MOS (119889 + 1) (1 minus 120573)1 minus 120573 + 120573sum

119898gt119899119902119898(120583(119886119898)

minus 120583(119886119899)

) (A2)

Proof Referring to work elements using (12)

119882(119886119889119898le119899)

= 120573sum

119898

1199021015840

119898119882(119886119889+1119898

1015840)= 120573119882

119886119889

119882(119886119889119898gt119899)

= 1 + 120573 (1 minus 120583(119886119898)

)sum

119898

1199021015840

119898119882(119886+119903119898 119889+1119898

1015840)

= 1 + 120573 (1 minus 120583(119886119898)

)119882119886119889

(A3)

And by (A3) expressions

119882119886119889= (1 minus sum

119898gt119899

119902119898)120573119882

119886119889+ sum

119898gt119899

119902119898(1 + 120573 (1 minus 120583

(119886119898))119882119886119889)

(A4)

This way isolating119882119886119889

119882119886119889=

sum119898gt119899

119902119898

1 minus 120573 + 120573sum119898gt119899

119902119898120583(119886119898)

(A5)

Analogously for reward elements using (11)

119877(119886119889119898le119899)

= 120573sum

119898

1199021015840

119898119877(119886119889+1119898

1015840)= 120573119877119886119889

119877(119886119889119898gt119899)

= 120573[(1 minus 120583(119886119898)

)sum

119898

1199021015840

119898119877(119886+119903119898 119898

1015840)+ 120583(119886119898)

sdot MOS (119889 + 1)]

= 120573 [(1 minus 120583(119886119898)

) 119877119886119889+ 120583(119886119898)

sdot MOS (119889 + 1)] (A6)

And thus from (A6)

119877119886119889= (1 minus sum

119898gt119899

119902119898)120573119877119886119889

+ sum

119898gt119899

119902119898(120573 [(1 minus 120583

(119886119898)) 119877119886119889+ 120583(119886119898)

sdot MOS (119889 + 1)])

(A7)

Isolating 119877119886119889

119877119886119889=120573sum119898gt119899

119902119898120583(119886119898)

MOS (119889 + 1)1 minus 120573 + 120573sum

119898gt119899119902119898120583(119886119898)

(A8)

Substituting (A5) and (A8) in (A1) and by simplifyingwe obtain

Vlowast(119886119889119899)

= (120573120583(119886119899)

(MOS (119889 + 1)

minus120573sum119898gt119899

119902119898120583(119886119898)

MOS (119889 + 1)1 minus 120573 + 120573sum

119898gt119899119902119898120583(119886119898)

))

times (1 minus 120573120583(119886119899)

sum119898gt119899

119902119898

1 minus 120573 + 120573sum119898gt119899

119902119898120583(119886119898)

)

minus1

= (120573120583(119886119899)

(MOS (119889 + 1) (1 minus 120573 + 120573sum119898gt119899

119902119898120583(119886119898)

)

minus120573sum

119898gt119899

119902119898120583(119886119898)

MOS (119889 + 1)))

times (1 minus 120573 + 120573sum

119898gt119899

119902119898120583(119886119898)

minus 120573120583(119886119899)

sum

119898gt119899

119902119898)

minus1

=120573120583(119886119899)

sdot MOS (119889 + 1) (1 minus 120573)1 minus 120573 + 120573sum

119898gt119899119902119898(120583(119886119898)

minus 120583(119886119899)

)

(A9)

And therefore for the undiscounted case the 120573 rarr 1

limit for expression (A9) is (A2)

Conflict of Interests

The authors declare that there is no conflict of interestsregarding the publication of this paper

14 Abstract and Applied Analysis

Acknowledgment

The research leading to these results has received fundingfrom the European Union Seventh Framework Programme(FP72007ndash2013) under Grant agreement 284863 (FP7 SECGERYON)

References

[1] ITU-T P10G100 Amendment 1 (0107) New Appendix I-definition of Quality of Experience (QoE) 2006

[2] S Aalto and P Lassila ldquoFlow-level stability and performance ofchannel-aware priority-based schedulersrdquo in Proceeedings of the6th Euro NF Conference on Next Generation Internet (NGI rsquo10)IEEE June 2010

[3] L KleinrockQueueing Systems vol 2 ofComputer ApplicationsJohn Wiley amp Sons 1976

[4] J Gittins ldquoBandit processes and dynamic allocation indicesrdquoJournal of the Royal Statistical Society B Methodological vol 41no 2 pp 148ndash177 1979

[5] K Avrachenkovt U Ayesta P Brown and E Nyberg ldquoDif-ferentiation between short and long TCP ows predictabilityof the response timerdquo in Proceedings of the 23rd Annual JointConference of the IEEE Computer and Communications Societies(INFOCOM rsquo04) vol 2 p 762 IEEE 2004

[6] U Ayesta M Erausquin and P Jacko ldquoA modeling frameworkfor optimizing the flow-level scheduling with time-varyingchannelsrdquo Performance Evaluation vol 67 no 11 pp 1014ndash10292010

[7] S Aalto U Ayesta and R Righter ldquoProperties of the Gittinsindex with application to optimal schedulingrdquo Probability in theEngineering and Informational Sciences vol 25 no 3 pp 269ndash288 2011

[8] M Puterman ldquoMarkov decision processesrdquo in Handbooks inOperations Research and Management Science vol 2 pp 331ndash434 1990

[9] C H Papadimitriou and J N Tsitsiklis ldquoThe complexityof Markov decision processesrdquo Mathematics of OperationsResearch vol 12 no 3 pp 441ndash450 1987

[10] J Nino-Mora ldquoDynamic priority allocation via restless banditmarginal productivity indicesrdquo TOP vol 15 no 2 pp 161ndash1982007

[11] P Whittle ldquoRestless bandits activity allocation in a changingworldrdquo Journal of Applied Probability vol 25 pp 287ndash298 1988

[12] U Ayesta M Erausquin M Jonckheere and I M VerloopldquoScheduling in a random environment stability and asymptoticoptimalityrdquo IEEEACM Transactions on Networking vol 21 no1 pp 258ndash271 2013

[13] S Khan S Thakolsri E Steinbach and W Kellerer ldquoQoE-based cross-layer optimization for wireless multiuser systemsrdquoin Proceedings of the 18th ITC Specialist Seminar on Quality ofExperience 2008

[14] S Thakolsri S Khan E Steinbach and W Kellerer ldquoQoE-driven cross-layer optimization for high speed downlink packetaccessrdquo Journal of Communications vol 4 no 9 pp 669ndash6802009

[15] P Ameigeiras J J Ramos-Munoz J Navarro-Ortiz PMogensen and J M Lopez-Soler ldquoQoE oriented cross-layer design of a resource allocation algorithm in beyond 3GsystemsrdquoComputer Communications vol 33 no 5 pp 571ndash5822010

[16] I Taboada F Liberal J O Fajardo and U Ayesta ldquoQoE-aware optimization of multimedia flow schedulingrdquo ComputerCommunications vol 36 no 15-16 pp 1629ndash1638 2013

[17] U Ayesta M Erausquin and P Jacko ldquoResource-sharing in asingle server with time-varying capacityrdquo in Proceeding of the49th Annual Allerton Conference on Communication Controland Computing (Allerton rsquo11) pp 377ndash383 Monticello Ill USASeptember 2011

[18] K Thompson G J Miller and R Wilder ldquoWide-area internettraffic patterns and characteristicsrdquo IEEE Network vol 11 no 6pp 10ndash23 1997

[19] M Fiedler T Hossfeld and P Tran-Gia ldquoA generic quantitativerelationship between quality of experience and quality ofservicerdquo IEEE Network vol 24 no 2 pp 36ndash41 2010

[20] A Richards M Antoniades V Witana and G Rogers ldquoMap-ping user level QoS from a single parameterrdquo in Proceedingsof the International Conference on Multimedia Networks andServices Citeseer 1998

[21] E Ibarrola F Liberal I Taboada and R Ortega ldquoWebQoE evaluation in multi-agent networks validation of ITU-TG1030rdquo in Proceedings of the 5th International Conference onAutonomic and Autonomous Systems (ICAS rsquo09) pp 289ndash294Valencia Spain April 2009

[22] S Aalto and U Ayesta ldquoOptimal scheduling of jobs with aDHR tail in the MG1 queuerdquo in Proceedings of the 3rd Inter-national Conference on Performance Evaluation Methodologiesand Tools (ValueTools rsquo08) article 50 Institute for ComputerSciences Social-Informatics and Telecommunications Engi-neering Athens Greece 2008

[23] V Visweswaran ldquoDecomposition techniques for MILPlagrangian relaxationrdquo in Encyclopedia of Optimization pp632ndash638 Springer New York NY USA 2009

[24] I Taboada J O Fajardo and F Liberal ldquoPerformance analysisof scheduling algorithms for web QoE optimization in wirelessnetworksrdquo Network Protocols and Algorithms vol 4 pp 27ndash432012

[25] J A van Mieghem ldquoDynamic scheduling with convex delaycosts the generalized 119888120583 rulerdquoTheAnnals of Applied Probabilityvol 5 no 3 pp 809ndash833 1995

[26] S Sesia I Toufik and M Baker LTE-The UMTS Long TermEvolution FromTheory to Practice Wiley 2011

[27] P Reichl B Tuffin and R Schatz ldquoLogarithmic laws inservice quality perception where microeconomics meets psy-chophysics and quality of experiencerdquo Telecommunication Sys-tems vol 52 no 2 pp 587ndash600 2013

[28] J C Ikuno M Wrulich and M Rupp ldquoSystem level simulationof LTE networksrdquo in Proceedings of the IEEE 71st VehicularTechnology Conference Taipei Taiwan May 2010

Submit your manuscripts athttpwwwhindawicom

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical Problems in Engineering

Hindawi Publishing Corporationhttpwwwhindawicom

Differential EquationsInternational Journal of

Volume 2014

Applied MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical PhysicsAdvances in

Complex AnalysisJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

OptimizationJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Operations ResearchAdvances in

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Function Spaces

Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of Mathematics and Mathematical Sciences

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Algebra

Discrete Dynamics in Nature and Society

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Decision SciencesAdvances in

Discrete MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Stochastic AnalysisInternational Journal of

Abstract and Applied Analysis 5

Conjecture 1 Problem (8) is indexable

Due to the complexity of our model proving the index-ability of Problem (8) is not an easy task In this way weassume Conjecture 1 and therefore we suppose that foreach (119886 119889 119899) state of user 119896 the Whittle index exists Fromnow on we omit user label 119896 In spite of considering theundiscounted case the mathematical analysis is carried outfor the discounted case so as to avoid infinite values causedby 1minus120573 terms in denominators for 120573 = 1 and then we obtainthe undiscounted index values in the limit 120573 rarr 1

Following Whittle index definition in [10] we formallywrite the Whittle index for Problem (8) Vlowast

(119886119889119899) as

Vlowast(119886119889119899)

= VF(119886119889119899)

=R⟨1F⟩

(119886119889119899)minusR⟨0F⟩

(119886119889119899)

W⟨1F⟩

(119886119889119899)minusW⟨0F⟩

(119886119889119899)

(9)

Whittle index VF(119886119889119899)

represents the rate betweenmarginal reward and marginal work where the marginalreward (work) is the difference of the expected reward earned(work required) by serving and not serving at the initial state(119886 119889 119899) and employing policyF afterwards

Lemma 2 For any state (119886 119889 119899) and under any policy F weobtain

VF(119886119889119899)

= (120573[(1 minus 120583(119886119899)

) sum

119898isinN

119902119898R

F

(119886+119903119899 119889+1119898)+ 120583(119886119899)

sdot MOS (119889 + 1) minus sum

119898isinN

119902119898R

F(119886119889+1119898)

])

times (1 + 120573[(1 minus 120583(119886119899)

) sum

119898isinN

119902119898W

F

(119886+119903119899 119889+1119898)

minus sum

119898isinN

119902119898W

F(119886119889+1119898)

])

minus1

(10)

Proof From the definition of reward and work respectivelywe have

RF(119886119889119899)

=

R⟨1F⟩

(119886119889119899)= 120573[(1 minus 120583

(119886119899)) sum

119898isinN

119902119898RF

(119886+119903119899 119889+1119898)

+ 120583(119886119899)

sdot MOS (119889 + 1) ]

(119886 119889 119899) isin F

R⟨0F⟩

(119886119889119899)= 120573 sum

119898isinN

119902119898RF(119886119889+1119898)

(119886 119889 119899) notin F

(11)

(1)F0= 0

(2) for 119894 = 1 rarr number of states do(3) (119886 119889 119899)

119894isin argmax VF119894minus1

(119886119889119899) (119886 119889 119899) notin F

119894minus1

(4) Vlowast(119886119889119899)119894

= VF119894minus1(119886119889119899)119894

F119894= F119894minus1cup (119886 119889 119899)

119894

(5) end for

Algorithm 1AG-algorithm for Problem (8)

WF(119886119889119899)

=

W⟨1F⟩

(119886119889119899)= 1 + 120573 (1 minus 120583

(119886119899)) sum

119898isinN

119902119898WF(119886+119903119899 119889+1119898)

(119886 119889 119899) isin F

W⟨0F⟩

(119886119889119899)= 120573 sum

119898isinN

119902119898WF(119886119889+1119898)

(119886 119889 119899) notin F

(12)

By substituting the expressions (11) and (12) in formula(9) we achieve a more complete VF

(119886119889119899)(10) formulation

Nevertheless in order to obtain an analytically tractableWhittle index expression for (10) in the state (119886 119889 119899) it isnecessary to determine the policyF of the future states thathave influence on the index computation of this state As canbe observed from (10) (11) and (12) expressions in thesefuture states the value of attained service is maintained orincreased whereas delay grows over time Since increasingattained service decreases instant completion probability fora Pareto distribution and increasing delay degrades QoE itwould seem intuitively correct to suppose that future stateswith the same or worse channel conditions are passive Insuch a way we could say that in a state with the same or worsechannel condition being attained service and delay higherthat the Whittle index value is lower However for futurestates with better channel conditions their passive or activeproperty is not trivial because the improvement in channelcondition does not guarantee being a better state due to theaforementioned behavior of the increasing attained serviceand delayTherefore finding the active set of all the influentialstates for Whittle index computation becomes a challengingtask

In such situation aimed at achieving the characteristicsof the optimal policy and the subsequent active set buildingevolution we will use an algorithm called Adaptive-GreedyshortlyAG-algorithm (see [10] for a survey) This algorithmcomputes Whittle indices numerically taking into accountoptimal active sets This way we have implemented AG-algorithm for our case study in order to verify the fundamen-tal properties of the Whittle index for our problem whichwill be useful to determine the active set and consequentlya closed-form index expression We provide the operation ofthe applied algorithm in Algorithm 1

We have performed several numerical experiments for120573 = 1 case in which we have reduced the state space ofboth delay and channel states so as to do the execution of thealgorithm computationally and in time viably We have onlyconsidered two channel conditions and eleven levels of delay

6 Abstract and Applied Analysis

times10minus3

6

5

4

3

2

1

00

0

5 5

10

10times107

ad

Vlowast (ad1)

(a)

0

0

5 5

10

10times107

a d

Vlowast (ad2)

001

0008

0006

0004

0002

0

(b)

Figure 3 Whittle index values obtained from AG-algorithm for two channel conditions and different attained services and delays For aPareto distribution with 120572 = 15 and E[119883] = 5Mb 119902

2= 05 119903

1= 84Kb 119903

2= 168Kb 119889min = 2 s and 119889max = 9 s

0

0

5 5

10

10

times107

ad

Vlowast (ad1)

001

0008

0006

0004

0002

0

q2 = 01

(a)

times10minus3

4

3

2

1

0

0

0

5 5

10

10

times107

a d

Vlowast (ad1)

q2 = 09

(b)

Figure 4 Whittle index values obtained fromAG-algorithm for different channel state probabilities For a Pareto distribution with 120572 = 15and E[119883] = 5Mb 119903

1= 84Kb 119903

2= 168Kb 119889min = 2 s and 119889max = 9 s

in the scenarios In such a way from experimental results weconjecture the following fundamental properties

(1) TheWhittle index value depends on attained servicedelay and channel state Moreover as can be seenin graphs from Figures 3 and 4 we can clearlydistinguish the ranges of MOS function above theupper delay threshold the index value is null

(2) For the same channel condition and the samedelay level Whittle index values are decreasing withattained service Vlowast

(119886119889119898)gt Vlowast(1198861015840119889119898)

forall1198861015840

gt 119886 (illus-trated in Figures 3 and 4)

(3) For the same channel condition and the same attainedservice levelWhittle index values are decreasing withdelay Vlowast

(119886119889119898)gt Vlowast(119886119889+1119898)

(See Figures 3 and 4)

Abstract and Applied Analysis 7

(4) In a better channel condition for the same attainedservice level and the same delay level the Whittleindex value is higher Vlowast

(1198861198891198981)gt Vlowast(1198861198891198982)

forall1198981gt 1198982

(Observe Figure 3)(5) Opposite to the previous channel-aware work [6]

where Whittle indices in the best channel conditionare infinite having absolute priority in this channelstate in our case being in the best channel conditiondoes not guarantee that the Whittle index value ishigher than in the rest of the channel conditions it ispossible to fulfil Vlowast

(119886119889119898lt119873)gt Vlowast(11988610158401198891015840119873)

If we comparethe two graphs in Figure 3 it can be easily seen howfor several attained services andor delayed values theWhittle index values in the worst channel conditionare higher than in the best channel condition (takingalso into account nonnull values)

(6) Dependency on channel condition probabilities hilefor channel condition 119873 the Whittle index valueremains invariant with channel state probability forthe rest of channel conditions its value decreases aslong as channel state probabilities increase in betterchannel conditions (shown in Figure 4)

Once we analyzed the properties of theWhittle index weset out to define the structure of the active set in order toobtain an analytically and computationally tractable Whittleindex expression In such a way to derive a closed-formcharacterization of the Whittle index of (119886 119889 119899) state first ofall it is necessary to specify the activity of (119886 119889 + 1119898) and(119886 + 119903

119899 119889 + 1119898) states As concluded from Whittle index

properties it is known that (119886 119889+1119898 le 119899)cup(119886+119903119899 119889+1119898 le

119899) states are passive However it is not trivial to specify abovewhich attained service level andor delay level if (119886 119889+1119898 gt

119899)cup (119886+ 119903119899 119889 + 1119898 gt 119899) states with better channel condition

are active or passiveNonetheless in the best channel the active set is totally

specified since all the states that compute in the index are pas-sive because of nonexisting channel improvement and beingin the future attained service equal or higher and delay higherAs shown in Proposition 3 for the best channel condition theWhittle index has a closed-form expression where the indexvalue is the multiplication between completion probabilityand normalizedMOS function Note that the obtainedWhit-tle index expression in the best channel state is equivalent tothe GM index achieved for constant channel [16]

Proposition 3 The Whittle index for Problem (8) in the bestchannel condition is given by

Vlowast(119886119889119873)

= 120583(119886119873)

sdot MOS (119889 + 1) (13)

Proof It is known that for Vlowast(119886119889119873)

index computation thatF = 0 In this way using (11) and (12) RF

(119886119889+1119898)=

120573sum119898isinN 119902

119898RF(119886119889+2119898)

RF(119886119889+2119898)

= 120573sum119898isinN 119902

119898RF(119886119889+3119898)

RF(119886119889gt119889max 119898)

= 0 Hence analogolously RF(119886+119903119899 119889+1119898)

= 0WF(119886119889+1119898)

= 0 and WF(119886+119903119899 119889+1119898)

= 0 Substituting thesenull terms in expression (10) and being 120573 = 1 we achieve(13)

Besides as stated before obtaining a computationallytractable Whittle index expression for any channel conditionis not possible In such situation in order to achieve atractable Whittle-based QoE-aware index rule for time-varying channels we propose a Whittle-based approximateheuristic in the next subsection

41 Approximate Whittle Index Proposal In the following wepropose an approximate solution for the QoE-aware Whittleindex in a time-varying channel context (10)

First of all we will determine the structure of the activeset For the Whittle index computation of (119886 119889 119899) state aspreviously concluded it is known that (119886 119889 + 1119898 le 119899) cup (119886+

119903119899 119889 + 1119898 le 119899) states are passive but we need to define the

unknown activity of (119886 119889 + 1119898 gt 119899) cup (119886 + 119903119899 119889 + 1119898 gt 119899)

states Thus so as to determine if these states are active orpassive we apply the following two considerations

(1) 120583 approximation

120583(119886119898)

asymp 120583(119886+119903119897 119898)

(14)

(2) MOS approximation

MOS (119889) asymp MOS (119889 + 1) (15)

On the one hand we assume that for any 119897 = 1 2 119873

value that 120583(119886119898)

asymp 120583(119886+119903119897 119898)

(14) It is known that (119886 119889119898 gt

119899) states are active and being 120583(119886119898)

asymp 120583(119886+119903119899 119898)

we couldconsider that (119886 + 119903

119899 119889 119898 gt 119899) states are active

On the other hand we assume MOS(119889) asymp MOS(119889 + 1)(15) since QoE degradation in a higher delay level is neg-ligible in the millisecond scale that delay evolves Takinginto account the approximation about MOS function being(119886 119889119898 gt 119899) states active we can suppose that (119886 119889 + 1119898 gt

119899) states are also active This way combining approximations(14) and (15) we consider that (119886 + 119903

119899 119889 + 1119898 gt 119899) states are

activeThen suppose that the aforementioned 120583 and

MOS approximations cause 119877119886119889

= sum119898119902119898119877(119886119889119898)

asymp

sum119898119902119898119877(119886+119903119897 119889+1119898)

and 119882119886119889

= sum119898119902119898119882(119886119889119898)

asymp

sum119898119902119898119882(119886+119903119897 119889+1119898)

So using these reward and workapproximations the first simplified expression for theWhittle index formulation (10) is

Vlowast(119886119889119899)

=120573120583(119886119899)

(MOS (119889 + 1) minus 119877119886119889)

1 minus 120573120583(119886119899)

119882119886119889

(16)

In such a way applying the previous simplificationswe obtain the Whittle index approximation presentedin Proposition 4 what we call attained service PI MOS(ASPIM) The methodology to compute the closed-formexpression of this QoE-aware Whittle-based index is pre-sented in the Appendix

Proposition 4 The formulation of the ASPIM index is

119860119878119875119868119872 = Vlowast(119886119889119899)

= lim120573rarr1

120573120583(119886119899)

sdot MOS (119889 + 1) (1 minus 120573)1 minus 120573 + 120573sum

119898gt119899119902119898(120583(119886119898)

minus 120583(119886119899)

)

(17)

8 Abstract and Applied Analysis

Table 1 CQIs and corresponding transmission rates (Mbps)

CQI 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15119904 0 42 672 84 11256 168 2184 252 2688 336 4468 504 5376 672 758 8064

If we analyze expression (17) we observe that this QoE-aware and channel-aware Whittle-based ASPIM index isimplementable This proposal depends on QoE size andchannel properties Moreover the obtained index for thebest channel condition is equal to the original Whittle index(13) since the summation in the denominator becomes nullHowever as shown in (18) when channel state is not the bestthe index value goes to zero

ASPIM119899 =119873

= lim120573rarr1

120583(119886119899)

sdot MOS (119889 + 1) (1 minus 120573)sum119898gt119899

119902119898(120583(119886119898)

minus 120583(119886119899)

)997888rarr 0

(18)

Hence a user in its best channel state with nonnullinstantaneous normalized MOS function has priority overa user which is not in its best channel condition Thus wesummarize the proposed QoE-aware Whittle-based ASPIMindex rule in Definition 5

Definition 5 The ASPIM index rule consists of at everydecision slot

(i) serving the user in its best channel condition withnonnull instantaneous normalized MOS functionwith the highest value of 120583

(119886119873)sdot MOS(119889 + 1)

(ii) if there is no user in its best channel condition withnonnull instantaneous normalized MOS functionserving the user with the highest value of (120583

(119886119899)sdot

MOS(119889 + 1))sum119898gt119899

119902119898(120583(119886119898)

minus 120583(119886119899)

) (using (18)normalized by (1 minus 120573))

So if channel condition is not the best the index valueis the ratio of the multiplication between the actual servicecompletion and the instantaneous normalizedMOS functionrespect to the expected potential improvement of the comple-tion probability

Therefore in the context of time-varying channels aimedat maximizing average MOS we have proposed ASPIMscheduling algorithm which is a simple and tractable QoE-aware channel-aware and size-based index rule Neverthe-less in order to verify the correct behavior of this heuristicwe analyze its performance in the subsequent section

5 Performance Evaluation

In this section we evaluate the performance of the proposedASPIM index rule (see proposal in Definition 5) To that endwe compare the QoE performance of our heuristic with theone achieved by well-known scheduling strategies in severalsimulation scenarios Below we provide a brief description ofthe scheduling priority rules used in our experiments

(i) Max rate (MR) this opportunistic policy consists inserving the user with the best channel capacity

(ii) Proportional fair (PF) this channel-aware disci-pline serves the user with the highest ratio betweeninstantaneous transmission rate and current attainedthroughput

(iii) Cost and attained service dependant 120583 (119888AS120583)although it might seem counter-intuitive minimizingmean delay does not result in maximizing averageMOS [24] This way in order to compare the per-formance with delay-aware solutions we propose toadapt the classical 119888120583 rule aimed at minimizing meandelay [25] to attained serviceThis adaptation consistsof serving the user with the highest value of 119888120583

(119886119899)

being 119888 the waiting cost paid for every slot while flowtransmission is uncompleted

(iv) GM [16] thisQoE-aware pioneering index rule origi-nally proposed for constant channel capacity choosesthe user with the highest value of 120583

(119886119899)sdot MOS(119889 + 1)

In case of ties we use random tie-breaking ruleConcerning traffic flows in the system we use the Pareto

size distribution given in (1) in simulations whose 120572 and 120574parameters determine mean flow size We consider typicalmean sizes used in wireless networks 05Mb (small traffica web page email) 5Mb (medium traffic a PDF documenta picture etc) and 50Mb (MP3 audio a group of picturesfrom a video sequence etc) User traffic flows arrive to thescheduler according to a Poisson process with 120582 rate

Referring to network characteristics we use transmis-sion rates that depend on CQIs used in real wirelessnetworks which are associated with 4G modulation andcoding schemes We show the mapping between CQI valuesand transmission rates in Table 1 The presented values areadapted and extrapolated from [26] for a cell of 20MHz

Moreover we consider different network loads in order toanalyze the behavior of scheduling strategies under differentnetwork conditions Flow arrival rate determines networkload 120588 where 120588 = 120582 sdot (E[119883]119904

119873) We consider seven network

states in this study low load (120588 = 025 0375) medium load(120588 = 05 075) and high load (120588 = 085 09 095)

Regarding QoE aspects as typically utilized in perceivedquality studies [27] we use a log-shaped MOS function forQoE degradation presented in (19) This way employing theMOS curve modelling provided in [21] as starting point andtaking into account the flow size the network type and theuser expertise we have suitably defined 119889min and 119889max delaythresholds used in simulations and consequently 119888

1and 1198882

MOS coefficients Consider

MOS = 1198881minus 1198882sdot log (119889) (19)

We have implemented the whole network environmentfor the scheduling of network resources in MATLAB Inrelation to the performed experiments for each scenario

Abstract and Applied Analysis 9

Table 2 Parameter set in the experimental scenarios

Scenario MOS Size Channel 119888

1 119889min = 05 s 119889max = 10 s 120572 = 15 120574 = 4 sdot 10minus7 CQI = 3 5 11198881= 407 119888

2= 133 E[119883] = 5Mbit 119902

2= 05

2 119889min = 04 s 119889max = 06 s 120572 = 15 120574 = 4 sdot 10minus7 CQI = 3 5 11198881= minus403 119888

2= 986 E[119883] = 5Mbit 119902

2= 05

3 119889min = 05 s 119889max = 10 s 120572 = 15 120574 = 4 sdot 10minus7 CQI = 3 9 11198881= 407 119888

2= 133 E[119883] = 5Mbit 119902

2= 05

4 119889min = 005 s 119889max = 1 s 120572 = 15 120574 = 4 sdot 10minus6 CQI = 3 5 11198881= 1 1198882= 133 E[119883] = 05Mbit 119902

2= 05

5

119889min1 = 05 s 119889max1 = 5 s 1205721= 15 120574

1= 4 sdot 10

minus7

see Table 3 5 111988811= 379 119888

21= 173 E[119883

1] = 5Mbit

119889min2 = 5 s 119889max2 = 30 s 1205722= 15 120574

2= 4 sdot 10

minus8

11988812= 859 119888

22= 223 E[119883

2] = 50Mbit

Table 3 Channel state probabilities in Scenario 5

CQI 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15119902 028 012 009 008 008 008 007 006 005 004 003 001 0009 00005 00003 00002

for each combination of scheduling discipline and networkload we have carried out a set of 10 rounds of 10000 slength simulations These rounds differ in the randomlypregenerated traces (input vector) of sizes and arrivals Thusnot only the overall average MOS values will be provided butwe will also include their 95 confidence intervals

Besides in order to guarantee that the obtained perfor-mance results are generally valid we will analyze differentsimulation scenarios We have chosen five relevant settingswhich differ inQoE size channel orand cost characteristicsThe parameters of these scenarios are summarized in Table 2Note that in the first four scenarios we restrict ourselves tothe case of two channel conditions so as to get fundamentalperformance results

We have carefully selected the scenarios used in simula-tions First of all we define a setting (Scenario 1) in which thevalues of all the parameters are typical or standard the valuesof these parameters introduce low error in the approxima-tions considered for our ASPIM scheduler proposal Thenin order to evaluate worst performance cases for ASPIMwe have chosen three scenarios (Scenario 2 Scenario 3 andScenario 4) in which the error of the approximation used forASPIM is high due to a different factor (gradient ofMOS rateand size resp)

Apart from that it is worthmentioning that for obtainingthe ASPIM index rule that we have considered an isolatedMDP model per user Hence the achieved solution is basedon a unique user type or single class However it would beinteresting to examine the proposal when there are differenttypes of users which is analyzed in Scenario 5 Furthermorethis last setting resembles a real 4G wireless network Notethat even though this technology allows the simultaneoustransmission of multiple flows per TTI we assume that asingle flow transmits in each TTI

03 04 05 06 07 08 09

25

3

35

4

45

5

E[M

OS]

ASPIMMRPF

cAS120583GM

120588

Figure 5 E[MOS] for the basic scenario

Next we describe the results we have achieved in theaforementioned scenarios

51 Scenario 1 Basic Case In this first family of simulationswe consider a basic or typical scenario which takes intoaccount the equiprobable channel case and medium-sizedself-similar flows Figure 5 shows averageMOS results for thissetting employing different scheduling policies and networkloadsThemain conclusion achieved from these results is thatASPIM outperforms the rest of disciplines which is betterthan the QoE-aware GM rule proposed for constant channelcapacity Moreover contrary to the other policies it holds

10 Abstract and Applied Analysis

0 02 04 06 08 10

1

2

3

4

d (s)

PDF(d

)

(a)

1 2 3 4 50

01

02

03

04

05

06

07

MOS

PDF(

MO

S)

ASPIMMRPF

cAS120583GM

(b)

1 2 3 4 50

02

04

06

08

1

MOS

CCD

F(M

OS)

ASPIMMRPF

cAS120583GM

Satisth

(c)

Figure 6 Statistics in Scenario 1 delay PDF (a) MOS PDF (b) and MOS CCDF (c)

averageMOSvalues in the same level for all120588 Referring to thetraditional channel-aware disciplines both PF and MR givebad behavior comparing to the rest of scheduling strategies

In order to justify the previous performance results wenow analyze delay andMOS statistics for the highest networkload considered In such a way as regarded in Figure 6(a) forthe Probability Density Function (PDF) of delay we highlightthe probabilitymass or peak that appears for some schedulingdisciplines before the low delay threshold (05 s) The heightand weight of a peak belonging to a policy have direct impacton its average MOS ASPIM discipline shows the highestand the widest peak among the presented ones followedby GM but with a notable difference Therefore our novelQoE-aware policy seems to try to finish the transmissionof most of the flows before 119889min which is reflected in theaforementioned peaks Note that the cAS120583 rule aimed atminimizing delay behaves in different manner which showsmore fairness among delays in the range shown compared tothe QoE-aware ones

What is previously concluded about delay statistics hasa direct reflection on QoE statistics This way as can beseen in the PDF of MOS presented in Figure 6(b) if theobjective is to maximize average QoE it is important onthe one hand to be the height and the weight of the peaksaround the maximum MOS high and on the other hand tobe the value around the minimum MOS low ASPIM fulfilsthese positive properties and consequently it is the bestin average QoE terms Besides if we analyze the Comple-mentary Cumulative Distribution Function (CCDF) of MOSprovided in Figure 6(c) we observe that ASPIM gives thehighest probability of being MOS value higher Consideringfor example a QoE satisfaction threshold satisth of MOS =4 in this point ASPIM improves GM a 10 whereas thedeterioration of MR and PF is higher than a 50

52 Scenario 2 High Demanding Users Case This scenarioreflects the case of users with low tolerance to delay in

Abstract and Applied Analysis 11

03 04 05 06 07 08 09

05

06

07

08

09

ASPIMMRPF

cAS120583GM

120588

E[M

OS]

2E

[MO

S]1

Figure 7 E[MOS] for abrupt MOS scenario

which theMOS function in theQoE degradation range showsa great slope Thus in this setting the error of the MOSapproximation (15) is high since the gradient of MOS amongconsecutive delay levels becomes significant In Figure 7we compare the results of this second scenario and thebasic scenario From the illustrated graph we could say thatour QoE-aware proposal adapts to any MOS function Thisis whereas for the rest of disciplines (excluding GM) thenegative effects due to the abrupt degradation of the QoEutility function are appreciable ASPIMmanages to avoid thisadapting to any MOS function

53 Scenario 3 Higher Error Case due to Rate During thesimplification process carried out for obtaining the ASPIMindex rule in Section 41 the 120583 approximation used (see (14))introduces a rate dependant error Note that increasing therate in the good channel state makes bigger the error of the 120583approximation used Therefore we will analyze the obtainedperformance for our new QoE-aware proposal under thishigher error due to rate In the previous scenarios we haveassumed that the transmission rate in the good channelcondition is twice the rate in the bad channel state In thissetting we consider that the transmission rate in the goodchannel is four times the transmission rate in the bad channelThis way according to the QoE results presented in Figure 8in spite of a higher error caused by rate ASPIM is superior tothe rest of policies

54 Scenario 4 High Error Case due to Size Besides it isknown that for a distribution with a decreasing hazard ratesuch as Pareto that the gradient of completion probabilityincreases as long as the mean size decreases which makesthe error of the 120583 approximation (14) for the ASPIM ruleachievement bigger In this way in this scenario we verifythat ASPIM gives satisfactory behavior under a more notableerror due to size where we consider a mean size ten times

03 04 05 06 07 08 09

3

35

4

45

5

E[M

OS]

ASPIMMRPF

cAS120583GM

120588

Figure 8 E[MOS] under higher error due to rate

03 04 05 06 07 08 093

35

4

45

5E

[MO

S]

ASPIMMRPF

cAS120583GM

120588

Figure 9 E[MOS] under high error due to size

lower We conclude from Figure 9 that even under a higherror caused by size that ASPIM shows good performancebeing the best alternative

55 Scenario 5 A Real Network Case This last scenarioreflects a real 4G wireless network context but with thesimplification that a single user transmits in each TTI To thatend we use CQI traces obtained from a system-level radioaccess simulator [28] considering users moving at 5 kmhin the cell For this setting with dynamic users we definechannel propagation by the Extended Pedestrian Amodel Asin current 4G networks we consider 16 channel conditionsThe state probabilities of these channel states are collectedin Table 3 which are decreasing with the improvement ofchannel quality In this scenario we consider two classes ofusers which differ in size QoE and cost The mean size

12 Abstract and Applied Analysis

03 04 05 06 07 08 09

1

2

3

4

5

E[M

OS]

120588

(a)

03 04 05 06 07 08 091

2

3

4

5

E[M

OS]

ASPIMMRPF

cAS120583GM

120588

(b)

03 04 05 06 07 08 091

2

3

4

5

E[M

OS]

ASPIMMRPF

cAS120583GM

120588

(c)

Figure 10 E[MOS] for 4G scenario aggregate (a) class 1 (b) class 2 (c)

in class 2 is ten times bigger than in class 1 and we givepriority to the small-sized class by having bigger cost andQoErequirement

Under this mixture of classes and realistic channel mod-els in Figure 10(a) we observe that scheduling strategiesshow the same tendency as in the previous scenarios It isremarkable that the difference between ASPIM and GM isbigger especially for high 120588 values Moreover under thehighest network load ASPIM achieves MOS values aboveQoE satisfaction threshold (MOS = 4) Furthermore averageMOS values forMR and PF dramatically decrease comparingwith the previous settings

Concerning class behaviour we show per-class resultsin Figures 10(b) and 10(c) As can be concluded fromthe presented results QoE-aware policies guarantee fairnessamong classes and ASPIM is the best choice also inside classAside from that for the cost dependant cAS120583 discipline thedeterioration of the class with no priority is significant

6 Conclusions

This paper goes into detail about obtaining a schedulingalgorithmaimed atmaximizing usersrsquo perception of quality inchannels with time-varying capacity Furthermore this workconsiders traffic flowswith nonexponential size distributionscontrary to previous approaches that assume memorylessdistributions

As first contribution we provide a MDP-based modelfor the scheduling problem presented This model combinesQoE-awareness size-awareness and channel-awarenessNev-ertheless since this model can not be solved either analyt-ically or computationally from the proposed MDP modelwe derive a simple tractable and implementable well-performing scheduling index rule by applying amethodologybased on Whittle approach

The proposed scheduling solution still combines QoE-size- and channel-awareness Our scheduling proposal givespriority to users in their best channel condition that are

Abstract and Applied Analysis 13

out of the service unavoidable range As verified fromsimulation results the proposed scheduling strategy showssuitable subjective quality performance in a wide range ofscenarios including the case of a simplified 4G network withheterogeneous users

Therefore the results of this work present a relevantmathematical basis for developing scheduling algorithmsaimed at maximizing QoE in current and future networkswith time-varying channel capacity This way the proposedchannel-aware ASPIM discipline will be useful for networkoperators in order to guarantee to their customers servicesatisfaction in time-varying wireless networks Apart fromthat the approximations and simplifications used in Whittlemethod are applicable to any area when the gradient ofsystem components is considerably small among consecutivedecision slots

As future research we will extend our work to amultiuserapproach considering the simultaneous transmission ofmul-tiple flows per TTI

Appendix

Proof of Proposition 4

LemmaA1 Suppose that states (119886 119889+1119898 le 119899)cup(119886+119903119899 119889+

1119898 le 119899) are passive and that states (119886 119889 + 1119898 gt 119899) cup (119886 +

119903119899 119889 + 1119898 gt 119899) are active and

Vlowast(119886119889119899)

=120573120583(119886119899)

( MOS (119889 + 1) minus 119877119886119889)

1 minus 120573120583(119886119899)

119882119886119889

(A1)

Along with this if we assume that 119877119886119889

= sum119898119902119898119877(119886119889119898)

asymp

sum119898119902119898119877(119886+119903119897 119889+1119898)

and 119882119886119889

= sum119898119902119898119882(119886119889119898)

asymp

sum119898119902119898119882(119886+119903119897 119889+1119898)

for any 119897 = 1 2 119873 value for theundiscounted case

Vlowast(119886119889119899)

= lim120573rarr1

120573120583(119886119899)

sdot MOS (119889 + 1) (1 minus 120573)1 minus 120573 + 120573sum

119898gt119899119902119898(120583(119886119898)

minus 120583(119886119899)

) (A2)

Proof Referring to work elements using (12)

119882(119886119889119898le119899)

= 120573sum

119898

1199021015840

119898119882(119886119889+1119898

1015840)= 120573119882

119886119889

119882(119886119889119898gt119899)

= 1 + 120573 (1 minus 120583(119886119898)

)sum

119898

1199021015840

119898119882(119886+119903119898 119889+1119898

1015840)

= 1 + 120573 (1 minus 120583(119886119898)

)119882119886119889

(A3)

And by (A3) expressions

119882119886119889= (1 minus sum

119898gt119899

119902119898)120573119882

119886119889+ sum

119898gt119899

119902119898(1 + 120573 (1 minus 120583

(119886119898))119882119886119889)

(A4)

This way isolating119882119886119889

119882119886119889=

sum119898gt119899

119902119898

1 minus 120573 + 120573sum119898gt119899

119902119898120583(119886119898)

(A5)

Analogously for reward elements using (11)

119877(119886119889119898le119899)

= 120573sum

119898

1199021015840

119898119877(119886119889+1119898

1015840)= 120573119877119886119889

119877(119886119889119898gt119899)

= 120573[(1 minus 120583(119886119898)

)sum

119898

1199021015840

119898119877(119886+119903119898 119898

1015840)+ 120583(119886119898)

sdot MOS (119889 + 1)]

= 120573 [(1 minus 120583(119886119898)

) 119877119886119889+ 120583(119886119898)

sdot MOS (119889 + 1)] (A6)

And thus from (A6)

119877119886119889= (1 minus sum

119898gt119899

119902119898)120573119877119886119889

+ sum

119898gt119899

119902119898(120573 [(1 minus 120583

(119886119898)) 119877119886119889+ 120583(119886119898)

sdot MOS (119889 + 1)])

(A7)

Isolating 119877119886119889

119877119886119889=120573sum119898gt119899

119902119898120583(119886119898)

MOS (119889 + 1)1 minus 120573 + 120573sum

119898gt119899119902119898120583(119886119898)

(A8)

Substituting (A5) and (A8) in (A1) and by simplifyingwe obtain

Vlowast(119886119889119899)

= (120573120583(119886119899)

(MOS (119889 + 1)

minus120573sum119898gt119899

119902119898120583(119886119898)

MOS (119889 + 1)1 minus 120573 + 120573sum

119898gt119899119902119898120583(119886119898)

))

times (1 minus 120573120583(119886119899)

sum119898gt119899

119902119898

1 minus 120573 + 120573sum119898gt119899

119902119898120583(119886119898)

)

minus1

= (120573120583(119886119899)

(MOS (119889 + 1) (1 minus 120573 + 120573sum119898gt119899

119902119898120583(119886119898)

)

minus120573sum

119898gt119899

119902119898120583(119886119898)

MOS (119889 + 1)))

times (1 minus 120573 + 120573sum

119898gt119899

119902119898120583(119886119898)

minus 120573120583(119886119899)

sum

119898gt119899

119902119898)

minus1

=120573120583(119886119899)

sdot MOS (119889 + 1) (1 minus 120573)1 minus 120573 + 120573sum

119898gt119899119902119898(120583(119886119898)

minus 120583(119886119899)

)

(A9)

And therefore for the undiscounted case the 120573 rarr 1

limit for expression (A9) is (A2)

Conflict of Interests

The authors declare that there is no conflict of interestsregarding the publication of this paper

14 Abstract and Applied Analysis

Acknowledgment

The research leading to these results has received fundingfrom the European Union Seventh Framework Programme(FP72007ndash2013) under Grant agreement 284863 (FP7 SECGERYON)

References

[1] ITU-T P10G100 Amendment 1 (0107) New Appendix I-definition of Quality of Experience (QoE) 2006

[2] S Aalto and P Lassila ldquoFlow-level stability and performance ofchannel-aware priority-based schedulersrdquo in Proceeedings of the6th Euro NF Conference on Next Generation Internet (NGI rsquo10)IEEE June 2010

[3] L KleinrockQueueing Systems vol 2 ofComputer ApplicationsJohn Wiley amp Sons 1976

[4] J Gittins ldquoBandit processes and dynamic allocation indicesrdquoJournal of the Royal Statistical Society B Methodological vol 41no 2 pp 148ndash177 1979

[5] K Avrachenkovt U Ayesta P Brown and E Nyberg ldquoDif-ferentiation between short and long TCP ows predictabilityof the response timerdquo in Proceedings of the 23rd Annual JointConference of the IEEE Computer and Communications Societies(INFOCOM rsquo04) vol 2 p 762 IEEE 2004

[6] U Ayesta M Erausquin and P Jacko ldquoA modeling frameworkfor optimizing the flow-level scheduling with time-varyingchannelsrdquo Performance Evaluation vol 67 no 11 pp 1014ndash10292010

[7] S Aalto U Ayesta and R Righter ldquoProperties of the Gittinsindex with application to optimal schedulingrdquo Probability in theEngineering and Informational Sciences vol 25 no 3 pp 269ndash288 2011

[8] M Puterman ldquoMarkov decision processesrdquo in Handbooks inOperations Research and Management Science vol 2 pp 331ndash434 1990

[9] C H Papadimitriou and J N Tsitsiklis ldquoThe complexityof Markov decision processesrdquo Mathematics of OperationsResearch vol 12 no 3 pp 441ndash450 1987

[10] J Nino-Mora ldquoDynamic priority allocation via restless banditmarginal productivity indicesrdquo TOP vol 15 no 2 pp 161ndash1982007

[11] P Whittle ldquoRestless bandits activity allocation in a changingworldrdquo Journal of Applied Probability vol 25 pp 287ndash298 1988

[12] U Ayesta M Erausquin M Jonckheere and I M VerloopldquoScheduling in a random environment stability and asymptoticoptimalityrdquo IEEEACM Transactions on Networking vol 21 no1 pp 258ndash271 2013

[13] S Khan S Thakolsri E Steinbach and W Kellerer ldquoQoE-based cross-layer optimization for wireless multiuser systemsrdquoin Proceedings of the 18th ITC Specialist Seminar on Quality ofExperience 2008

[14] S Thakolsri S Khan E Steinbach and W Kellerer ldquoQoE-driven cross-layer optimization for high speed downlink packetaccessrdquo Journal of Communications vol 4 no 9 pp 669ndash6802009

[15] P Ameigeiras J J Ramos-Munoz J Navarro-Ortiz PMogensen and J M Lopez-Soler ldquoQoE oriented cross-layer design of a resource allocation algorithm in beyond 3GsystemsrdquoComputer Communications vol 33 no 5 pp 571ndash5822010

[16] I Taboada F Liberal J O Fajardo and U Ayesta ldquoQoE-aware optimization of multimedia flow schedulingrdquo ComputerCommunications vol 36 no 15-16 pp 1629ndash1638 2013

[17] U Ayesta M Erausquin and P Jacko ldquoResource-sharing in asingle server with time-varying capacityrdquo in Proceeding of the49th Annual Allerton Conference on Communication Controland Computing (Allerton rsquo11) pp 377ndash383 Monticello Ill USASeptember 2011

[18] K Thompson G J Miller and R Wilder ldquoWide-area internettraffic patterns and characteristicsrdquo IEEE Network vol 11 no 6pp 10ndash23 1997

[19] M Fiedler T Hossfeld and P Tran-Gia ldquoA generic quantitativerelationship between quality of experience and quality ofservicerdquo IEEE Network vol 24 no 2 pp 36ndash41 2010

[20] A Richards M Antoniades V Witana and G Rogers ldquoMap-ping user level QoS from a single parameterrdquo in Proceedingsof the International Conference on Multimedia Networks andServices Citeseer 1998

[21] E Ibarrola F Liberal I Taboada and R Ortega ldquoWebQoE evaluation in multi-agent networks validation of ITU-TG1030rdquo in Proceedings of the 5th International Conference onAutonomic and Autonomous Systems (ICAS rsquo09) pp 289ndash294Valencia Spain April 2009

[22] S Aalto and U Ayesta ldquoOptimal scheduling of jobs with aDHR tail in the MG1 queuerdquo in Proceedings of the 3rd Inter-national Conference on Performance Evaluation Methodologiesand Tools (ValueTools rsquo08) article 50 Institute for ComputerSciences Social-Informatics and Telecommunications Engi-neering Athens Greece 2008

[23] V Visweswaran ldquoDecomposition techniques for MILPlagrangian relaxationrdquo in Encyclopedia of Optimization pp632ndash638 Springer New York NY USA 2009

[24] I Taboada J O Fajardo and F Liberal ldquoPerformance analysisof scheduling algorithms for web QoE optimization in wirelessnetworksrdquo Network Protocols and Algorithms vol 4 pp 27ndash432012

[25] J A van Mieghem ldquoDynamic scheduling with convex delaycosts the generalized 119888120583 rulerdquoTheAnnals of Applied Probabilityvol 5 no 3 pp 809ndash833 1995

[26] S Sesia I Toufik and M Baker LTE-The UMTS Long TermEvolution FromTheory to Practice Wiley 2011

[27] P Reichl B Tuffin and R Schatz ldquoLogarithmic laws inservice quality perception where microeconomics meets psy-chophysics and quality of experiencerdquo Telecommunication Sys-tems vol 52 no 2 pp 587ndash600 2013

[28] J C Ikuno M Wrulich and M Rupp ldquoSystem level simulationof LTE networksrdquo in Proceedings of the IEEE 71st VehicularTechnology Conference Taipei Taiwan May 2010

Submit your manuscripts athttpwwwhindawicom

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical Problems in Engineering

Hindawi Publishing Corporationhttpwwwhindawicom

Differential EquationsInternational Journal of

Volume 2014

Applied MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical PhysicsAdvances in

Complex AnalysisJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

OptimizationJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Operations ResearchAdvances in

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Function Spaces

Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of Mathematics and Mathematical Sciences

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Algebra

Discrete Dynamics in Nature and Society

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Decision SciencesAdvances in

Discrete MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Stochastic AnalysisInternational Journal of

6 Abstract and Applied Analysis

times10minus3

6

5

4

3

2

1

00

0

5 5

10

10times107

ad

Vlowast (ad1)

(a)

0

0

5 5

10

10times107

a d

Vlowast (ad2)

001

0008

0006

0004

0002

0

(b)

Figure 3 Whittle index values obtained from AG-algorithm for two channel conditions and different attained services and delays For aPareto distribution with 120572 = 15 and E[119883] = 5Mb 119902

2= 05 119903

1= 84Kb 119903

2= 168Kb 119889min = 2 s and 119889max = 9 s

0

0

5 5

10

10

times107

ad

Vlowast (ad1)

001

0008

0006

0004

0002

0

q2 = 01

(a)

times10minus3

4

3

2

1

0

0

0

5 5

10

10

times107

a d

Vlowast (ad1)

q2 = 09

(b)

Figure 4 Whittle index values obtained fromAG-algorithm for different channel state probabilities For a Pareto distribution with 120572 = 15and E[119883] = 5Mb 119903

1= 84Kb 119903

2= 168Kb 119889min = 2 s and 119889max = 9 s

in the scenarios In such a way from experimental results weconjecture the following fundamental properties

(1) TheWhittle index value depends on attained servicedelay and channel state Moreover as can be seenin graphs from Figures 3 and 4 we can clearlydistinguish the ranges of MOS function above theupper delay threshold the index value is null

(2) For the same channel condition and the samedelay level Whittle index values are decreasing withattained service Vlowast

(119886119889119898)gt Vlowast(1198861015840119889119898)

forall1198861015840

gt 119886 (illus-trated in Figures 3 and 4)

(3) For the same channel condition and the same attainedservice levelWhittle index values are decreasing withdelay Vlowast

(119886119889119898)gt Vlowast(119886119889+1119898)

(See Figures 3 and 4)

Abstract and Applied Analysis 7

(4) In a better channel condition for the same attainedservice level and the same delay level the Whittleindex value is higher Vlowast

(1198861198891198981)gt Vlowast(1198861198891198982)

forall1198981gt 1198982

(Observe Figure 3)(5) Opposite to the previous channel-aware work [6]

where Whittle indices in the best channel conditionare infinite having absolute priority in this channelstate in our case being in the best channel conditiondoes not guarantee that the Whittle index value ishigher than in the rest of the channel conditions it ispossible to fulfil Vlowast

(119886119889119898lt119873)gt Vlowast(11988610158401198891015840119873)

If we comparethe two graphs in Figure 3 it can be easily seen howfor several attained services andor delayed values theWhittle index values in the worst channel conditionare higher than in the best channel condition (takingalso into account nonnull values)

(6) Dependency on channel condition probabilities hilefor channel condition 119873 the Whittle index valueremains invariant with channel state probability forthe rest of channel conditions its value decreases aslong as channel state probabilities increase in betterchannel conditions (shown in Figure 4)

Once we analyzed the properties of theWhittle index weset out to define the structure of the active set in order toobtain an analytically and computationally tractable Whittleindex expression In such a way to derive a closed-formcharacterization of the Whittle index of (119886 119889 119899) state first ofall it is necessary to specify the activity of (119886 119889 + 1119898) and(119886 + 119903

119899 119889 + 1119898) states As concluded from Whittle index

properties it is known that (119886 119889+1119898 le 119899)cup(119886+119903119899 119889+1119898 le

119899) states are passive However it is not trivial to specify abovewhich attained service level andor delay level if (119886 119889+1119898 gt

119899)cup (119886+ 119903119899 119889 + 1119898 gt 119899) states with better channel condition

are active or passiveNonetheless in the best channel the active set is totally

specified since all the states that compute in the index are pas-sive because of nonexisting channel improvement and beingin the future attained service equal or higher and delay higherAs shown in Proposition 3 for the best channel condition theWhittle index has a closed-form expression where the indexvalue is the multiplication between completion probabilityand normalizedMOS function Note that the obtainedWhit-tle index expression in the best channel state is equivalent tothe GM index achieved for constant channel [16]

Proposition 3 The Whittle index for Problem (8) in the bestchannel condition is given by

Vlowast(119886119889119873)

= 120583(119886119873)

sdot MOS (119889 + 1) (13)

Proof It is known that for Vlowast(119886119889119873)

index computation thatF = 0 In this way using (11) and (12) RF

(119886119889+1119898)=

120573sum119898isinN 119902

119898RF(119886119889+2119898)

RF(119886119889+2119898)

= 120573sum119898isinN 119902

119898RF(119886119889+3119898)

RF(119886119889gt119889max 119898)

= 0 Hence analogolously RF(119886+119903119899 119889+1119898)

= 0WF(119886119889+1119898)

= 0 and WF(119886+119903119899 119889+1119898)

= 0 Substituting thesenull terms in expression (10) and being 120573 = 1 we achieve(13)

Besides as stated before obtaining a computationallytractable Whittle index expression for any channel conditionis not possible In such situation in order to achieve atractable Whittle-based QoE-aware index rule for time-varying channels we propose a Whittle-based approximateheuristic in the next subsection

41 Approximate Whittle Index Proposal In the following wepropose an approximate solution for the QoE-aware Whittleindex in a time-varying channel context (10)

First of all we will determine the structure of the activeset For the Whittle index computation of (119886 119889 119899) state aspreviously concluded it is known that (119886 119889 + 1119898 le 119899) cup (119886+

119903119899 119889 + 1119898 le 119899) states are passive but we need to define the

unknown activity of (119886 119889 + 1119898 gt 119899) cup (119886 + 119903119899 119889 + 1119898 gt 119899)

states Thus so as to determine if these states are active orpassive we apply the following two considerations

(1) 120583 approximation

120583(119886119898)

asymp 120583(119886+119903119897 119898)

(14)

(2) MOS approximation

MOS (119889) asymp MOS (119889 + 1) (15)

On the one hand we assume that for any 119897 = 1 2 119873

value that 120583(119886119898)

asymp 120583(119886+119903119897 119898)

(14) It is known that (119886 119889119898 gt

119899) states are active and being 120583(119886119898)

asymp 120583(119886+119903119899 119898)

we couldconsider that (119886 + 119903

119899 119889 119898 gt 119899) states are active

On the other hand we assume MOS(119889) asymp MOS(119889 + 1)(15) since QoE degradation in a higher delay level is neg-ligible in the millisecond scale that delay evolves Takinginto account the approximation about MOS function being(119886 119889119898 gt 119899) states active we can suppose that (119886 119889 + 1119898 gt

119899) states are also active This way combining approximations(14) and (15) we consider that (119886 + 119903

119899 119889 + 1119898 gt 119899) states are

activeThen suppose that the aforementioned 120583 and

MOS approximations cause 119877119886119889

= sum119898119902119898119877(119886119889119898)

asymp

sum119898119902119898119877(119886+119903119897 119889+1119898)

and 119882119886119889

= sum119898119902119898119882(119886119889119898)

asymp

sum119898119902119898119882(119886+119903119897 119889+1119898)

So using these reward and workapproximations the first simplified expression for theWhittle index formulation (10) is

Vlowast(119886119889119899)

=120573120583(119886119899)

(MOS (119889 + 1) minus 119877119886119889)

1 minus 120573120583(119886119899)

119882119886119889

(16)

In such a way applying the previous simplificationswe obtain the Whittle index approximation presentedin Proposition 4 what we call attained service PI MOS(ASPIM) The methodology to compute the closed-formexpression of this QoE-aware Whittle-based index is pre-sented in the Appendix

Proposition 4 The formulation of the ASPIM index is

119860119878119875119868119872 = Vlowast(119886119889119899)

= lim120573rarr1

120573120583(119886119899)

sdot MOS (119889 + 1) (1 minus 120573)1 minus 120573 + 120573sum

119898gt119899119902119898(120583(119886119898)

minus 120583(119886119899)

)

(17)

8 Abstract and Applied Analysis

Table 1 CQIs and corresponding transmission rates (Mbps)

CQI 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15119904 0 42 672 84 11256 168 2184 252 2688 336 4468 504 5376 672 758 8064

If we analyze expression (17) we observe that this QoE-aware and channel-aware Whittle-based ASPIM index isimplementable This proposal depends on QoE size andchannel properties Moreover the obtained index for thebest channel condition is equal to the original Whittle index(13) since the summation in the denominator becomes nullHowever as shown in (18) when channel state is not the bestthe index value goes to zero

ASPIM119899 =119873

= lim120573rarr1

120583(119886119899)

sdot MOS (119889 + 1) (1 minus 120573)sum119898gt119899

119902119898(120583(119886119898)

minus 120583(119886119899)

)997888rarr 0

(18)

Hence a user in its best channel state with nonnullinstantaneous normalized MOS function has priority overa user which is not in its best channel condition Thus wesummarize the proposed QoE-aware Whittle-based ASPIMindex rule in Definition 5

Definition 5 The ASPIM index rule consists of at everydecision slot

(i) serving the user in its best channel condition withnonnull instantaneous normalized MOS functionwith the highest value of 120583

(119886119873)sdot MOS(119889 + 1)

(ii) if there is no user in its best channel condition withnonnull instantaneous normalized MOS functionserving the user with the highest value of (120583

(119886119899)sdot

MOS(119889 + 1))sum119898gt119899

119902119898(120583(119886119898)

minus 120583(119886119899)

) (using (18)normalized by (1 minus 120573))

So if channel condition is not the best the index valueis the ratio of the multiplication between the actual servicecompletion and the instantaneous normalizedMOS functionrespect to the expected potential improvement of the comple-tion probability

Therefore in the context of time-varying channels aimedat maximizing average MOS we have proposed ASPIMscheduling algorithm which is a simple and tractable QoE-aware channel-aware and size-based index rule Neverthe-less in order to verify the correct behavior of this heuristicwe analyze its performance in the subsequent section

5 Performance Evaluation

In this section we evaluate the performance of the proposedASPIM index rule (see proposal in Definition 5) To that endwe compare the QoE performance of our heuristic with theone achieved by well-known scheduling strategies in severalsimulation scenarios Below we provide a brief description ofthe scheduling priority rules used in our experiments

(i) Max rate (MR) this opportunistic policy consists inserving the user with the best channel capacity

(ii) Proportional fair (PF) this channel-aware disci-pline serves the user with the highest ratio betweeninstantaneous transmission rate and current attainedthroughput

(iii) Cost and attained service dependant 120583 (119888AS120583)although it might seem counter-intuitive minimizingmean delay does not result in maximizing averageMOS [24] This way in order to compare the per-formance with delay-aware solutions we propose toadapt the classical 119888120583 rule aimed at minimizing meandelay [25] to attained serviceThis adaptation consistsof serving the user with the highest value of 119888120583

(119886119899)

being 119888 the waiting cost paid for every slot while flowtransmission is uncompleted

(iv) GM [16] thisQoE-aware pioneering index rule origi-nally proposed for constant channel capacity choosesthe user with the highest value of 120583

(119886119899)sdot MOS(119889 + 1)

In case of ties we use random tie-breaking ruleConcerning traffic flows in the system we use the Pareto

size distribution given in (1) in simulations whose 120572 and 120574parameters determine mean flow size We consider typicalmean sizes used in wireless networks 05Mb (small traffica web page email) 5Mb (medium traffic a PDF documenta picture etc) and 50Mb (MP3 audio a group of picturesfrom a video sequence etc) User traffic flows arrive to thescheduler according to a Poisson process with 120582 rate

Referring to network characteristics we use transmis-sion rates that depend on CQIs used in real wirelessnetworks which are associated with 4G modulation andcoding schemes We show the mapping between CQI valuesand transmission rates in Table 1 The presented values areadapted and extrapolated from [26] for a cell of 20MHz

Moreover we consider different network loads in order toanalyze the behavior of scheduling strategies under differentnetwork conditions Flow arrival rate determines networkload 120588 where 120588 = 120582 sdot (E[119883]119904

119873) We consider seven network

states in this study low load (120588 = 025 0375) medium load(120588 = 05 075) and high load (120588 = 085 09 095)

Regarding QoE aspects as typically utilized in perceivedquality studies [27] we use a log-shaped MOS function forQoE degradation presented in (19) This way employing theMOS curve modelling provided in [21] as starting point andtaking into account the flow size the network type and theuser expertise we have suitably defined 119889min and 119889max delaythresholds used in simulations and consequently 119888

1and 1198882

MOS coefficients Consider

MOS = 1198881minus 1198882sdot log (119889) (19)

We have implemented the whole network environmentfor the scheduling of network resources in MATLAB Inrelation to the performed experiments for each scenario

Abstract and Applied Analysis 9

Table 2 Parameter set in the experimental scenarios

Scenario MOS Size Channel 119888

1 119889min = 05 s 119889max = 10 s 120572 = 15 120574 = 4 sdot 10minus7 CQI = 3 5 11198881= 407 119888

2= 133 E[119883] = 5Mbit 119902

2= 05

2 119889min = 04 s 119889max = 06 s 120572 = 15 120574 = 4 sdot 10minus7 CQI = 3 5 11198881= minus403 119888

2= 986 E[119883] = 5Mbit 119902

2= 05

3 119889min = 05 s 119889max = 10 s 120572 = 15 120574 = 4 sdot 10minus7 CQI = 3 9 11198881= 407 119888

2= 133 E[119883] = 5Mbit 119902

2= 05

4 119889min = 005 s 119889max = 1 s 120572 = 15 120574 = 4 sdot 10minus6 CQI = 3 5 11198881= 1 1198882= 133 E[119883] = 05Mbit 119902

2= 05

5

119889min1 = 05 s 119889max1 = 5 s 1205721= 15 120574

1= 4 sdot 10

minus7

see Table 3 5 111988811= 379 119888

21= 173 E[119883

1] = 5Mbit

119889min2 = 5 s 119889max2 = 30 s 1205722= 15 120574

2= 4 sdot 10

minus8

11988812= 859 119888

22= 223 E[119883

2] = 50Mbit

Table 3 Channel state probabilities in Scenario 5

CQI 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15119902 028 012 009 008 008 008 007 006 005 004 003 001 0009 00005 00003 00002

for each combination of scheduling discipline and networkload we have carried out a set of 10 rounds of 10000 slength simulations These rounds differ in the randomlypregenerated traces (input vector) of sizes and arrivals Thusnot only the overall average MOS values will be provided butwe will also include their 95 confidence intervals

Besides in order to guarantee that the obtained perfor-mance results are generally valid we will analyze differentsimulation scenarios We have chosen five relevant settingswhich differ inQoE size channel orand cost characteristicsThe parameters of these scenarios are summarized in Table 2Note that in the first four scenarios we restrict ourselves tothe case of two channel conditions so as to get fundamentalperformance results

We have carefully selected the scenarios used in simula-tions First of all we define a setting (Scenario 1) in which thevalues of all the parameters are typical or standard the valuesof these parameters introduce low error in the approxima-tions considered for our ASPIM scheduler proposal Thenin order to evaluate worst performance cases for ASPIMwe have chosen three scenarios (Scenario 2 Scenario 3 andScenario 4) in which the error of the approximation used forASPIM is high due to a different factor (gradient ofMOS rateand size resp)

Apart from that it is worthmentioning that for obtainingthe ASPIM index rule that we have considered an isolatedMDP model per user Hence the achieved solution is basedon a unique user type or single class However it would beinteresting to examine the proposal when there are differenttypes of users which is analyzed in Scenario 5 Furthermorethis last setting resembles a real 4G wireless network Notethat even though this technology allows the simultaneoustransmission of multiple flows per TTI we assume that asingle flow transmits in each TTI

03 04 05 06 07 08 09

25

3

35

4

45

5

E[M

OS]

ASPIMMRPF

cAS120583GM

120588

Figure 5 E[MOS] for the basic scenario

Next we describe the results we have achieved in theaforementioned scenarios

51 Scenario 1 Basic Case In this first family of simulationswe consider a basic or typical scenario which takes intoaccount the equiprobable channel case and medium-sizedself-similar flows Figure 5 shows averageMOS results for thissetting employing different scheduling policies and networkloadsThemain conclusion achieved from these results is thatASPIM outperforms the rest of disciplines which is betterthan the QoE-aware GM rule proposed for constant channelcapacity Moreover contrary to the other policies it holds

10 Abstract and Applied Analysis

0 02 04 06 08 10

1

2

3

4

d (s)

PDF(d

)

(a)

1 2 3 4 50

01

02

03

04

05

06

07

MOS

PDF(

MO

S)

ASPIMMRPF

cAS120583GM

(b)

1 2 3 4 50

02

04

06

08

1

MOS

CCD

F(M

OS)

ASPIMMRPF

cAS120583GM

Satisth

(c)

Figure 6 Statistics in Scenario 1 delay PDF (a) MOS PDF (b) and MOS CCDF (c)

averageMOSvalues in the same level for all120588 Referring to thetraditional channel-aware disciplines both PF and MR givebad behavior comparing to the rest of scheduling strategies

In order to justify the previous performance results wenow analyze delay andMOS statistics for the highest networkload considered In such a way as regarded in Figure 6(a) forthe Probability Density Function (PDF) of delay we highlightthe probabilitymass or peak that appears for some schedulingdisciplines before the low delay threshold (05 s) The heightand weight of a peak belonging to a policy have direct impacton its average MOS ASPIM discipline shows the highestand the widest peak among the presented ones followedby GM but with a notable difference Therefore our novelQoE-aware policy seems to try to finish the transmissionof most of the flows before 119889min which is reflected in theaforementioned peaks Note that the cAS120583 rule aimed atminimizing delay behaves in different manner which showsmore fairness among delays in the range shown compared tothe QoE-aware ones

What is previously concluded about delay statistics hasa direct reflection on QoE statistics This way as can beseen in the PDF of MOS presented in Figure 6(b) if theobjective is to maximize average QoE it is important onthe one hand to be the height and the weight of the peaksaround the maximum MOS high and on the other hand tobe the value around the minimum MOS low ASPIM fulfilsthese positive properties and consequently it is the bestin average QoE terms Besides if we analyze the Comple-mentary Cumulative Distribution Function (CCDF) of MOSprovided in Figure 6(c) we observe that ASPIM gives thehighest probability of being MOS value higher Consideringfor example a QoE satisfaction threshold satisth of MOS =4 in this point ASPIM improves GM a 10 whereas thedeterioration of MR and PF is higher than a 50

52 Scenario 2 High Demanding Users Case This scenarioreflects the case of users with low tolerance to delay in

Abstract and Applied Analysis 11

03 04 05 06 07 08 09

05

06

07

08

09

ASPIMMRPF

cAS120583GM

120588

E[M

OS]

2E

[MO

S]1

Figure 7 E[MOS] for abrupt MOS scenario

which theMOS function in theQoE degradation range showsa great slope Thus in this setting the error of the MOSapproximation (15) is high since the gradient of MOS amongconsecutive delay levels becomes significant In Figure 7we compare the results of this second scenario and thebasic scenario From the illustrated graph we could say thatour QoE-aware proposal adapts to any MOS function Thisis whereas for the rest of disciplines (excluding GM) thenegative effects due to the abrupt degradation of the QoEutility function are appreciable ASPIMmanages to avoid thisadapting to any MOS function

53 Scenario 3 Higher Error Case due to Rate During thesimplification process carried out for obtaining the ASPIMindex rule in Section 41 the 120583 approximation used (see (14))introduces a rate dependant error Note that increasing therate in the good channel state makes bigger the error of the 120583approximation used Therefore we will analyze the obtainedperformance for our new QoE-aware proposal under thishigher error due to rate In the previous scenarios we haveassumed that the transmission rate in the good channelcondition is twice the rate in the bad channel state In thissetting we consider that the transmission rate in the goodchannel is four times the transmission rate in the bad channelThis way according to the QoE results presented in Figure 8in spite of a higher error caused by rate ASPIM is superior tothe rest of policies

54 Scenario 4 High Error Case due to Size Besides it isknown that for a distribution with a decreasing hazard ratesuch as Pareto that the gradient of completion probabilityincreases as long as the mean size decreases which makesthe error of the 120583 approximation (14) for the ASPIM ruleachievement bigger In this way in this scenario we verifythat ASPIM gives satisfactory behavior under a more notableerror due to size where we consider a mean size ten times

03 04 05 06 07 08 09

3

35

4

45

5

E[M

OS]

ASPIMMRPF

cAS120583GM

120588

Figure 8 E[MOS] under higher error due to rate

03 04 05 06 07 08 093

35

4

45

5E

[MO

S]

ASPIMMRPF

cAS120583GM

120588

Figure 9 E[MOS] under high error due to size

lower We conclude from Figure 9 that even under a higherror caused by size that ASPIM shows good performancebeing the best alternative

55 Scenario 5 A Real Network Case This last scenarioreflects a real 4G wireless network context but with thesimplification that a single user transmits in each TTI To thatend we use CQI traces obtained from a system-level radioaccess simulator [28] considering users moving at 5 kmhin the cell For this setting with dynamic users we definechannel propagation by the Extended Pedestrian Amodel Asin current 4G networks we consider 16 channel conditionsThe state probabilities of these channel states are collectedin Table 3 which are decreasing with the improvement ofchannel quality In this scenario we consider two classes ofusers which differ in size QoE and cost The mean size

12 Abstract and Applied Analysis

03 04 05 06 07 08 09

1

2

3

4

5

E[M

OS]

120588

(a)

03 04 05 06 07 08 091

2

3

4

5

E[M

OS]

ASPIMMRPF

cAS120583GM

120588

(b)

03 04 05 06 07 08 091

2

3

4

5

E[M

OS]

ASPIMMRPF

cAS120583GM

120588

(c)

Figure 10 E[MOS] for 4G scenario aggregate (a) class 1 (b) class 2 (c)

in class 2 is ten times bigger than in class 1 and we givepriority to the small-sized class by having bigger cost andQoErequirement

Under this mixture of classes and realistic channel mod-els in Figure 10(a) we observe that scheduling strategiesshow the same tendency as in the previous scenarios It isremarkable that the difference between ASPIM and GM isbigger especially for high 120588 values Moreover under thehighest network load ASPIM achieves MOS values aboveQoE satisfaction threshold (MOS = 4) Furthermore averageMOS values forMR and PF dramatically decrease comparingwith the previous settings

Concerning class behaviour we show per-class resultsin Figures 10(b) and 10(c) As can be concluded fromthe presented results QoE-aware policies guarantee fairnessamong classes and ASPIM is the best choice also inside classAside from that for the cost dependant cAS120583 discipline thedeterioration of the class with no priority is significant

6 Conclusions

This paper goes into detail about obtaining a schedulingalgorithmaimed atmaximizing usersrsquo perception of quality inchannels with time-varying capacity Furthermore this workconsiders traffic flowswith nonexponential size distributionscontrary to previous approaches that assume memorylessdistributions

As first contribution we provide a MDP-based modelfor the scheduling problem presented This model combinesQoE-awareness size-awareness and channel-awarenessNev-ertheless since this model can not be solved either analyt-ically or computationally from the proposed MDP modelwe derive a simple tractable and implementable well-performing scheduling index rule by applying amethodologybased on Whittle approach

The proposed scheduling solution still combines QoE-size- and channel-awareness Our scheduling proposal givespriority to users in their best channel condition that are

Abstract and Applied Analysis 13

out of the service unavoidable range As verified fromsimulation results the proposed scheduling strategy showssuitable subjective quality performance in a wide range ofscenarios including the case of a simplified 4G network withheterogeneous users

Therefore the results of this work present a relevantmathematical basis for developing scheduling algorithmsaimed at maximizing QoE in current and future networkswith time-varying channel capacity This way the proposedchannel-aware ASPIM discipline will be useful for networkoperators in order to guarantee to their customers servicesatisfaction in time-varying wireless networks Apart fromthat the approximations and simplifications used in Whittlemethod are applicable to any area when the gradient ofsystem components is considerably small among consecutivedecision slots

As future research we will extend our work to amultiuserapproach considering the simultaneous transmission ofmul-tiple flows per TTI

Appendix

Proof of Proposition 4

LemmaA1 Suppose that states (119886 119889+1119898 le 119899)cup(119886+119903119899 119889+

1119898 le 119899) are passive and that states (119886 119889 + 1119898 gt 119899) cup (119886 +

119903119899 119889 + 1119898 gt 119899) are active and

Vlowast(119886119889119899)

=120573120583(119886119899)

( MOS (119889 + 1) minus 119877119886119889)

1 minus 120573120583(119886119899)

119882119886119889

(A1)

Along with this if we assume that 119877119886119889

= sum119898119902119898119877(119886119889119898)

asymp

sum119898119902119898119877(119886+119903119897 119889+1119898)

and 119882119886119889

= sum119898119902119898119882(119886119889119898)

asymp

sum119898119902119898119882(119886+119903119897 119889+1119898)

for any 119897 = 1 2 119873 value for theundiscounted case

Vlowast(119886119889119899)

= lim120573rarr1

120573120583(119886119899)

sdot MOS (119889 + 1) (1 minus 120573)1 minus 120573 + 120573sum

119898gt119899119902119898(120583(119886119898)

minus 120583(119886119899)

) (A2)

Proof Referring to work elements using (12)

119882(119886119889119898le119899)

= 120573sum

119898

1199021015840

119898119882(119886119889+1119898

1015840)= 120573119882

119886119889

119882(119886119889119898gt119899)

= 1 + 120573 (1 minus 120583(119886119898)

)sum

119898

1199021015840

119898119882(119886+119903119898 119889+1119898

1015840)

= 1 + 120573 (1 minus 120583(119886119898)

)119882119886119889

(A3)

And by (A3) expressions

119882119886119889= (1 minus sum

119898gt119899

119902119898)120573119882

119886119889+ sum

119898gt119899

119902119898(1 + 120573 (1 minus 120583

(119886119898))119882119886119889)

(A4)

This way isolating119882119886119889

119882119886119889=

sum119898gt119899

119902119898

1 minus 120573 + 120573sum119898gt119899

119902119898120583(119886119898)

(A5)

Analogously for reward elements using (11)

119877(119886119889119898le119899)

= 120573sum

119898

1199021015840

119898119877(119886119889+1119898

1015840)= 120573119877119886119889

119877(119886119889119898gt119899)

= 120573[(1 minus 120583(119886119898)

)sum

119898

1199021015840

119898119877(119886+119903119898 119898

1015840)+ 120583(119886119898)

sdot MOS (119889 + 1)]

= 120573 [(1 minus 120583(119886119898)

) 119877119886119889+ 120583(119886119898)

sdot MOS (119889 + 1)] (A6)

And thus from (A6)

119877119886119889= (1 minus sum

119898gt119899

119902119898)120573119877119886119889

+ sum

119898gt119899

119902119898(120573 [(1 minus 120583

(119886119898)) 119877119886119889+ 120583(119886119898)

sdot MOS (119889 + 1)])

(A7)

Isolating 119877119886119889

119877119886119889=120573sum119898gt119899

119902119898120583(119886119898)

MOS (119889 + 1)1 minus 120573 + 120573sum

119898gt119899119902119898120583(119886119898)

(A8)

Substituting (A5) and (A8) in (A1) and by simplifyingwe obtain

Vlowast(119886119889119899)

= (120573120583(119886119899)

(MOS (119889 + 1)

minus120573sum119898gt119899

119902119898120583(119886119898)

MOS (119889 + 1)1 minus 120573 + 120573sum

119898gt119899119902119898120583(119886119898)

))

times (1 minus 120573120583(119886119899)

sum119898gt119899

119902119898

1 minus 120573 + 120573sum119898gt119899

119902119898120583(119886119898)

)

minus1

= (120573120583(119886119899)

(MOS (119889 + 1) (1 minus 120573 + 120573sum119898gt119899

119902119898120583(119886119898)

)

minus120573sum

119898gt119899

119902119898120583(119886119898)

MOS (119889 + 1)))

times (1 minus 120573 + 120573sum

119898gt119899

119902119898120583(119886119898)

minus 120573120583(119886119899)

sum

119898gt119899

119902119898)

minus1

=120573120583(119886119899)

sdot MOS (119889 + 1) (1 minus 120573)1 minus 120573 + 120573sum

119898gt119899119902119898(120583(119886119898)

minus 120583(119886119899)

)

(A9)

And therefore for the undiscounted case the 120573 rarr 1

limit for expression (A9) is (A2)

Conflict of Interests

The authors declare that there is no conflict of interestsregarding the publication of this paper

14 Abstract and Applied Analysis

Acknowledgment

The research leading to these results has received fundingfrom the European Union Seventh Framework Programme(FP72007ndash2013) under Grant agreement 284863 (FP7 SECGERYON)

References

[1] ITU-T P10G100 Amendment 1 (0107) New Appendix I-definition of Quality of Experience (QoE) 2006

[2] S Aalto and P Lassila ldquoFlow-level stability and performance ofchannel-aware priority-based schedulersrdquo in Proceeedings of the6th Euro NF Conference on Next Generation Internet (NGI rsquo10)IEEE June 2010

[3] L KleinrockQueueing Systems vol 2 ofComputer ApplicationsJohn Wiley amp Sons 1976

[4] J Gittins ldquoBandit processes and dynamic allocation indicesrdquoJournal of the Royal Statistical Society B Methodological vol 41no 2 pp 148ndash177 1979

[5] K Avrachenkovt U Ayesta P Brown and E Nyberg ldquoDif-ferentiation between short and long TCP ows predictabilityof the response timerdquo in Proceedings of the 23rd Annual JointConference of the IEEE Computer and Communications Societies(INFOCOM rsquo04) vol 2 p 762 IEEE 2004

[6] U Ayesta M Erausquin and P Jacko ldquoA modeling frameworkfor optimizing the flow-level scheduling with time-varyingchannelsrdquo Performance Evaluation vol 67 no 11 pp 1014ndash10292010

[7] S Aalto U Ayesta and R Righter ldquoProperties of the Gittinsindex with application to optimal schedulingrdquo Probability in theEngineering and Informational Sciences vol 25 no 3 pp 269ndash288 2011

[8] M Puterman ldquoMarkov decision processesrdquo in Handbooks inOperations Research and Management Science vol 2 pp 331ndash434 1990

[9] C H Papadimitriou and J N Tsitsiklis ldquoThe complexityof Markov decision processesrdquo Mathematics of OperationsResearch vol 12 no 3 pp 441ndash450 1987

[10] J Nino-Mora ldquoDynamic priority allocation via restless banditmarginal productivity indicesrdquo TOP vol 15 no 2 pp 161ndash1982007

[11] P Whittle ldquoRestless bandits activity allocation in a changingworldrdquo Journal of Applied Probability vol 25 pp 287ndash298 1988

[12] U Ayesta M Erausquin M Jonckheere and I M VerloopldquoScheduling in a random environment stability and asymptoticoptimalityrdquo IEEEACM Transactions on Networking vol 21 no1 pp 258ndash271 2013

[13] S Khan S Thakolsri E Steinbach and W Kellerer ldquoQoE-based cross-layer optimization for wireless multiuser systemsrdquoin Proceedings of the 18th ITC Specialist Seminar on Quality ofExperience 2008

[14] S Thakolsri S Khan E Steinbach and W Kellerer ldquoQoE-driven cross-layer optimization for high speed downlink packetaccessrdquo Journal of Communications vol 4 no 9 pp 669ndash6802009

[15] P Ameigeiras J J Ramos-Munoz J Navarro-Ortiz PMogensen and J M Lopez-Soler ldquoQoE oriented cross-layer design of a resource allocation algorithm in beyond 3GsystemsrdquoComputer Communications vol 33 no 5 pp 571ndash5822010

[16] I Taboada F Liberal J O Fajardo and U Ayesta ldquoQoE-aware optimization of multimedia flow schedulingrdquo ComputerCommunications vol 36 no 15-16 pp 1629ndash1638 2013

[17] U Ayesta M Erausquin and P Jacko ldquoResource-sharing in asingle server with time-varying capacityrdquo in Proceeding of the49th Annual Allerton Conference on Communication Controland Computing (Allerton rsquo11) pp 377ndash383 Monticello Ill USASeptember 2011

[18] K Thompson G J Miller and R Wilder ldquoWide-area internettraffic patterns and characteristicsrdquo IEEE Network vol 11 no 6pp 10ndash23 1997

[19] M Fiedler T Hossfeld and P Tran-Gia ldquoA generic quantitativerelationship between quality of experience and quality ofservicerdquo IEEE Network vol 24 no 2 pp 36ndash41 2010

[20] A Richards M Antoniades V Witana and G Rogers ldquoMap-ping user level QoS from a single parameterrdquo in Proceedingsof the International Conference on Multimedia Networks andServices Citeseer 1998

[21] E Ibarrola F Liberal I Taboada and R Ortega ldquoWebQoE evaluation in multi-agent networks validation of ITU-TG1030rdquo in Proceedings of the 5th International Conference onAutonomic and Autonomous Systems (ICAS rsquo09) pp 289ndash294Valencia Spain April 2009

[22] S Aalto and U Ayesta ldquoOptimal scheduling of jobs with aDHR tail in the MG1 queuerdquo in Proceedings of the 3rd Inter-national Conference on Performance Evaluation Methodologiesand Tools (ValueTools rsquo08) article 50 Institute for ComputerSciences Social-Informatics and Telecommunications Engi-neering Athens Greece 2008

[23] V Visweswaran ldquoDecomposition techniques for MILPlagrangian relaxationrdquo in Encyclopedia of Optimization pp632ndash638 Springer New York NY USA 2009

[24] I Taboada J O Fajardo and F Liberal ldquoPerformance analysisof scheduling algorithms for web QoE optimization in wirelessnetworksrdquo Network Protocols and Algorithms vol 4 pp 27ndash432012

[25] J A van Mieghem ldquoDynamic scheduling with convex delaycosts the generalized 119888120583 rulerdquoTheAnnals of Applied Probabilityvol 5 no 3 pp 809ndash833 1995

[26] S Sesia I Toufik and M Baker LTE-The UMTS Long TermEvolution FromTheory to Practice Wiley 2011

[27] P Reichl B Tuffin and R Schatz ldquoLogarithmic laws inservice quality perception where microeconomics meets psy-chophysics and quality of experiencerdquo Telecommunication Sys-tems vol 52 no 2 pp 587ndash600 2013

[28] J C Ikuno M Wrulich and M Rupp ldquoSystem level simulationof LTE networksrdquo in Proceedings of the IEEE 71st VehicularTechnology Conference Taipei Taiwan May 2010

Submit your manuscripts athttpwwwhindawicom

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical Problems in Engineering

Hindawi Publishing Corporationhttpwwwhindawicom

Differential EquationsInternational Journal of

Volume 2014

Applied MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical PhysicsAdvances in

Complex AnalysisJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

OptimizationJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Operations ResearchAdvances in

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Function Spaces

Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of Mathematics and Mathematical Sciences

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Algebra

Discrete Dynamics in Nature and Society

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Decision SciencesAdvances in

Discrete MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Stochastic AnalysisInternational Journal of

Abstract and Applied Analysis 7

(4) In a better channel condition for the same attainedservice level and the same delay level the Whittleindex value is higher Vlowast

(1198861198891198981)gt Vlowast(1198861198891198982)

forall1198981gt 1198982

(Observe Figure 3)(5) Opposite to the previous channel-aware work [6]

where Whittle indices in the best channel conditionare infinite having absolute priority in this channelstate in our case being in the best channel conditiondoes not guarantee that the Whittle index value ishigher than in the rest of the channel conditions it ispossible to fulfil Vlowast

(119886119889119898lt119873)gt Vlowast(11988610158401198891015840119873)

If we comparethe two graphs in Figure 3 it can be easily seen howfor several attained services andor delayed values theWhittle index values in the worst channel conditionare higher than in the best channel condition (takingalso into account nonnull values)

(6) Dependency on channel condition probabilities hilefor channel condition 119873 the Whittle index valueremains invariant with channel state probability forthe rest of channel conditions its value decreases aslong as channel state probabilities increase in betterchannel conditions (shown in Figure 4)

Once we analyzed the properties of theWhittle index weset out to define the structure of the active set in order toobtain an analytically and computationally tractable Whittleindex expression In such a way to derive a closed-formcharacterization of the Whittle index of (119886 119889 119899) state first ofall it is necessary to specify the activity of (119886 119889 + 1119898) and(119886 + 119903

119899 119889 + 1119898) states As concluded from Whittle index

properties it is known that (119886 119889+1119898 le 119899)cup(119886+119903119899 119889+1119898 le

119899) states are passive However it is not trivial to specify abovewhich attained service level andor delay level if (119886 119889+1119898 gt

119899)cup (119886+ 119903119899 119889 + 1119898 gt 119899) states with better channel condition

are active or passiveNonetheless in the best channel the active set is totally

specified since all the states that compute in the index are pas-sive because of nonexisting channel improvement and beingin the future attained service equal or higher and delay higherAs shown in Proposition 3 for the best channel condition theWhittle index has a closed-form expression where the indexvalue is the multiplication between completion probabilityand normalizedMOS function Note that the obtainedWhit-tle index expression in the best channel state is equivalent tothe GM index achieved for constant channel [16]

Proposition 3 The Whittle index for Problem (8) in the bestchannel condition is given by

Vlowast(119886119889119873)

= 120583(119886119873)

sdot MOS (119889 + 1) (13)

Proof It is known that for Vlowast(119886119889119873)

index computation thatF = 0 In this way using (11) and (12) RF

(119886119889+1119898)=

120573sum119898isinN 119902

119898RF(119886119889+2119898)

RF(119886119889+2119898)

= 120573sum119898isinN 119902

119898RF(119886119889+3119898)

RF(119886119889gt119889max 119898)

= 0 Hence analogolously RF(119886+119903119899 119889+1119898)

= 0WF(119886119889+1119898)

= 0 and WF(119886+119903119899 119889+1119898)

= 0 Substituting thesenull terms in expression (10) and being 120573 = 1 we achieve(13)

Besides as stated before obtaining a computationallytractable Whittle index expression for any channel conditionis not possible In such situation in order to achieve atractable Whittle-based QoE-aware index rule for time-varying channels we propose a Whittle-based approximateheuristic in the next subsection

41 Approximate Whittle Index Proposal In the following wepropose an approximate solution for the QoE-aware Whittleindex in a time-varying channel context (10)

First of all we will determine the structure of the activeset For the Whittle index computation of (119886 119889 119899) state aspreviously concluded it is known that (119886 119889 + 1119898 le 119899) cup (119886+

119903119899 119889 + 1119898 le 119899) states are passive but we need to define the

unknown activity of (119886 119889 + 1119898 gt 119899) cup (119886 + 119903119899 119889 + 1119898 gt 119899)

states Thus so as to determine if these states are active orpassive we apply the following two considerations

(1) 120583 approximation

120583(119886119898)

asymp 120583(119886+119903119897 119898)

(14)

(2) MOS approximation

MOS (119889) asymp MOS (119889 + 1) (15)

On the one hand we assume that for any 119897 = 1 2 119873

value that 120583(119886119898)

asymp 120583(119886+119903119897 119898)

(14) It is known that (119886 119889119898 gt

119899) states are active and being 120583(119886119898)

asymp 120583(119886+119903119899 119898)

we couldconsider that (119886 + 119903

119899 119889 119898 gt 119899) states are active

On the other hand we assume MOS(119889) asymp MOS(119889 + 1)(15) since QoE degradation in a higher delay level is neg-ligible in the millisecond scale that delay evolves Takinginto account the approximation about MOS function being(119886 119889119898 gt 119899) states active we can suppose that (119886 119889 + 1119898 gt

119899) states are also active This way combining approximations(14) and (15) we consider that (119886 + 119903

119899 119889 + 1119898 gt 119899) states are

activeThen suppose that the aforementioned 120583 and

MOS approximations cause 119877119886119889

= sum119898119902119898119877(119886119889119898)

asymp

sum119898119902119898119877(119886+119903119897 119889+1119898)

and 119882119886119889

= sum119898119902119898119882(119886119889119898)

asymp

sum119898119902119898119882(119886+119903119897 119889+1119898)

So using these reward and workapproximations the first simplified expression for theWhittle index formulation (10) is

Vlowast(119886119889119899)

=120573120583(119886119899)

(MOS (119889 + 1) minus 119877119886119889)

1 minus 120573120583(119886119899)

119882119886119889

(16)

In such a way applying the previous simplificationswe obtain the Whittle index approximation presentedin Proposition 4 what we call attained service PI MOS(ASPIM) The methodology to compute the closed-formexpression of this QoE-aware Whittle-based index is pre-sented in the Appendix

Proposition 4 The formulation of the ASPIM index is

119860119878119875119868119872 = Vlowast(119886119889119899)

= lim120573rarr1

120573120583(119886119899)

sdot MOS (119889 + 1) (1 minus 120573)1 minus 120573 + 120573sum

119898gt119899119902119898(120583(119886119898)

minus 120583(119886119899)

)

(17)

8 Abstract and Applied Analysis

Table 1 CQIs and corresponding transmission rates (Mbps)

CQI 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15119904 0 42 672 84 11256 168 2184 252 2688 336 4468 504 5376 672 758 8064

If we analyze expression (17) we observe that this QoE-aware and channel-aware Whittle-based ASPIM index isimplementable This proposal depends on QoE size andchannel properties Moreover the obtained index for thebest channel condition is equal to the original Whittle index(13) since the summation in the denominator becomes nullHowever as shown in (18) when channel state is not the bestthe index value goes to zero

ASPIM119899 =119873

= lim120573rarr1

120583(119886119899)

sdot MOS (119889 + 1) (1 minus 120573)sum119898gt119899

119902119898(120583(119886119898)

minus 120583(119886119899)

)997888rarr 0

(18)

Hence a user in its best channel state with nonnullinstantaneous normalized MOS function has priority overa user which is not in its best channel condition Thus wesummarize the proposed QoE-aware Whittle-based ASPIMindex rule in Definition 5

Definition 5 The ASPIM index rule consists of at everydecision slot

(i) serving the user in its best channel condition withnonnull instantaneous normalized MOS functionwith the highest value of 120583

(119886119873)sdot MOS(119889 + 1)

(ii) if there is no user in its best channel condition withnonnull instantaneous normalized MOS functionserving the user with the highest value of (120583

(119886119899)sdot

MOS(119889 + 1))sum119898gt119899

119902119898(120583(119886119898)

minus 120583(119886119899)

) (using (18)normalized by (1 minus 120573))

So if channel condition is not the best the index valueis the ratio of the multiplication between the actual servicecompletion and the instantaneous normalizedMOS functionrespect to the expected potential improvement of the comple-tion probability

Therefore in the context of time-varying channels aimedat maximizing average MOS we have proposed ASPIMscheduling algorithm which is a simple and tractable QoE-aware channel-aware and size-based index rule Neverthe-less in order to verify the correct behavior of this heuristicwe analyze its performance in the subsequent section

5 Performance Evaluation

In this section we evaluate the performance of the proposedASPIM index rule (see proposal in Definition 5) To that endwe compare the QoE performance of our heuristic with theone achieved by well-known scheduling strategies in severalsimulation scenarios Below we provide a brief description ofthe scheduling priority rules used in our experiments

(i) Max rate (MR) this opportunistic policy consists inserving the user with the best channel capacity

(ii) Proportional fair (PF) this channel-aware disci-pline serves the user with the highest ratio betweeninstantaneous transmission rate and current attainedthroughput

(iii) Cost and attained service dependant 120583 (119888AS120583)although it might seem counter-intuitive minimizingmean delay does not result in maximizing averageMOS [24] This way in order to compare the per-formance with delay-aware solutions we propose toadapt the classical 119888120583 rule aimed at minimizing meandelay [25] to attained serviceThis adaptation consistsof serving the user with the highest value of 119888120583

(119886119899)

being 119888 the waiting cost paid for every slot while flowtransmission is uncompleted

(iv) GM [16] thisQoE-aware pioneering index rule origi-nally proposed for constant channel capacity choosesthe user with the highest value of 120583

(119886119899)sdot MOS(119889 + 1)

In case of ties we use random tie-breaking ruleConcerning traffic flows in the system we use the Pareto

size distribution given in (1) in simulations whose 120572 and 120574parameters determine mean flow size We consider typicalmean sizes used in wireless networks 05Mb (small traffica web page email) 5Mb (medium traffic a PDF documenta picture etc) and 50Mb (MP3 audio a group of picturesfrom a video sequence etc) User traffic flows arrive to thescheduler according to a Poisson process with 120582 rate

Referring to network characteristics we use transmis-sion rates that depend on CQIs used in real wirelessnetworks which are associated with 4G modulation andcoding schemes We show the mapping between CQI valuesand transmission rates in Table 1 The presented values areadapted and extrapolated from [26] for a cell of 20MHz

Moreover we consider different network loads in order toanalyze the behavior of scheduling strategies under differentnetwork conditions Flow arrival rate determines networkload 120588 where 120588 = 120582 sdot (E[119883]119904

119873) We consider seven network

states in this study low load (120588 = 025 0375) medium load(120588 = 05 075) and high load (120588 = 085 09 095)

Regarding QoE aspects as typically utilized in perceivedquality studies [27] we use a log-shaped MOS function forQoE degradation presented in (19) This way employing theMOS curve modelling provided in [21] as starting point andtaking into account the flow size the network type and theuser expertise we have suitably defined 119889min and 119889max delaythresholds used in simulations and consequently 119888

1and 1198882

MOS coefficients Consider

MOS = 1198881minus 1198882sdot log (119889) (19)

We have implemented the whole network environmentfor the scheduling of network resources in MATLAB Inrelation to the performed experiments for each scenario

Abstract and Applied Analysis 9

Table 2 Parameter set in the experimental scenarios

Scenario MOS Size Channel 119888

1 119889min = 05 s 119889max = 10 s 120572 = 15 120574 = 4 sdot 10minus7 CQI = 3 5 11198881= 407 119888

2= 133 E[119883] = 5Mbit 119902

2= 05

2 119889min = 04 s 119889max = 06 s 120572 = 15 120574 = 4 sdot 10minus7 CQI = 3 5 11198881= minus403 119888

2= 986 E[119883] = 5Mbit 119902

2= 05

3 119889min = 05 s 119889max = 10 s 120572 = 15 120574 = 4 sdot 10minus7 CQI = 3 9 11198881= 407 119888

2= 133 E[119883] = 5Mbit 119902

2= 05

4 119889min = 005 s 119889max = 1 s 120572 = 15 120574 = 4 sdot 10minus6 CQI = 3 5 11198881= 1 1198882= 133 E[119883] = 05Mbit 119902

2= 05

5

119889min1 = 05 s 119889max1 = 5 s 1205721= 15 120574

1= 4 sdot 10

minus7

see Table 3 5 111988811= 379 119888

21= 173 E[119883

1] = 5Mbit

119889min2 = 5 s 119889max2 = 30 s 1205722= 15 120574

2= 4 sdot 10

minus8

11988812= 859 119888

22= 223 E[119883

2] = 50Mbit

Table 3 Channel state probabilities in Scenario 5

CQI 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15119902 028 012 009 008 008 008 007 006 005 004 003 001 0009 00005 00003 00002

for each combination of scheduling discipline and networkload we have carried out a set of 10 rounds of 10000 slength simulations These rounds differ in the randomlypregenerated traces (input vector) of sizes and arrivals Thusnot only the overall average MOS values will be provided butwe will also include their 95 confidence intervals

Besides in order to guarantee that the obtained perfor-mance results are generally valid we will analyze differentsimulation scenarios We have chosen five relevant settingswhich differ inQoE size channel orand cost characteristicsThe parameters of these scenarios are summarized in Table 2Note that in the first four scenarios we restrict ourselves tothe case of two channel conditions so as to get fundamentalperformance results

We have carefully selected the scenarios used in simula-tions First of all we define a setting (Scenario 1) in which thevalues of all the parameters are typical or standard the valuesof these parameters introduce low error in the approxima-tions considered for our ASPIM scheduler proposal Thenin order to evaluate worst performance cases for ASPIMwe have chosen three scenarios (Scenario 2 Scenario 3 andScenario 4) in which the error of the approximation used forASPIM is high due to a different factor (gradient ofMOS rateand size resp)

Apart from that it is worthmentioning that for obtainingthe ASPIM index rule that we have considered an isolatedMDP model per user Hence the achieved solution is basedon a unique user type or single class However it would beinteresting to examine the proposal when there are differenttypes of users which is analyzed in Scenario 5 Furthermorethis last setting resembles a real 4G wireless network Notethat even though this technology allows the simultaneoustransmission of multiple flows per TTI we assume that asingle flow transmits in each TTI

03 04 05 06 07 08 09

25

3

35

4

45

5

E[M

OS]

ASPIMMRPF

cAS120583GM

120588

Figure 5 E[MOS] for the basic scenario

Next we describe the results we have achieved in theaforementioned scenarios

51 Scenario 1 Basic Case In this first family of simulationswe consider a basic or typical scenario which takes intoaccount the equiprobable channel case and medium-sizedself-similar flows Figure 5 shows averageMOS results for thissetting employing different scheduling policies and networkloadsThemain conclusion achieved from these results is thatASPIM outperforms the rest of disciplines which is betterthan the QoE-aware GM rule proposed for constant channelcapacity Moreover contrary to the other policies it holds

10 Abstract and Applied Analysis

0 02 04 06 08 10

1

2

3

4

d (s)

PDF(d

)

(a)

1 2 3 4 50

01

02

03

04

05

06

07

MOS

PDF(

MO

S)

ASPIMMRPF

cAS120583GM

(b)

1 2 3 4 50

02

04

06

08

1

MOS

CCD

F(M

OS)

ASPIMMRPF

cAS120583GM

Satisth

(c)

Figure 6 Statistics in Scenario 1 delay PDF (a) MOS PDF (b) and MOS CCDF (c)

averageMOSvalues in the same level for all120588 Referring to thetraditional channel-aware disciplines both PF and MR givebad behavior comparing to the rest of scheduling strategies

In order to justify the previous performance results wenow analyze delay andMOS statistics for the highest networkload considered In such a way as regarded in Figure 6(a) forthe Probability Density Function (PDF) of delay we highlightthe probabilitymass or peak that appears for some schedulingdisciplines before the low delay threshold (05 s) The heightand weight of a peak belonging to a policy have direct impacton its average MOS ASPIM discipline shows the highestand the widest peak among the presented ones followedby GM but with a notable difference Therefore our novelQoE-aware policy seems to try to finish the transmissionof most of the flows before 119889min which is reflected in theaforementioned peaks Note that the cAS120583 rule aimed atminimizing delay behaves in different manner which showsmore fairness among delays in the range shown compared tothe QoE-aware ones

What is previously concluded about delay statistics hasa direct reflection on QoE statistics This way as can beseen in the PDF of MOS presented in Figure 6(b) if theobjective is to maximize average QoE it is important onthe one hand to be the height and the weight of the peaksaround the maximum MOS high and on the other hand tobe the value around the minimum MOS low ASPIM fulfilsthese positive properties and consequently it is the bestin average QoE terms Besides if we analyze the Comple-mentary Cumulative Distribution Function (CCDF) of MOSprovided in Figure 6(c) we observe that ASPIM gives thehighest probability of being MOS value higher Consideringfor example a QoE satisfaction threshold satisth of MOS =4 in this point ASPIM improves GM a 10 whereas thedeterioration of MR and PF is higher than a 50

52 Scenario 2 High Demanding Users Case This scenarioreflects the case of users with low tolerance to delay in

Abstract and Applied Analysis 11

03 04 05 06 07 08 09

05

06

07

08

09

ASPIMMRPF

cAS120583GM

120588

E[M

OS]

2E

[MO

S]1

Figure 7 E[MOS] for abrupt MOS scenario

which theMOS function in theQoE degradation range showsa great slope Thus in this setting the error of the MOSapproximation (15) is high since the gradient of MOS amongconsecutive delay levels becomes significant In Figure 7we compare the results of this second scenario and thebasic scenario From the illustrated graph we could say thatour QoE-aware proposal adapts to any MOS function Thisis whereas for the rest of disciplines (excluding GM) thenegative effects due to the abrupt degradation of the QoEutility function are appreciable ASPIMmanages to avoid thisadapting to any MOS function

53 Scenario 3 Higher Error Case due to Rate During thesimplification process carried out for obtaining the ASPIMindex rule in Section 41 the 120583 approximation used (see (14))introduces a rate dependant error Note that increasing therate in the good channel state makes bigger the error of the 120583approximation used Therefore we will analyze the obtainedperformance for our new QoE-aware proposal under thishigher error due to rate In the previous scenarios we haveassumed that the transmission rate in the good channelcondition is twice the rate in the bad channel state In thissetting we consider that the transmission rate in the goodchannel is four times the transmission rate in the bad channelThis way according to the QoE results presented in Figure 8in spite of a higher error caused by rate ASPIM is superior tothe rest of policies

54 Scenario 4 High Error Case due to Size Besides it isknown that for a distribution with a decreasing hazard ratesuch as Pareto that the gradient of completion probabilityincreases as long as the mean size decreases which makesthe error of the 120583 approximation (14) for the ASPIM ruleachievement bigger In this way in this scenario we verifythat ASPIM gives satisfactory behavior under a more notableerror due to size where we consider a mean size ten times

03 04 05 06 07 08 09

3

35

4

45

5

E[M

OS]

ASPIMMRPF

cAS120583GM

120588

Figure 8 E[MOS] under higher error due to rate

03 04 05 06 07 08 093

35

4

45

5E

[MO

S]

ASPIMMRPF

cAS120583GM

120588

Figure 9 E[MOS] under high error due to size

lower We conclude from Figure 9 that even under a higherror caused by size that ASPIM shows good performancebeing the best alternative

55 Scenario 5 A Real Network Case This last scenarioreflects a real 4G wireless network context but with thesimplification that a single user transmits in each TTI To thatend we use CQI traces obtained from a system-level radioaccess simulator [28] considering users moving at 5 kmhin the cell For this setting with dynamic users we definechannel propagation by the Extended Pedestrian Amodel Asin current 4G networks we consider 16 channel conditionsThe state probabilities of these channel states are collectedin Table 3 which are decreasing with the improvement ofchannel quality In this scenario we consider two classes ofusers which differ in size QoE and cost The mean size

12 Abstract and Applied Analysis

03 04 05 06 07 08 09

1

2

3

4

5

E[M

OS]

120588

(a)

03 04 05 06 07 08 091

2

3

4

5

E[M

OS]

ASPIMMRPF

cAS120583GM

120588

(b)

03 04 05 06 07 08 091

2

3

4

5

E[M

OS]

ASPIMMRPF

cAS120583GM

120588

(c)

Figure 10 E[MOS] for 4G scenario aggregate (a) class 1 (b) class 2 (c)

in class 2 is ten times bigger than in class 1 and we givepriority to the small-sized class by having bigger cost andQoErequirement

Under this mixture of classes and realistic channel mod-els in Figure 10(a) we observe that scheduling strategiesshow the same tendency as in the previous scenarios It isremarkable that the difference between ASPIM and GM isbigger especially for high 120588 values Moreover under thehighest network load ASPIM achieves MOS values aboveQoE satisfaction threshold (MOS = 4) Furthermore averageMOS values forMR and PF dramatically decrease comparingwith the previous settings

Concerning class behaviour we show per-class resultsin Figures 10(b) and 10(c) As can be concluded fromthe presented results QoE-aware policies guarantee fairnessamong classes and ASPIM is the best choice also inside classAside from that for the cost dependant cAS120583 discipline thedeterioration of the class with no priority is significant

6 Conclusions

This paper goes into detail about obtaining a schedulingalgorithmaimed atmaximizing usersrsquo perception of quality inchannels with time-varying capacity Furthermore this workconsiders traffic flowswith nonexponential size distributionscontrary to previous approaches that assume memorylessdistributions

As first contribution we provide a MDP-based modelfor the scheduling problem presented This model combinesQoE-awareness size-awareness and channel-awarenessNev-ertheless since this model can not be solved either analyt-ically or computationally from the proposed MDP modelwe derive a simple tractable and implementable well-performing scheduling index rule by applying amethodologybased on Whittle approach

The proposed scheduling solution still combines QoE-size- and channel-awareness Our scheduling proposal givespriority to users in their best channel condition that are

Abstract and Applied Analysis 13

out of the service unavoidable range As verified fromsimulation results the proposed scheduling strategy showssuitable subjective quality performance in a wide range ofscenarios including the case of a simplified 4G network withheterogeneous users

Therefore the results of this work present a relevantmathematical basis for developing scheduling algorithmsaimed at maximizing QoE in current and future networkswith time-varying channel capacity This way the proposedchannel-aware ASPIM discipline will be useful for networkoperators in order to guarantee to their customers servicesatisfaction in time-varying wireless networks Apart fromthat the approximations and simplifications used in Whittlemethod are applicable to any area when the gradient ofsystem components is considerably small among consecutivedecision slots

As future research we will extend our work to amultiuserapproach considering the simultaneous transmission ofmul-tiple flows per TTI

Appendix

Proof of Proposition 4

LemmaA1 Suppose that states (119886 119889+1119898 le 119899)cup(119886+119903119899 119889+

1119898 le 119899) are passive and that states (119886 119889 + 1119898 gt 119899) cup (119886 +

119903119899 119889 + 1119898 gt 119899) are active and

Vlowast(119886119889119899)

=120573120583(119886119899)

( MOS (119889 + 1) minus 119877119886119889)

1 minus 120573120583(119886119899)

119882119886119889

(A1)

Along with this if we assume that 119877119886119889

= sum119898119902119898119877(119886119889119898)

asymp

sum119898119902119898119877(119886+119903119897 119889+1119898)

and 119882119886119889

= sum119898119902119898119882(119886119889119898)

asymp

sum119898119902119898119882(119886+119903119897 119889+1119898)

for any 119897 = 1 2 119873 value for theundiscounted case

Vlowast(119886119889119899)

= lim120573rarr1

120573120583(119886119899)

sdot MOS (119889 + 1) (1 minus 120573)1 minus 120573 + 120573sum

119898gt119899119902119898(120583(119886119898)

minus 120583(119886119899)

) (A2)

Proof Referring to work elements using (12)

119882(119886119889119898le119899)

= 120573sum

119898

1199021015840

119898119882(119886119889+1119898

1015840)= 120573119882

119886119889

119882(119886119889119898gt119899)

= 1 + 120573 (1 minus 120583(119886119898)

)sum

119898

1199021015840

119898119882(119886+119903119898 119889+1119898

1015840)

= 1 + 120573 (1 minus 120583(119886119898)

)119882119886119889

(A3)

And by (A3) expressions

119882119886119889= (1 minus sum

119898gt119899

119902119898)120573119882

119886119889+ sum

119898gt119899

119902119898(1 + 120573 (1 minus 120583

(119886119898))119882119886119889)

(A4)

This way isolating119882119886119889

119882119886119889=

sum119898gt119899

119902119898

1 minus 120573 + 120573sum119898gt119899

119902119898120583(119886119898)

(A5)

Analogously for reward elements using (11)

119877(119886119889119898le119899)

= 120573sum

119898

1199021015840

119898119877(119886119889+1119898

1015840)= 120573119877119886119889

119877(119886119889119898gt119899)

= 120573[(1 minus 120583(119886119898)

)sum

119898

1199021015840

119898119877(119886+119903119898 119898

1015840)+ 120583(119886119898)

sdot MOS (119889 + 1)]

= 120573 [(1 minus 120583(119886119898)

) 119877119886119889+ 120583(119886119898)

sdot MOS (119889 + 1)] (A6)

And thus from (A6)

119877119886119889= (1 minus sum

119898gt119899

119902119898)120573119877119886119889

+ sum

119898gt119899

119902119898(120573 [(1 minus 120583

(119886119898)) 119877119886119889+ 120583(119886119898)

sdot MOS (119889 + 1)])

(A7)

Isolating 119877119886119889

119877119886119889=120573sum119898gt119899

119902119898120583(119886119898)

MOS (119889 + 1)1 minus 120573 + 120573sum

119898gt119899119902119898120583(119886119898)

(A8)

Substituting (A5) and (A8) in (A1) and by simplifyingwe obtain

Vlowast(119886119889119899)

= (120573120583(119886119899)

(MOS (119889 + 1)

minus120573sum119898gt119899

119902119898120583(119886119898)

MOS (119889 + 1)1 minus 120573 + 120573sum

119898gt119899119902119898120583(119886119898)

))

times (1 minus 120573120583(119886119899)

sum119898gt119899

119902119898

1 minus 120573 + 120573sum119898gt119899

119902119898120583(119886119898)

)

minus1

= (120573120583(119886119899)

(MOS (119889 + 1) (1 minus 120573 + 120573sum119898gt119899

119902119898120583(119886119898)

)

minus120573sum

119898gt119899

119902119898120583(119886119898)

MOS (119889 + 1)))

times (1 minus 120573 + 120573sum

119898gt119899

119902119898120583(119886119898)

minus 120573120583(119886119899)

sum

119898gt119899

119902119898)

minus1

=120573120583(119886119899)

sdot MOS (119889 + 1) (1 minus 120573)1 minus 120573 + 120573sum

119898gt119899119902119898(120583(119886119898)

minus 120583(119886119899)

)

(A9)

And therefore for the undiscounted case the 120573 rarr 1

limit for expression (A9) is (A2)

Conflict of Interests

The authors declare that there is no conflict of interestsregarding the publication of this paper

14 Abstract and Applied Analysis

Acknowledgment

The research leading to these results has received fundingfrom the European Union Seventh Framework Programme(FP72007ndash2013) under Grant agreement 284863 (FP7 SECGERYON)

References

[1] ITU-T P10G100 Amendment 1 (0107) New Appendix I-definition of Quality of Experience (QoE) 2006

[2] S Aalto and P Lassila ldquoFlow-level stability and performance ofchannel-aware priority-based schedulersrdquo in Proceeedings of the6th Euro NF Conference on Next Generation Internet (NGI rsquo10)IEEE June 2010

[3] L KleinrockQueueing Systems vol 2 ofComputer ApplicationsJohn Wiley amp Sons 1976

[4] J Gittins ldquoBandit processes and dynamic allocation indicesrdquoJournal of the Royal Statistical Society B Methodological vol 41no 2 pp 148ndash177 1979

[5] K Avrachenkovt U Ayesta P Brown and E Nyberg ldquoDif-ferentiation between short and long TCP ows predictabilityof the response timerdquo in Proceedings of the 23rd Annual JointConference of the IEEE Computer and Communications Societies(INFOCOM rsquo04) vol 2 p 762 IEEE 2004

[6] U Ayesta M Erausquin and P Jacko ldquoA modeling frameworkfor optimizing the flow-level scheduling with time-varyingchannelsrdquo Performance Evaluation vol 67 no 11 pp 1014ndash10292010

[7] S Aalto U Ayesta and R Righter ldquoProperties of the Gittinsindex with application to optimal schedulingrdquo Probability in theEngineering and Informational Sciences vol 25 no 3 pp 269ndash288 2011

[8] M Puterman ldquoMarkov decision processesrdquo in Handbooks inOperations Research and Management Science vol 2 pp 331ndash434 1990

[9] C H Papadimitriou and J N Tsitsiklis ldquoThe complexityof Markov decision processesrdquo Mathematics of OperationsResearch vol 12 no 3 pp 441ndash450 1987

[10] J Nino-Mora ldquoDynamic priority allocation via restless banditmarginal productivity indicesrdquo TOP vol 15 no 2 pp 161ndash1982007

[11] P Whittle ldquoRestless bandits activity allocation in a changingworldrdquo Journal of Applied Probability vol 25 pp 287ndash298 1988

[12] U Ayesta M Erausquin M Jonckheere and I M VerloopldquoScheduling in a random environment stability and asymptoticoptimalityrdquo IEEEACM Transactions on Networking vol 21 no1 pp 258ndash271 2013

[13] S Khan S Thakolsri E Steinbach and W Kellerer ldquoQoE-based cross-layer optimization for wireless multiuser systemsrdquoin Proceedings of the 18th ITC Specialist Seminar on Quality ofExperience 2008

[14] S Thakolsri S Khan E Steinbach and W Kellerer ldquoQoE-driven cross-layer optimization for high speed downlink packetaccessrdquo Journal of Communications vol 4 no 9 pp 669ndash6802009

[15] P Ameigeiras J J Ramos-Munoz J Navarro-Ortiz PMogensen and J M Lopez-Soler ldquoQoE oriented cross-layer design of a resource allocation algorithm in beyond 3GsystemsrdquoComputer Communications vol 33 no 5 pp 571ndash5822010

[16] I Taboada F Liberal J O Fajardo and U Ayesta ldquoQoE-aware optimization of multimedia flow schedulingrdquo ComputerCommunications vol 36 no 15-16 pp 1629ndash1638 2013

[17] U Ayesta M Erausquin and P Jacko ldquoResource-sharing in asingle server with time-varying capacityrdquo in Proceeding of the49th Annual Allerton Conference on Communication Controland Computing (Allerton rsquo11) pp 377ndash383 Monticello Ill USASeptember 2011

[18] K Thompson G J Miller and R Wilder ldquoWide-area internettraffic patterns and characteristicsrdquo IEEE Network vol 11 no 6pp 10ndash23 1997

[19] M Fiedler T Hossfeld and P Tran-Gia ldquoA generic quantitativerelationship between quality of experience and quality ofservicerdquo IEEE Network vol 24 no 2 pp 36ndash41 2010

[20] A Richards M Antoniades V Witana and G Rogers ldquoMap-ping user level QoS from a single parameterrdquo in Proceedingsof the International Conference on Multimedia Networks andServices Citeseer 1998

[21] E Ibarrola F Liberal I Taboada and R Ortega ldquoWebQoE evaluation in multi-agent networks validation of ITU-TG1030rdquo in Proceedings of the 5th International Conference onAutonomic and Autonomous Systems (ICAS rsquo09) pp 289ndash294Valencia Spain April 2009

[22] S Aalto and U Ayesta ldquoOptimal scheduling of jobs with aDHR tail in the MG1 queuerdquo in Proceedings of the 3rd Inter-national Conference on Performance Evaluation Methodologiesand Tools (ValueTools rsquo08) article 50 Institute for ComputerSciences Social-Informatics and Telecommunications Engi-neering Athens Greece 2008

[23] V Visweswaran ldquoDecomposition techniques for MILPlagrangian relaxationrdquo in Encyclopedia of Optimization pp632ndash638 Springer New York NY USA 2009

[24] I Taboada J O Fajardo and F Liberal ldquoPerformance analysisof scheduling algorithms for web QoE optimization in wirelessnetworksrdquo Network Protocols and Algorithms vol 4 pp 27ndash432012

[25] J A van Mieghem ldquoDynamic scheduling with convex delaycosts the generalized 119888120583 rulerdquoTheAnnals of Applied Probabilityvol 5 no 3 pp 809ndash833 1995

[26] S Sesia I Toufik and M Baker LTE-The UMTS Long TermEvolution FromTheory to Practice Wiley 2011

[27] P Reichl B Tuffin and R Schatz ldquoLogarithmic laws inservice quality perception where microeconomics meets psy-chophysics and quality of experiencerdquo Telecommunication Sys-tems vol 52 no 2 pp 587ndash600 2013

[28] J C Ikuno M Wrulich and M Rupp ldquoSystem level simulationof LTE networksrdquo in Proceedings of the IEEE 71st VehicularTechnology Conference Taipei Taiwan May 2010

Submit your manuscripts athttpwwwhindawicom

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical Problems in Engineering

Hindawi Publishing Corporationhttpwwwhindawicom

Differential EquationsInternational Journal of

Volume 2014

Applied MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical PhysicsAdvances in

Complex AnalysisJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

OptimizationJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Operations ResearchAdvances in

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Function Spaces

Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of Mathematics and Mathematical Sciences

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Algebra

Discrete Dynamics in Nature and Society

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Decision SciencesAdvances in

Discrete MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Stochastic AnalysisInternational Journal of

8 Abstract and Applied Analysis

Table 1 CQIs and corresponding transmission rates (Mbps)

CQI 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15119904 0 42 672 84 11256 168 2184 252 2688 336 4468 504 5376 672 758 8064

If we analyze expression (17) we observe that this QoE-aware and channel-aware Whittle-based ASPIM index isimplementable This proposal depends on QoE size andchannel properties Moreover the obtained index for thebest channel condition is equal to the original Whittle index(13) since the summation in the denominator becomes nullHowever as shown in (18) when channel state is not the bestthe index value goes to zero

ASPIM119899 =119873

= lim120573rarr1

120583(119886119899)

sdot MOS (119889 + 1) (1 minus 120573)sum119898gt119899

119902119898(120583(119886119898)

minus 120583(119886119899)

)997888rarr 0

(18)

Hence a user in its best channel state with nonnullinstantaneous normalized MOS function has priority overa user which is not in its best channel condition Thus wesummarize the proposed QoE-aware Whittle-based ASPIMindex rule in Definition 5

Definition 5 The ASPIM index rule consists of at everydecision slot

(i) serving the user in its best channel condition withnonnull instantaneous normalized MOS functionwith the highest value of 120583

(119886119873)sdot MOS(119889 + 1)

(ii) if there is no user in its best channel condition withnonnull instantaneous normalized MOS functionserving the user with the highest value of (120583

(119886119899)sdot

MOS(119889 + 1))sum119898gt119899

119902119898(120583(119886119898)

minus 120583(119886119899)

) (using (18)normalized by (1 minus 120573))

So if channel condition is not the best the index valueis the ratio of the multiplication between the actual servicecompletion and the instantaneous normalizedMOS functionrespect to the expected potential improvement of the comple-tion probability

Therefore in the context of time-varying channels aimedat maximizing average MOS we have proposed ASPIMscheduling algorithm which is a simple and tractable QoE-aware channel-aware and size-based index rule Neverthe-less in order to verify the correct behavior of this heuristicwe analyze its performance in the subsequent section

5 Performance Evaluation

In this section we evaluate the performance of the proposedASPIM index rule (see proposal in Definition 5) To that endwe compare the QoE performance of our heuristic with theone achieved by well-known scheduling strategies in severalsimulation scenarios Below we provide a brief description ofthe scheduling priority rules used in our experiments

(i) Max rate (MR) this opportunistic policy consists inserving the user with the best channel capacity

(ii) Proportional fair (PF) this channel-aware disci-pline serves the user with the highest ratio betweeninstantaneous transmission rate and current attainedthroughput

(iii) Cost and attained service dependant 120583 (119888AS120583)although it might seem counter-intuitive minimizingmean delay does not result in maximizing averageMOS [24] This way in order to compare the per-formance with delay-aware solutions we propose toadapt the classical 119888120583 rule aimed at minimizing meandelay [25] to attained serviceThis adaptation consistsof serving the user with the highest value of 119888120583

(119886119899)

being 119888 the waiting cost paid for every slot while flowtransmission is uncompleted

(iv) GM [16] thisQoE-aware pioneering index rule origi-nally proposed for constant channel capacity choosesthe user with the highest value of 120583

(119886119899)sdot MOS(119889 + 1)

In case of ties we use random tie-breaking ruleConcerning traffic flows in the system we use the Pareto

size distribution given in (1) in simulations whose 120572 and 120574parameters determine mean flow size We consider typicalmean sizes used in wireless networks 05Mb (small traffica web page email) 5Mb (medium traffic a PDF documenta picture etc) and 50Mb (MP3 audio a group of picturesfrom a video sequence etc) User traffic flows arrive to thescheduler according to a Poisson process with 120582 rate

Referring to network characteristics we use transmis-sion rates that depend on CQIs used in real wirelessnetworks which are associated with 4G modulation andcoding schemes We show the mapping between CQI valuesand transmission rates in Table 1 The presented values areadapted and extrapolated from [26] for a cell of 20MHz

Moreover we consider different network loads in order toanalyze the behavior of scheduling strategies under differentnetwork conditions Flow arrival rate determines networkload 120588 where 120588 = 120582 sdot (E[119883]119904

119873) We consider seven network

states in this study low load (120588 = 025 0375) medium load(120588 = 05 075) and high load (120588 = 085 09 095)

Regarding QoE aspects as typically utilized in perceivedquality studies [27] we use a log-shaped MOS function forQoE degradation presented in (19) This way employing theMOS curve modelling provided in [21] as starting point andtaking into account the flow size the network type and theuser expertise we have suitably defined 119889min and 119889max delaythresholds used in simulations and consequently 119888

1and 1198882

MOS coefficients Consider

MOS = 1198881minus 1198882sdot log (119889) (19)

We have implemented the whole network environmentfor the scheduling of network resources in MATLAB Inrelation to the performed experiments for each scenario

Abstract and Applied Analysis 9

Table 2 Parameter set in the experimental scenarios

Scenario MOS Size Channel 119888

1 119889min = 05 s 119889max = 10 s 120572 = 15 120574 = 4 sdot 10minus7 CQI = 3 5 11198881= 407 119888

2= 133 E[119883] = 5Mbit 119902

2= 05

2 119889min = 04 s 119889max = 06 s 120572 = 15 120574 = 4 sdot 10minus7 CQI = 3 5 11198881= minus403 119888

2= 986 E[119883] = 5Mbit 119902

2= 05

3 119889min = 05 s 119889max = 10 s 120572 = 15 120574 = 4 sdot 10minus7 CQI = 3 9 11198881= 407 119888

2= 133 E[119883] = 5Mbit 119902

2= 05

4 119889min = 005 s 119889max = 1 s 120572 = 15 120574 = 4 sdot 10minus6 CQI = 3 5 11198881= 1 1198882= 133 E[119883] = 05Mbit 119902

2= 05

5

119889min1 = 05 s 119889max1 = 5 s 1205721= 15 120574

1= 4 sdot 10

minus7

see Table 3 5 111988811= 379 119888

21= 173 E[119883

1] = 5Mbit

119889min2 = 5 s 119889max2 = 30 s 1205722= 15 120574

2= 4 sdot 10

minus8

11988812= 859 119888

22= 223 E[119883

2] = 50Mbit

Table 3 Channel state probabilities in Scenario 5

CQI 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15119902 028 012 009 008 008 008 007 006 005 004 003 001 0009 00005 00003 00002

for each combination of scheduling discipline and networkload we have carried out a set of 10 rounds of 10000 slength simulations These rounds differ in the randomlypregenerated traces (input vector) of sizes and arrivals Thusnot only the overall average MOS values will be provided butwe will also include their 95 confidence intervals

Besides in order to guarantee that the obtained perfor-mance results are generally valid we will analyze differentsimulation scenarios We have chosen five relevant settingswhich differ inQoE size channel orand cost characteristicsThe parameters of these scenarios are summarized in Table 2Note that in the first four scenarios we restrict ourselves tothe case of two channel conditions so as to get fundamentalperformance results

We have carefully selected the scenarios used in simula-tions First of all we define a setting (Scenario 1) in which thevalues of all the parameters are typical or standard the valuesof these parameters introduce low error in the approxima-tions considered for our ASPIM scheduler proposal Thenin order to evaluate worst performance cases for ASPIMwe have chosen three scenarios (Scenario 2 Scenario 3 andScenario 4) in which the error of the approximation used forASPIM is high due to a different factor (gradient ofMOS rateand size resp)

Apart from that it is worthmentioning that for obtainingthe ASPIM index rule that we have considered an isolatedMDP model per user Hence the achieved solution is basedon a unique user type or single class However it would beinteresting to examine the proposal when there are differenttypes of users which is analyzed in Scenario 5 Furthermorethis last setting resembles a real 4G wireless network Notethat even though this technology allows the simultaneoustransmission of multiple flows per TTI we assume that asingle flow transmits in each TTI

03 04 05 06 07 08 09

25

3

35

4

45

5

E[M

OS]

ASPIMMRPF

cAS120583GM

120588

Figure 5 E[MOS] for the basic scenario

Next we describe the results we have achieved in theaforementioned scenarios

51 Scenario 1 Basic Case In this first family of simulationswe consider a basic or typical scenario which takes intoaccount the equiprobable channel case and medium-sizedself-similar flows Figure 5 shows averageMOS results for thissetting employing different scheduling policies and networkloadsThemain conclusion achieved from these results is thatASPIM outperforms the rest of disciplines which is betterthan the QoE-aware GM rule proposed for constant channelcapacity Moreover contrary to the other policies it holds

10 Abstract and Applied Analysis

0 02 04 06 08 10

1

2

3

4

d (s)

PDF(d

)

(a)

1 2 3 4 50

01

02

03

04

05

06

07

MOS

PDF(

MO

S)

ASPIMMRPF

cAS120583GM

(b)

1 2 3 4 50

02

04

06

08

1

MOS

CCD

F(M

OS)

ASPIMMRPF

cAS120583GM

Satisth

(c)

Figure 6 Statistics in Scenario 1 delay PDF (a) MOS PDF (b) and MOS CCDF (c)

averageMOSvalues in the same level for all120588 Referring to thetraditional channel-aware disciplines both PF and MR givebad behavior comparing to the rest of scheduling strategies

In order to justify the previous performance results wenow analyze delay andMOS statistics for the highest networkload considered In such a way as regarded in Figure 6(a) forthe Probability Density Function (PDF) of delay we highlightthe probabilitymass or peak that appears for some schedulingdisciplines before the low delay threshold (05 s) The heightand weight of a peak belonging to a policy have direct impacton its average MOS ASPIM discipline shows the highestand the widest peak among the presented ones followedby GM but with a notable difference Therefore our novelQoE-aware policy seems to try to finish the transmissionof most of the flows before 119889min which is reflected in theaforementioned peaks Note that the cAS120583 rule aimed atminimizing delay behaves in different manner which showsmore fairness among delays in the range shown compared tothe QoE-aware ones

What is previously concluded about delay statistics hasa direct reflection on QoE statistics This way as can beseen in the PDF of MOS presented in Figure 6(b) if theobjective is to maximize average QoE it is important onthe one hand to be the height and the weight of the peaksaround the maximum MOS high and on the other hand tobe the value around the minimum MOS low ASPIM fulfilsthese positive properties and consequently it is the bestin average QoE terms Besides if we analyze the Comple-mentary Cumulative Distribution Function (CCDF) of MOSprovided in Figure 6(c) we observe that ASPIM gives thehighest probability of being MOS value higher Consideringfor example a QoE satisfaction threshold satisth of MOS =4 in this point ASPIM improves GM a 10 whereas thedeterioration of MR and PF is higher than a 50

52 Scenario 2 High Demanding Users Case This scenarioreflects the case of users with low tolerance to delay in

Abstract and Applied Analysis 11

03 04 05 06 07 08 09

05

06

07

08

09

ASPIMMRPF

cAS120583GM

120588

E[M

OS]

2E

[MO

S]1

Figure 7 E[MOS] for abrupt MOS scenario

which theMOS function in theQoE degradation range showsa great slope Thus in this setting the error of the MOSapproximation (15) is high since the gradient of MOS amongconsecutive delay levels becomes significant In Figure 7we compare the results of this second scenario and thebasic scenario From the illustrated graph we could say thatour QoE-aware proposal adapts to any MOS function Thisis whereas for the rest of disciplines (excluding GM) thenegative effects due to the abrupt degradation of the QoEutility function are appreciable ASPIMmanages to avoid thisadapting to any MOS function

53 Scenario 3 Higher Error Case due to Rate During thesimplification process carried out for obtaining the ASPIMindex rule in Section 41 the 120583 approximation used (see (14))introduces a rate dependant error Note that increasing therate in the good channel state makes bigger the error of the 120583approximation used Therefore we will analyze the obtainedperformance for our new QoE-aware proposal under thishigher error due to rate In the previous scenarios we haveassumed that the transmission rate in the good channelcondition is twice the rate in the bad channel state In thissetting we consider that the transmission rate in the goodchannel is four times the transmission rate in the bad channelThis way according to the QoE results presented in Figure 8in spite of a higher error caused by rate ASPIM is superior tothe rest of policies

54 Scenario 4 High Error Case due to Size Besides it isknown that for a distribution with a decreasing hazard ratesuch as Pareto that the gradient of completion probabilityincreases as long as the mean size decreases which makesthe error of the 120583 approximation (14) for the ASPIM ruleachievement bigger In this way in this scenario we verifythat ASPIM gives satisfactory behavior under a more notableerror due to size where we consider a mean size ten times

03 04 05 06 07 08 09

3

35

4

45

5

E[M

OS]

ASPIMMRPF

cAS120583GM

120588

Figure 8 E[MOS] under higher error due to rate

03 04 05 06 07 08 093

35

4

45

5E

[MO

S]

ASPIMMRPF

cAS120583GM

120588

Figure 9 E[MOS] under high error due to size

lower We conclude from Figure 9 that even under a higherror caused by size that ASPIM shows good performancebeing the best alternative

55 Scenario 5 A Real Network Case This last scenarioreflects a real 4G wireless network context but with thesimplification that a single user transmits in each TTI To thatend we use CQI traces obtained from a system-level radioaccess simulator [28] considering users moving at 5 kmhin the cell For this setting with dynamic users we definechannel propagation by the Extended Pedestrian Amodel Asin current 4G networks we consider 16 channel conditionsThe state probabilities of these channel states are collectedin Table 3 which are decreasing with the improvement ofchannel quality In this scenario we consider two classes ofusers which differ in size QoE and cost The mean size

12 Abstract and Applied Analysis

03 04 05 06 07 08 09

1

2

3

4

5

E[M

OS]

120588

(a)

03 04 05 06 07 08 091

2

3

4

5

E[M

OS]

ASPIMMRPF

cAS120583GM

120588

(b)

03 04 05 06 07 08 091

2

3

4

5

E[M

OS]

ASPIMMRPF

cAS120583GM

120588

(c)

Figure 10 E[MOS] for 4G scenario aggregate (a) class 1 (b) class 2 (c)

in class 2 is ten times bigger than in class 1 and we givepriority to the small-sized class by having bigger cost andQoErequirement

Under this mixture of classes and realistic channel mod-els in Figure 10(a) we observe that scheduling strategiesshow the same tendency as in the previous scenarios It isremarkable that the difference between ASPIM and GM isbigger especially for high 120588 values Moreover under thehighest network load ASPIM achieves MOS values aboveQoE satisfaction threshold (MOS = 4) Furthermore averageMOS values forMR and PF dramatically decrease comparingwith the previous settings

Concerning class behaviour we show per-class resultsin Figures 10(b) and 10(c) As can be concluded fromthe presented results QoE-aware policies guarantee fairnessamong classes and ASPIM is the best choice also inside classAside from that for the cost dependant cAS120583 discipline thedeterioration of the class with no priority is significant

6 Conclusions

This paper goes into detail about obtaining a schedulingalgorithmaimed atmaximizing usersrsquo perception of quality inchannels with time-varying capacity Furthermore this workconsiders traffic flowswith nonexponential size distributionscontrary to previous approaches that assume memorylessdistributions

As first contribution we provide a MDP-based modelfor the scheduling problem presented This model combinesQoE-awareness size-awareness and channel-awarenessNev-ertheless since this model can not be solved either analyt-ically or computationally from the proposed MDP modelwe derive a simple tractable and implementable well-performing scheduling index rule by applying amethodologybased on Whittle approach

The proposed scheduling solution still combines QoE-size- and channel-awareness Our scheduling proposal givespriority to users in their best channel condition that are

Abstract and Applied Analysis 13

out of the service unavoidable range As verified fromsimulation results the proposed scheduling strategy showssuitable subjective quality performance in a wide range ofscenarios including the case of a simplified 4G network withheterogeneous users

Therefore the results of this work present a relevantmathematical basis for developing scheduling algorithmsaimed at maximizing QoE in current and future networkswith time-varying channel capacity This way the proposedchannel-aware ASPIM discipline will be useful for networkoperators in order to guarantee to their customers servicesatisfaction in time-varying wireless networks Apart fromthat the approximations and simplifications used in Whittlemethod are applicable to any area when the gradient ofsystem components is considerably small among consecutivedecision slots

As future research we will extend our work to amultiuserapproach considering the simultaneous transmission ofmul-tiple flows per TTI

Appendix

Proof of Proposition 4

LemmaA1 Suppose that states (119886 119889+1119898 le 119899)cup(119886+119903119899 119889+

1119898 le 119899) are passive and that states (119886 119889 + 1119898 gt 119899) cup (119886 +

119903119899 119889 + 1119898 gt 119899) are active and

Vlowast(119886119889119899)

=120573120583(119886119899)

( MOS (119889 + 1) minus 119877119886119889)

1 minus 120573120583(119886119899)

119882119886119889

(A1)

Along with this if we assume that 119877119886119889

= sum119898119902119898119877(119886119889119898)

asymp

sum119898119902119898119877(119886+119903119897 119889+1119898)

and 119882119886119889

= sum119898119902119898119882(119886119889119898)

asymp

sum119898119902119898119882(119886+119903119897 119889+1119898)

for any 119897 = 1 2 119873 value for theundiscounted case

Vlowast(119886119889119899)

= lim120573rarr1

120573120583(119886119899)

sdot MOS (119889 + 1) (1 minus 120573)1 minus 120573 + 120573sum

119898gt119899119902119898(120583(119886119898)

minus 120583(119886119899)

) (A2)

Proof Referring to work elements using (12)

119882(119886119889119898le119899)

= 120573sum

119898

1199021015840

119898119882(119886119889+1119898

1015840)= 120573119882

119886119889

119882(119886119889119898gt119899)

= 1 + 120573 (1 minus 120583(119886119898)

)sum

119898

1199021015840

119898119882(119886+119903119898 119889+1119898

1015840)

= 1 + 120573 (1 minus 120583(119886119898)

)119882119886119889

(A3)

And by (A3) expressions

119882119886119889= (1 minus sum

119898gt119899

119902119898)120573119882

119886119889+ sum

119898gt119899

119902119898(1 + 120573 (1 minus 120583

(119886119898))119882119886119889)

(A4)

This way isolating119882119886119889

119882119886119889=

sum119898gt119899

119902119898

1 minus 120573 + 120573sum119898gt119899

119902119898120583(119886119898)

(A5)

Analogously for reward elements using (11)

119877(119886119889119898le119899)

= 120573sum

119898

1199021015840

119898119877(119886119889+1119898

1015840)= 120573119877119886119889

119877(119886119889119898gt119899)

= 120573[(1 minus 120583(119886119898)

)sum

119898

1199021015840

119898119877(119886+119903119898 119898

1015840)+ 120583(119886119898)

sdot MOS (119889 + 1)]

= 120573 [(1 minus 120583(119886119898)

) 119877119886119889+ 120583(119886119898)

sdot MOS (119889 + 1)] (A6)

And thus from (A6)

119877119886119889= (1 minus sum

119898gt119899

119902119898)120573119877119886119889

+ sum

119898gt119899

119902119898(120573 [(1 minus 120583

(119886119898)) 119877119886119889+ 120583(119886119898)

sdot MOS (119889 + 1)])

(A7)

Isolating 119877119886119889

119877119886119889=120573sum119898gt119899

119902119898120583(119886119898)

MOS (119889 + 1)1 minus 120573 + 120573sum

119898gt119899119902119898120583(119886119898)

(A8)

Substituting (A5) and (A8) in (A1) and by simplifyingwe obtain

Vlowast(119886119889119899)

= (120573120583(119886119899)

(MOS (119889 + 1)

minus120573sum119898gt119899

119902119898120583(119886119898)

MOS (119889 + 1)1 minus 120573 + 120573sum

119898gt119899119902119898120583(119886119898)

))

times (1 minus 120573120583(119886119899)

sum119898gt119899

119902119898

1 minus 120573 + 120573sum119898gt119899

119902119898120583(119886119898)

)

minus1

= (120573120583(119886119899)

(MOS (119889 + 1) (1 minus 120573 + 120573sum119898gt119899

119902119898120583(119886119898)

)

minus120573sum

119898gt119899

119902119898120583(119886119898)

MOS (119889 + 1)))

times (1 minus 120573 + 120573sum

119898gt119899

119902119898120583(119886119898)

minus 120573120583(119886119899)

sum

119898gt119899

119902119898)

minus1

=120573120583(119886119899)

sdot MOS (119889 + 1) (1 minus 120573)1 minus 120573 + 120573sum

119898gt119899119902119898(120583(119886119898)

minus 120583(119886119899)

)

(A9)

And therefore for the undiscounted case the 120573 rarr 1

limit for expression (A9) is (A2)

Conflict of Interests

The authors declare that there is no conflict of interestsregarding the publication of this paper

14 Abstract and Applied Analysis

Acknowledgment

The research leading to these results has received fundingfrom the European Union Seventh Framework Programme(FP72007ndash2013) under Grant agreement 284863 (FP7 SECGERYON)

References

[1] ITU-T P10G100 Amendment 1 (0107) New Appendix I-definition of Quality of Experience (QoE) 2006

[2] S Aalto and P Lassila ldquoFlow-level stability and performance ofchannel-aware priority-based schedulersrdquo in Proceeedings of the6th Euro NF Conference on Next Generation Internet (NGI rsquo10)IEEE June 2010

[3] L KleinrockQueueing Systems vol 2 ofComputer ApplicationsJohn Wiley amp Sons 1976

[4] J Gittins ldquoBandit processes and dynamic allocation indicesrdquoJournal of the Royal Statistical Society B Methodological vol 41no 2 pp 148ndash177 1979

[5] K Avrachenkovt U Ayesta P Brown and E Nyberg ldquoDif-ferentiation between short and long TCP ows predictabilityof the response timerdquo in Proceedings of the 23rd Annual JointConference of the IEEE Computer and Communications Societies(INFOCOM rsquo04) vol 2 p 762 IEEE 2004

[6] U Ayesta M Erausquin and P Jacko ldquoA modeling frameworkfor optimizing the flow-level scheduling with time-varyingchannelsrdquo Performance Evaluation vol 67 no 11 pp 1014ndash10292010

[7] S Aalto U Ayesta and R Righter ldquoProperties of the Gittinsindex with application to optimal schedulingrdquo Probability in theEngineering and Informational Sciences vol 25 no 3 pp 269ndash288 2011

[8] M Puterman ldquoMarkov decision processesrdquo in Handbooks inOperations Research and Management Science vol 2 pp 331ndash434 1990

[9] C H Papadimitriou and J N Tsitsiklis ldquoThe complexityof Markov decision processesrdquo Mathematics of OperationsResearch vol 12 no 3 pp 441ndash450 1987

[10] J Nino-Mora ldquoDynamic priority allocation via restless banditmarginal productivity indicesrdquo TOP vol 15 no 2 pp 161ndash1982007

[11] P Whittle ldquoRestless bandits activity allocation in a changingworldrdquo Journal of Applied Probability vol 25 pp 287ndash298 1988

[12] U Ayesta M Erausquin M Jonckheere and I M VerloopldquoScheduling in a random environment stability and asymptoticoptimalityrdquo IEEEACM Transactions on Networking vol 21 no1 pp 258ndash271 2013

[13] S Khan S Thakolsri E Steinbach and W Kellerer ldquoQoE-based cross-layer optimization for wireless multiuser systemsrdquoin Proceedings of the 18th ITC Specialist Seminar on Quality ofExperience 2008

[14] S Thakolsri S Khan E Steinbach and W Kellerer ldquoQoE-driven cross-layer optimization for high speed downlink packetaccessrdquo Journal of Communications vol 4 no 9 pp 669ndash6802009

[15] P Ameigeiras J J Ramos-Munoz J Navarro-Ortiz PMogensen and J M Lopez-Soler ldquoQoE oriented cross-layer design of a resource allocation algorithm in beyond 3GsystemsrdquoComputer Communications vol 33 no 5 pp 571ndash5822010

[16] I Taboada F Liberal J O Fajardo and U Ayesta ldquoQoE-aware optimization of multimedia flow schedulingrdquo ComputerCommunications vol 36 no 15-16 pp 1629ndash1638 2013

[17] U Ayesta M Erausquin and P Jacko ldquoResource-sharing in asingle server with time-varying capacityrdquo in Proceeding of the49th Annual Allerton Conference on Communication Controland Computing (Allerton rsquo11) pp 377ndash383 Monticello Ill USASeptember 2011

[18] K Thompson G J Miller and R Wilder ldquoWide-area internettraffic patterns and characteristicsrdquo IEEE Network vol 11 no 6pp 10ndash23 1997

[19] M Fiedler T Hossfeld and P Tran-Gia ldquoA generic quantitativerelationship between quality of experience and quality ofservicerdquo IEEE Network vol 24 no 2 pp 36ndash41 2010

[20] A Richards M Antoniades V Witana and G Rogers ldquoMap-ping user level QoS from a single parameterrdquo in Proceedingsof the International Conference on Multimedia Networks andServices Citeseer 1998

[21] E Ibarrola F Liberal I Taboada and R Ortega ldquoWebQoE evaluation in multi-agent networks validation of ITU-TG1030rdquo in Proceedings of the 5th International Conference onAutonomic and Autonomous Systems (ICAS rsquo09) pp 289ndash294Valencia Spain April 2009

[22] S Aalto and U Ayesta ldquoOptimal scheduling of jobs with aDHR tail in the MG1 queuerdquo in Proceedings of the 3rd Inter-national Conference on Performance Evaluation Methodologiesand Tools (ValueTools rsquo08) article 50 Institute for ComputerSciences Social-Informatics and Telecommunications Engi-neering Athens Greece 2008

[23] V Visweswaran ldquoDecomposition techniques for MILPlagrangian relaxationrdquo in Encyclopedia of Optimization pp632ndash638 Springer New York NY USA 2009

[24] I Taboada J O Fajardo and F Liberal ldquoPerformance analysisof scheduling algorithms for web QoE optimization in wirelessnetworksrdquo Network Protocols and Algorithms vol 4 pp 27ndash432012

[25] J A van Mieghem ldquoDynamic scheduling with convex delaycosts the generalized 119888120583 rulerdquoTheAnnals of Applied Probabilityvol 5 no 3 pp 809ndash833 1995

[26] S Sesia I Toufik and M Baker LTE-The UMTS Long TermEvolution FromTheory to Practice Wiley 2011

[27] P Reichl B Tuffin and R Schatz ldquoLogarithmic laws inservice quality perception where microeconomics meets psy-chophysics and quality of experiencerdquo Telecommunication Sys-tems vol 52 no 2 pp 587ndash600 2013

[28] J C Ikuno M Wrulich and M Rupp ldquoSystem level simulationof LTE networksrdquo in Proceedings of the IEEE 71st VehicularTechnology Conference Taipei Taiwan May 2010

Submit your manuscripts athttpwwwhindawicom

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical Problems in Engineering

Hindawi Publishing Corporationhttpwwwhindawicom

Differential EquationsInternational Journal of

Volume 2014

Applied MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical PhysicsAdvances in

Complex AnalysisJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

OptimizationJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Operations ResearchAdvances in

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Function Spaces

Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of Mathematics and Mathematical Sciences

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Algebra

Discrete Dynamics in Nature and Society

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Decision SciencesAdvances in

Discrete MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Stochastic AnalysisInternational Journal of

Abstract and Applied Analysis 9

Table 2 Parameter set in the experimental scenarios

Scenario MOS Size Channel 119888

1 119889min = 05 s 119889max = 10 s 120572 = 15 120574 = 4 sdot 10minus7 CQI = 3 5 11198881= 407 119888

2= 133 E[119883] = 5Mbit 119902

2= 05

2 119889min = 04 s 119889max = 06 s 120572 = 15 120574 = 4 sdot 10minus7 CQI = 3 5 11198881= minus403 119888

2= 986 E[119883] = 5Mbit 119902

2= 05

3 119889min = 05 s 119889max = 10 s 120572 = 15 120574 = 4 sdot 10minus7 CQI = 3 9 11198881= 407 119888

2= 133 E[119883] = 5Mbit 119902

2= 05

4 119889min = 005 s 119889max = 1 s 120572 = 15 120574 = 4 sdot 10minus6 CQI = 3 5 11198881= 1 1198882= 133 E[119883] = 05Mbit 119902

2= 05

5

119889min1 = 05 s 119889max1 = 5 s 1205721= 15 120574

1= 4 sdot 10

minus7

see Table 3 5 111988811= 379 119888

21= 173 E[119883

1] = 5Mbit

119889min2 = 5 s 119889max2 = 30 s 1205722= 15 120574

2= 4 sdot 10

minus8

11988812= 859 119888

22= 223 E[119883

2] = 50Mbit

Table 3 Channel state probabilities in Scenario 5

CQI 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15119902 028 012 009 008 008 008 007 006 005 004 003 001 0009 00005 00003 00002

for each combination of scheduling discipline and networkload we have carried out a set of 10 rounds of 10000 slength simulations These rounds differ in the randomlypregenerated traces (input vector) of sizes and arrivals Thusnot only the overall average MOS values will be provided butwe will also include their 95 confidence intervals

Besides in order to guarantee that the obtained perfor-mance results are generally valid we will analyze differentsimulation scenarios We have chosen five relevant settingswhich differ inQoE size channel orand cost characteristicsThe parameters of these scenarios are summarized in Table 2Note that in the first four scenarios we restrict ourselves tothe case of two channel conditions so as to get fundamentalperformance results

We have carefully selected the scenarios used in simula-tions First of all we define a setting (Scenario 1) in which thevalues of all the parameters are typical or standard the valuesof these parameters introduce low error in the approxima-tions considered for our ASPIM scheduler proposal Thenin order to evaluate worst performance cases for ASPIMwe have chosen three scenarios (Scenario 2 Scenario 3 andScenario 4) in which the error of the approximation used forASPIM is high due to a different factor (gradient ofMOS rateand size resp)

Apart from that it is worthmentioning that for obtainingthe ASPIM index rule that we have considered an isolatedMDP model per user Hence the achieved solution is basedon a unique user type or single class However it would beinteresting to examine the proposal when there are differenttypes of users which is analyzed in Scenario 5 Furthermorethis last setting resembles a real 4G wireless network Notethat even though this technology allows the simultaneoustransmission of multiple flows per TTI we assume that asingle flow transmits in each TTI

03 04 05 06 07 08 09

25

3

35

4

45

5

E[M

OS]

ASPIMMRPF

cAS120583GM

120588

Figure 5 E[MOS] for the basic scenario

Next we describe the results we have achieved in theaforementioned scenarios

51 Scenario 1 Basic Case In this first family of simulationswe consider a basic or typical scenario which takes intoaccount the equiprobable channel case and medium-sizedself-similar flows Figure 5 shows averageMOS results for thissetting employing different scheduling policies and networkloadsThemain conclusion achieved from these results is thatASPIM outperforms the rest of disciplines which is betterthan the QoE-aware GM rule proposed for constant channelcapacity Moreover contrary to the other policies it holds

10 Abstract and Applied Analysis

0 02 04 06 08 10

1

2

3

4

d (s)

PDF(d

)

(a)

1 2 3 4 50

01

02

03

04

05

06

07

MOS

PDF(

MO

S)

ASPIMMRPF

cAS120583GM

(b)

1 2 3 4 50

02

04

06

08

1

MOS

CCD

F(M

OS)

ASPIMMRPF

cAS120583GM

Satisth

(c)

Figure 6 Statistics in Scenario 1 delay PDF (a) MOS PDF (b) and MOS CCDF (c)

averageMOSvalues in the same level for all120588 Referring to thetraditional channel-aware disciplines both PF and MR givebad behavior comparing to the rest of scheduling strategies

In order to justify the previous performance results wenow analyze delay andMOS statistics for the highest networkload considered In such a way as regarded in Figure 6(a) forthe Probability Density Function (PDF) of delay we highlightthe probabilitymass or peak that appears for some schedulingdisciplines before the low delay threshold (05 s) The heightand weight of a peak belonging to a policy have direct impacton its average MOS ASPIM discipline shows the highestand the widest peak among the presented ones followedby GM but with a notable difference Therefore our novelQoE-aware policy seems to try to finish the transmissionof most of the flows before 119889min which is reflected in theaforementioned peaks Note that the cAS120583 rule aimed atminimizing delay behaves in different manner which showsmore fairness among delays in the range shown compared tothe QoE-aware ones

What is previously concluded about delay statistics hasa direct reflection on QoE statistics This way as can beseen in the PDF of MOS presented in Figure 6(b) if theobjective is to maximize average QoE it is important onthe one hand to be the height and the weight of the peaksaround the maximum MOS high and on the other hand tobe the value around the minimum MOS low ASPIM fulfilsthese positive properties and consequently it is the bestin average QoE terms Besides if we analyze the Comple-mentary Cumulative Distribution Function (CCDF) of MOSprovided in Figure 6(c) we observe that ASPIM gives thehighest probability of being MOS value higher Consideringfor example a QoE satisfaction threshold satisth of MOS =4 in this point ASPIM improves GM a 10 whereas thedeterioration of MR and PF is higher than a 50

52 Scenario 2 High Demanding Users Case This scenarioreflects the case of users with low tolerance to delay in

Abstract and Applied Analysis 11

03 04 05 06 07 08 09

05

06

07

08

09

ASPIMMRPF

cAS120583GM

120588

E[M

OS]

2E

[MO

S]1

Figure 7 E[MOS] for abrupt MOS scenario

which theMOS function in theQoE degradation range showsa great slope Thus in this setting the error of the MOSapproximation (15) is high since the gradient of MOS amongconsecutive delay levels becomes significant In Figure 7we compare the results of this second scenario and thebasic scenario From the illustrated graph we could say thatour QoE-aware proposal adapts to any MOS function Thisis whereas for the rest of disciplines (excluding GM) thenegative effects due to the abrupt degradation of the QoEutility function are appreciable ASPIMmanages to avoid thisadapting to any MOS function

53 Scenario 3 Higher Error Case due to Rate During thesimplification process carried out for obtaining the ASPIMindex rule in Section 41 the 120583 approximation used (see (14))introduces a rate dependant error Note that increasing therate in the good channel state makes bigger the error of the 120583approximation used Therefore we will analyze the obtainedperformance for our new QoE-aware proposal under thishigher error due to rate In the previous scenarios we haveassumed that the transmission rate in the good channelcondition is twice the rate in the bad channel state In thissetting we consider that the transmission rate in the goodchannel is four times the transmission rate in the bad channelThis way according to the QoE results presented in Figure 8in spite of a higher error caused by rate ASPIM is superior tothe rest of policies

54 Scenario 4 High Error Case due to Size Besides it isknown that for a distribution with a decreasing hazard ratesuch as Pareto that the gradient of completion probabilityincreases as long as the mean size decreases which makesthe error of the 120583 approximation (14) for the ASPIM ruleachievement bigger In this way in this scenario we verifythat ASPIM gives satisfactory behavior under a more notableerror due to size where we consider a mean size ten times

03 04 05 06 07 08 09

3

35

4

45

5

E[M

OS]

ASPIMMRPF

cAS120583GM

120588

Figure 8 E[MOS] under higher error due to rate

03 04 05 06 07 08 093

35

4

45

5E

[MO

S]

ASPIMMRPF

cAS120583GM

120588

Figure 9 E[MOS] under high error due to size

lower We conclude from Figure 9 that even under a higherror caused by size that ASPIM shows good performancebeing the best alternative

55 Scenario 5 A Real Network Case This last scenarioreflects a real 4G wireless network context but with thesimplification that a single user transmits in each TTI To thatend we use CQI traces obtained from a system-level radioaccess simulator [28] considering users moving at 5 kmhin the cell For this setting with dynamic users we definechannel propagation by the Extended Pedestrian Amodel Asin current 4G networks we consider 16 channel conditionsThe state probabilities of these channel states are collectedin Table 3 which are decreasing with the improvement ofchannel quality In this scenario we consider two classes ofusers which differ in size QoE and cost The mean size

12 Abstract and Applied Analysis

03 04 05 06 07 08 09

1

2

3

4

5

E[M

OS]

120588

(a)

03 04 05 06 07 08 091

2

3

4

5

E[M

OS]

ASPIMMRPF

cAS120583GM

120588

(b)

03 04 05 06 07 08 091

2

3

4

5

E[M

OS]

ASPIMMRPF

cAS120583GM

120588

(c)

Figure 10 E[MOS] for 4G scenario aggregate (a) class 1 (b) class 2 (c)

in class 2 is ten times bigger than in class 1 and we givepriority to the small-sized class by having bigger cost andQoErequirement

Under this mixture of classes and realistic channel mod-els in Figure 10(a) we observe that scheduling strategiesshow the same tendency as in the previous scenarios It isremarkable that the difference between ASPIM and GM isbigger especially for high 120588 values Moreover under thehighest network load ASPIM achieves MOS values aboveQoE satisfaction threshold (MOS = 4) Furthermore averageMOS values forMR and PF dramatically decrease comparingwith the previous settings

Concerning class behaviour we show per-class resultsin Figures 10(b) and 10(c) As can be concluded fromthe presented results QoE-aware policies guarantee fairnessamong classes and ASPIM is the best choice also inside classAside from that for the cost dependant cAS120583 discipline thedeterioration of the class with no priority is significant

6 Conclusions

This paper goes into detail about obtaining a schedulingalgorithmaimed atmaximizing usersrsquo perception of quality inchannels with time-varying capacity Furthermore this workconsiders traffic flowswith nonexponential size distributionscontrary to previous approaches that assume memorylessdistributions

As first contribution we provide a MDP-based modelfor the scheduling problem presented This model combinesQoE-awareness size-awareness and channel-awarenessNev-ertheless since this model can not be solved either analyt-ically or computationally from the proposed MDP modelwe derive a simple tractable and implementable well-performing scheduling index rule by applying amethodologybased on Whittle approach

The proposed scheduling solution still combines QoE-size- and channel-awareness Our scheduling proposal givespriority to users in their best channel condition that are

Abstract and Applied Analysis 13

out of the service unavoidable range As verified fromsimulation results the proposed scheduling strategy showssuitable subjective quality performance in a wide range ofscenarios including the case of a simplified 4G network withheterogeneous users

Therefore the results of this work present a relevantmathematical basis for developing scheduling algorithmsaimed at maximizing QoE in current and future networkswith time-varying channel capacity This way the proposedchannel-aware ASPIM discipline will be useful for networkoperators in order to guarantee to their customers servicesatisfaction in time-varying wireless networks Apart fromthat the approximations and simplifications used in Whittlemethod are applicable to any area when the gradient ofsystem components is considerably small among consecutivedecision slots

As future research we will extend our work to amultiuserapproach considering the simultaneous transmission ofmul-tiple flows per TTI

Appendix

Proof of Proposition 4

LemmaA1 Suppose that states (119886 119889+1119898 le 119899)cup(119886+119903119899 119889+

1119898 le 119899) are passive and that states (119886 119889 + 1119898 gt 119899) cup (119886 +

119903119899 119889 + 1119898 gt 119899) are active and

Vlowast(119886119889119899)

=120573120583(119886119899)

( MOS (119889 + 1) minus 119877119886119889)

1 minus 120573120583(119886119899)

119882119886119889

(A1)

Along with this if we assume that 119877119886119889

= sum119898119902119898119877(119886119889119898)

asymp

sum119898119902119898119877(119886+119903119897 119889+1119898)

and 119882119886119889

= sum119898119902119898119882(119886119889119898)

asymp

sum119898119902119898119882(119886+119903119897 119889+1119898)

for any 119897 = 1 2 119873 value for theundiscounted case

Vlowast(119886119889119899)

= lim120573rarr1

120573120583(119886119899)

sdot MOS (119889 + 1) (1 minus 120573)1 minus 120573 + 120573sum

119898gt119899119902119898(120583(119886119898)

minus 120583(119886119899)

) (A2)

Proof Referring to work elements using (12)

119882(119886119889119898le119899)

= 120573sum

119898

1199021015840

119898119882(119886119889+1119898

1015840)= 120573119882

119886119889

119882(119886119889119898gt119899)

= 1 + 120573 (1 minus 120583(119886119898)

)sum

119898

1199021015840

119898119882(119886+119903119898 119889+1119898

1015840)

= 1 + 120573 (1 minus 120583(119886119898)

)119882119886119889

(A3)

And by (A3) expressions

119882119886119889= (1 minus sum

119898gt119899

119902119898)120573119882

119886119889+ sum

119898gt119899

119902119898(1 + 120573 (1 minus 120583

(119886119898))119882119886119889)

(A4)

This way isolating119882119886119889

119882119886119889=

sum119898gt119899

119902119898

1 minus 120573 + 120573sum119898gt119899

119902119898120583(119886119898)

(A5)

Analogously for reward elements using (11)

119877(119886119889119898le119899)

= 120573sum

119898

1199021015840

119898119877(119886119889+1119898

1015840)= 120573119877119886119889

119877(119886119889119898gt119899)

= 120573[(1 minus 120583(119886119898)

)sum

119898

1199021015840

119898119877(119886+119903119898 119898

1015840)+ 120583(119886119898)

sdot MOS (119889 + 1)]

= 120573 [(1 minus 120583(119886119898)

) 119877119886119889+ 120583(119886119898)

sdot MOS (119889 + 1)] (A6)

And thus from (A6)

119877119886119889= (1 minus sum

119898gt119899

119902119898)120573119877119886119889

+ sum

119898gt119899

119902119898(120573 [(1 minus 120583

(119886119898)) 119877119886119889+ 120583(119886119898)

sdot MOS (119889 + 1)])

(A7)

Isolating 119877119886119889

119877119886119889=120573sum119898gt119899

119902119898120583(119886119898)

MOS (119889 + 1)1 minus 120573 + 120573sum

119898gt119899119902119898120583(119886119898)

(A8)

Substituting (A5) and (A8) in (A1) and by simplifyingwe obtain

Vlowast(119886119889119899)

= (120573120583(119886119899)

(MOS (119889 + 1)

minus120573sum119898gt119899

119902119898120583(119886119898)

MOS (119889 + 1)1 minus 120573 + 120573sum

119898gt119899119902119898120583(119886119898)

))

times (1 minus 120573120583(119886119899)

sum119898gt119899

119902119898

1 minus 120573 + 120573sum119898gt119899

119902119898120583(119886119898)

)

minus1

= (120573120583(119886119899)

(MOS (119889 + 1) (1 minus 120573 + 120573sum119898gt119899

119902119898120583(119886119898)

)

minus120573sum

119898gt119899

119902119898120583(119886119898)

MOS (119889 + 1)))

times (1 minus 120573 + 120573sum

119898gt119899

119902119898120583(119886119898)

minus 120573120583(119886119899)

sum

119898gt119899

119902119898)

minus1

=120573120583(119886119899)

sdot MOS (119889 + 1) (1 minus 120573)1 minus 120573 + 120573sum

119898gt119899119902119898(120583(119886119898)

minus 120583(119886119899)

)

(A9)

And therefore for the undiscounted case the 120573 rarr 1

limit for expression (A9) is (A2)

Conflict of Interests

The authors declare that there is no conflict of interestsregarding the publication of this paper

14 Abstract and Applied Analysis

Acknowledgment

The research leading to these results has received fundingfrom the European Union Seventh Framework Programme(FP72007ndash2013) under Grant agreement 284863 (FP7 SECGERYON)

References

[1] ITU-T P10G100 Amendment 1 (0107) New Appendix I-definition of Quality of Experience (QoE) 2006

[2] S Aalto and P Lassila ldquoFlow-level stability and performance ofchannel-aware priority-based schedulersrdquo in Proceeedings of the6th Euro NF Conference on Next Generation Internet (NGI rsquo10)IEEE June 2010

[3] L KleinrockQueueing Systems vol 2 ofComputer ApplicationsJohn Wiley amp Sons 1976

[4] J Gittins ldquoBandit processes and dynamic allocation indicesrdquoJournal of the Royal Statistical Society B Methodological vol 41no 2 pp 148ndash177 1979

[5] K Avrachenkovt U Ayesta P Brown and E Nyberg ldquoDif-ferentiation between short and long TCP ows predictabilityof the response timerdquo in Proceedings of the 23rd Annual JointConference of the IEEE Computer and Communications Societies(INFOCOM rsquo04) vol 2 p 762 IEEE 2004

[6] U Ayesta M Erausquin and P Jacko ldquoA modeling frameworkfor optimizing the flow-level scheduling with time-varyingchannelsrdquo Performance Evaluation vol 67 no 11 pp 1014ndash10292010

[7] S Aalto U Ayesta and R Righter ldquoProperties of the Gittinsindex with application to optimal schedulingrdquo Probability in theEngineering and Informational Sciences vol 25 no 3 pp 269ndash288 2011

[8] M Puterman ldquoMarkov decision processesrdquo in Handbooks inOperations Research and Management Science vol 2 pp 331ndash434 1990

[9] C H Papadimitriou and J N Tsitsiklis ldquoThe complexityof Markov decision processesrdquo Mathematics of OperationsResearch vol 12 no 3 pp 441ndash450 1987

[10] J Nino-Mora ldquoDynamic priority allocation via restless banditmarginal productivity indicesrdquo TOP vol 15 no 2 pp 161ndash1982007

[11] P Whittle ldquoRestless bandits activity allocation in a changingworldrdquo Journal of Applied Probability vol 25 pp 287ndash298 1988

[12] U Ayesta M Erausquin M Jonckheere and I M VerloopldquoScheduling in a random environment stability and asymptoticoptimalityrdquo IEEEACM Transactions on Networking vol 21 no1 pp 258ndash271 2013

[13] S Khan S Thakolsri E Steinbach and W Kellerer ldquoQoE-based cross-layer optimization for wireless multiuser systemsrdquoin Proceedings of the 18th ITC Specialist Seminar on Quality ofExperience 2008

[14] S Thakolsri S Khan E Steinbach and W Kellerer ldquoQoE-driven cross-layer optimization for high speed downlink packetaccessrdquo Journal of Communications vol 4 no 9 pp 669ndash6802009

[15] P Ameigeiras J J Ramos-Munoz J Navarro-Ortiz PMogensen and J M Lopez-Soler ldquoQoE oriented cross-layer design of a resource allocation algorithm in beyond 3GsystemsrdquoComputer Communications vol 33 no 5 pp 571ndash5822010

[16] I Taboada F Liberal J O Fajardo and U Ayesta ldquoQoE-aware optimization of multimedia flow schedulingrdquo ComputerCommunications vol 36 no 15-16 pp 1629ndash1638 2013

[17] U Ayesta M Erausquin and P Jacko ldquoResource-sharing in asingle server with time-varying capacityrdquo in Proceeding of the49th Annual Allerton Conference on Communication Controland Computing (Allerton rsquo11) pp 377ndash383 Monticello Ill USASeptember 2011

[18] K Thompson G J Miller and R Wilder ldquoWide-area internettraffic patterns and characteristicsrdquo IEEE Network vol 11 no 6pp 10ndash23 1997

[19] M Fiedler T Hossfeld and P Tran-Gia ldquoA generic quantitativerelationship between quality of experience and quality ofservicerdquo IEEE Network vol 24 no 2 pp 36ndash41 2010

[20] A Richards M Antoniades V Witana and G Rogers ldquoMap-ping user level QoS from a single parameterrdquo in Proceedingsof the International Conference on Multimedia Networks andServices Citeseer 1998

[21] E Ibarrola F Liberal I Taboada and R Ortega ldquoWebQoE evaluation in multi-agent networks validation of ITU-TG1030rdquo in Proceedings of the 5th International Conference onAutonomic and Autonomous Systems (ICAS rsquo09) pp 289ndash294Valencia Spain April 2009

[22] S Aalto and U Ayesta ldquoOptimal scheduling of jobs with aDHR tail in the MG1 queuerdquo in Proceedings of the 3rd Inter-national Conference on Performance Evaluation Methodologiesand Tools (ValueTools rsquo08) article 50 Institute for ComputerSciences Social-Informatics and Telecommunications Engi-neering Athens Greece 2008

[23] V Visweswaran ldquoDecomposition techniques for MILPlagrangian relaxationrdquo in Encyclopedia of Optimization pp632ndash638 Springer New York NY USA 2009

[24] I Taboada J O Fajardo and F Liberal ldquoPerformance analysisof scheduling algorithms for web QoE optimization in wirelessnetworksrdquo Network Protocols and Algorithms vol 4 pp 27ndash432012

[25] J A van Mieghem ldquoDynamic scheduling with convex delaycosts the generalized 119888120583 rulerdquoTheAnnals of Applied Probabilityvol 5 no 3 pp 809ndash833 1995

[26] S Sesia I Toufik and M Baker LTE-The UMTS Long TermEvolution FromTheory to Practice Wiley 2011

[27] P Reichl B Tuffin and R Schatz ldquoLogarithmic laws inservice quality perception where microeconomics meets psy-chophysics and quality of experiencerdquo Telecommunication Sys-tems vol 52 no 2 pp 587ndash600 2013

[28] J C Ikuno M Wrulich and M Rupp ldquoSystem level simulationof LTE networksrdquo in Proceedings of the IEEE 71st VehicularTechnology Conference Taipei Taiwan May 2010

Submit your manuscripts athttpwwwhindawicom

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical Problems in Engineering

Hindawi Publishing Corporationhttpwwwhindawicom

Differential EquationsInternational Journal of

Volume 2014

Applied MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical PhysicsAdvances in

Complex AnalysisJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

OptimizationJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Operations ResearchAdvances in

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Function Spaces

Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of Mathematics and Mathematical Sciences

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Algebra

Discrete Dynamics in Nature and Society

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Decision SciencesAdvances in

Discrete MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Stochastic AnalysisInternational Journal of

10 Abstract and Applied Analysis

0 02 04 06 08 10

1

2

3

4

d (s)

PDF(d

)

(a)

1 2 3 4 50

01

02

03

04

05

06

07

MOS

PDF(

MO

S)

ASPIMMRPF

cAS120583GM

(b)

1 2 3 4 50

02

04

06

08

1

MOS

CCD

F(M

OS)

ASPIMMRPF

cAS120583GM

Satisth

(c)

Figure 6 Statistics in Scenario 1 delay PDF (a) MOS PDF (b) and MOS CCDF (c)

averageMOSvalues in the same level for all120588 Referring to thetraditional channel-aware disciplines both PF and MR givebad behavior comparing to the rest of scheduling strategies

In order to justify the previous performance results wenow analyze delay andMOS statistics for the highest networkload considered In such a way as regarded in Figure 6(a) forthe Probability Density Function (PDF) of delay we highlightthe probabilitymass or peak that appears for some schedulingdisciplines before the low delay threshold (05 s) The heightand weight of a peak belonging to a policy have direct impacton its average MOS ASPIM discipline shows the highestand the widest peak among the presented ones followedby GM but with a notable difference Therefore our novelQoE-aware policy seems to try to finish the transmissionof most of the flows before 119889min which is reflected in theaforementioned peaks Note that the cAS120583 rule aimed atminimizing delay behaves in different manner which showsmore fairness among delays in the range shown compared tothe QoE-aware ones

What is previously concluded about delay statistics hasa direct reflection on QoE statistics This way as can beseen in the PDF of MOS presented in Figure 6(b) if theobjective is to maximize average QoE it is important onthe one hand to be the height and the weight of the peaksaround the maximum MOS high and on the other hand tobe the value around the minimum MOS low ASPIM fulfilsthese positive properties and consequently it is the bestin average QoE terms Besides if we analyze the Comple-mentary Cumulative Distribution Function (CCDF) of MOSprovided in Figure 6(c) we observe that ASPIM gives thehighest probability of being MOS value higher Consideringfor example a QoE satisfaction threshold satisth of MOS =4 in this point ASPIM improves GM a 10 whereas thedeterioration of MR and PF is higher than a 50

52 Scenario 2 High Demanding Users Case This scenarioreflects the case of users with low tolerance to delay in

Abstract and Applied Analysis 11

03 04 05 06 07 08 09

05

06

07

08

09

ASPIMMRPF

cAS120583GM

120588

E[M

OS]

2E

[MO

S]1

Figure 7 E[MOS] for abrupt MOS scenario

which theMOS function in theQoE degradation range showsa great slope Thus in this setting the error of the MOSapproximation (15) is high since the gradient of MOS amongconsecutive delay levels becomes significant In Figure 7we compare the results of this second scenario and thebasic scenario From the illustrated graph we could say thatour QoE-aware proposal adapts to any MOS function Thisis whereas for the rest of disciplines (excluding GM) thenegative effects due to the abrupt degradation of the QoEutility function are appreciable ASPIMmanages to avoid thisadapting to any MOS function

53 Scenario 3 Higher Error Case due to Rate During thesimplification process carried out for obtaining the ASPIMindex rule in Section 41 the 120583 approximation used (see (14))introduces a rate dependant error Note that increasing therate in the good channel state makes bigger the error of the 120583approximation used Therefore we will analyze the obtainedperformance for our new QoE-aware proposal under thishigher error due to rate In the previous scenarios we haveassumed that the transmission rate in the good channelcondition is twice the rate in the bad channel state In thissetting we consider that the transmission rate in the goodchannel is four times the transmission rate in the bad channelThis way according to the QoE results presented in Figure 8in spite of a higher error caused by rate ASPIM is superior tothe rest of policies

54 Scenario 4 High Error Case due to Size Besides it isknown that for a distribution with a decreasing hazard ratesuch as Pareto that the gradient of completion probabilityincreases as long as the mean size decreases which makesthe error of the 120583 approximation (14) for the ASPIM ruleachievement bigger In this way in this scenario we verifythat ASPIM gives satisfactory behavior under a more notableerror due to size where we consider a mean size ten times

03 04 05 06 07 08 09

3

35

4

45

5

E[M

OS]

ASPIMMRPF

cAS120583GM

120588

Figure 8 E[MOS] under higher error due to rate

03 04 05 06 07 08 093

35

4

45

5E

[MO

S]

ASPIMMRPF

cAS120583GM

120588

Figure 9 E[MOS] under high error due to size

lower We conclude from Figure 9 that even under a higherror caused by size that ASPIM shows good performancebeing the best alternative

55 Scenario 5 A Real Network Case This last scenarioreflects a real 4G wireless network context but with thesimplification that a single user transmits in each TTI To thatend we use CQI traces obtained from a system-level radioaccess simulator [28] considering users moving at 5 kmhin the cell For this setting with dynamic users we definechannel propagation by the Extended Pedestrian Amodel Asin current 4G networks we consider 16 channel conditionsThe state probabilities of these channel states are collectedin Table 3 which are decreasing with the improvement ofchannel quality In this scenario we consider two classes ofusers which differ in size QoE and cost The mean size

12 Abstract and Applied Analysis

03 04 05 06 07 08 09

1

2

3

4

5

E[M

OS]

120588

(a)

03 04 05 06 07 08 091

2

3

4

5

E[M

OS]

ASPIMMRPF

cAS120583GM

120588

(b)

03 04 05 06 07 08 091

2

3

4

5

E[M

OS]

ASPIMMRPF

cAS120583GM

120588

(c)

Figure 10 E[MOS] for 4G scenario aggregate (a) class 1 (b) class 2 (c)

in class 2 is ten times bigger than in class 1 and we givepriority to the small-sized class by having bigger cost andQoErequirement

Under this mixture of classes and realistic channel mod-els in Figure 10(a) we observe that scheduling strategiesshow the same tendency as in the previous scenarios It isremarkable that the difference between ASPIM and GM isbigger especially for high 120588 values Moreover under thehighest network load ASPIM achieves MOS values aboveQoE satisfaction threshold (MOS = 4) Furthermore averageMOS values forMR and PF dramatically decrease comparingwith the previous settings

Concerning class behaviour we show per-class resultsin Figures 10(b) and 10(c) As can be concluded fromthe presented results QoE-aware policies guarantee fairnessamong classes and ASPIM is the best choice also inside classAside from that for the cost dependant cAS120583 discipline thedeterioration of the class with no priority is significant

6 Conclusions

This paper goes into detail about obtaining a schedulingalgorithmaimed atmaximizing usersrsquo perception of quality inchannels with time-varying capacity Furthermore this workconsiders traffic flowswith nonexponential size distributionscontrary to previous approaches that assume memorylessdistributions

As first contribution we provide a MDP-based modelfor the scheduling problem presented This model combinesQoE-awareness size-awareness and channel-awarenessNev-ertheless since this model can not be solved either analyt-ically or computationally from the proposed MDP modelwe derive a simple tractable and implementable well-performing scheduling index rule by applying amethodologybased on Whittle approach

The proposed scheduling solution still combines QoE-size- and channel-awareness Our scheduling proposal givespriority to users in their best channel condition that are

Abstract and Applied Analysis 13

out of the service unavoidable range As verified fromsimulation results the proposed scheduling strategy showssuitable subjective quality performance in a wide range ofscenarios including the case of a simplified 4G network withheterogeneous users

Therefore the results of this work present a relevantmathematical basis for developing scheduling algorithmsaimed at maximizing QoE in current and future networkswith time-varying channel capacity This way the proposedchannel-aware ASPIM discipline will be useful for networkoperators in order to guarantee to their customers servicesatisfaction in time-varying wireless networks Apart fromthat the approximations and simplifications used in Whittlemethod are applicable to any area when the gradient ofsystem components is considerably small among consecutivedecision slots

As future research we will extend our work to amultiuserapproach considering the simultaneous transmission ofmul-tiple flows per TTI

Appendix

Proof of Proposition 4

LemmaA1 Suppose that states (119886 119889+1119898 le 119899)cup(119886+119903119899 119889+

1119898 le 119899) are passive and that states (119886 119889 + 1119898 gt 119899) cup (119886 +

119903119899 119889 + 1119898 gt 119899) are active and

Vlowast(119886119889119899)

=120573120583(119886119899)

( MOS (119889 + 1) minus 119877119886119889)

1 minus 120573120583(119886119899)

119882119886119889

(A1)

Along with this if we assume that 119877119886119889

= sum119898119902119898119877(119886119889119898)

asymp

sum119898119902119898119877(119886+119903119897 119889+1119898)

and 119882119886119889

= sum119898119902119898119882(119886119889119898)

asymp

sum119898119902119898119882(119886+119903119897 119889+1119898)

for any 119897 = 1 2 119873 value for theundiscounted case

Vlowast(119886119889119899)

= lim120573rarr1

120573120583(119886119899)

sdot MOS (119889 + 1) (1 minus 120573)1 minus 120573 + 120573sum

119898gt119899119902119898(120583(119886119898)

minus 120583(119886119899)

) (A2)

Proof Referring to work elements using (12)

119882(119886119889119898le119899)

= 120573sum

119898

1199021015840

119898119882(119886119889+1119898

1015840)= 120573119882

119886119889

119882(119886119889119898gt119899)

= 1 + 120573 (1 minus 120583(119886119898)

)sum

119898

1199021015840

119898119882(119886+119903119898 119889+1119898

1015840)

= 1 + 120573 (1 minus 120583(119886119898)

)119882119886119889

(A3)

And by (A3) expressions

119882119886119889= (1 minus sum

119898gt119899

119902119898)120573119882

119886119889+ sum

119898gt119899

119902119898(1 + 120573 (1 minus 120583

(119886119898))119882119886119889)

(A4)

This way isolating119882119886119889

119882119886119889=

sum119898gt119899

119902119898

1 minus 120573 + 120573sum119898gt119899

119902119898120583(119886119898)

(A5)

Analogously for reward elements using (11)

119877(119886119889119898le119899)

= 120573sum

119898

1199021015840

119898119877(119886119889+1119898

1015840)= 120573119877119886119889

119877(119886119889119898gt119899)

= 120573[(1 minus 120583(119886119898)

)sum

119898

1199021015840

119898119877(119886+119903119898 119898

1015840)+ 120583(119886119898)

sdot MOS (119889 + 1)]

= 120573 [(1 minus 120583(119886119898)

) 119877119886119889+ 120583(119886119898)

sdot MOS (119889 + 1)] (A6)

And thus from (A6)

119877119886119889= (1 minus sum

119898gt119899

119902119898)120573119877119886119889

+ sum

119898gt119899

119902119898(120573 [(1 minus 120583

(119886119898)) 119877119886119889+ 120583(119886119898)

sdot MOS (119889 + 1)])

(A7)

Isolating 119877119886119889

119877119886119889=120573sum119898gt119899

119902119898120583(119886119898)

MOS (119889 + 1)1 minus 120573 + 120573sum

119898gt119899119902119898120583(119886119898)

(A8)

Substituting (A5) and (A8) in (A1) and by simplifyingwe obtain

Vlowast(119886119889119899)

= (120573120583(119886119899)

(MOS (119889 + 1)

minus120573sum119898gt119899

119902119898120583(119886119898)

MOS (119889 + 1)1 minus 120573 + 120573sum

119898gt119899119902119898120583(119886119898)

))

times (1 minus 120573120583(119886119899)

sum119898gt119899

119902119898

1 minus 120573 + 120573sum119898gt119899

119902119898120583(119886119898)

)

minus1

= (120573120583(119886119899)

(MOS (119889 + 1) (1 minus 120573 + 120573sum119898gt119899

119902119898120583(119886119898)

)

minus120573sum

119898gt119899

119902119898120583(119886119898)

MOS (119889 + 1)))

times (1 minus 120573 + 120573sum

119898gt119899

119902119898120583(119886119898)

minus 120573120583(119886119899)

sum

119898gt119899

119902119898)

minus1

=120573120583(119886119899)

sdot MOS (119889 + 1) (1 minus 120573)1 minus 120573 + 120573sum

119898gt119899119902119898(120583(119886119898)

minus 120583(119886119899)

)

(A9)

And therefore for the undiscounted case the 120573 rarr 1

limit for expression (A9) is (A2)

Conflict of Interests

The authors declare that there is no conflict of interestsregarding the publication of this paper

14 Abstract and Applied Analysis

Acknowledgment

The research leading to these results has received fundingfrom the European Union Seventh Framework Programme(FP72007ndash2013) under Grant agreement 284863 (FP7 SECGERYON)

References

[1] ITU-T P10G100 Amendment 1 (0107) New Appendix I-definition of Quality of Experience (QoE) 2006

[2] S Aalto and P Lassila ldquoFlow-level stability and performance ofchannel-aware priority-based schedulersrdquo in Proceeedings of the6th Euro NF Conference on Next Generation Internet (NGI rsquo10)IEEE June 2010

[3] L KleinrockQueueing Systems vol 2 ofComputer ApplicationsJohn Wiley amp Sons 1976

[4] J Gittins ldquoBandit processes and dynamic allocation indicesrdquoJournal of the Royal Statistical Society B Methodological vol 41no 2 pp 148ndash177 1979

[5] K Avrachenkovt U Ayesta P Brown and E Nyberg ldquoDif-ferentiation between short and long TCP ows predictabilityof the response timerdquo in Proceedings of the 23rd Annual JointConference of the IEEE Computer and Communications Societies(INFOCOM rsquo04) vol 2 p 762 IEEE 2004

[6] U Ayesta M Erausquin and P Jacko ldquoA modeling frameworkfor optimizing the flow-level scheduling with time-varyingchannelsrdquo Performance Evaluation vol 67 no 11 pp 1014ndash10292010

[7] S Aalto U Ayesta and R Righter ldquoProperties of the Gittinsindex with application to optimal schedulingrdquo Probability in theEngineering and Informational Sciences vol 25 no 3 pp 269ndash288 2011

[8] M Puterman ldquoMarkov decision processesrdquo in Handbooks inOperations Research and Management Science vol 2 pp 331ndash434 1990

[9] C H Papadimitriou and J N Tsitsiklis ldquoThe complexityof Markov decision processesrdquo Mathematics of OperationsResearch vol 12 no 3 pp 441ndash450 1987

[10] J Nino-Mora ldquoDynamic priority allocation via restless banditmarginal productivity indicesrdquo TOP vol 15 no 2 pp 161ndash1982007

[11] P Whittle ldquoRestless bandits activity allocation in a changingworldrdquo Journal of Applied Probability vol 25 pp 287ndash298 1988

[12] U Ayesta M Erausquin M Jonckheere and I M VerloopldquoScheduling in a random environment stability and asymptoticoptimalityrdquo IEEEACM Transactions on Networking vol 21 no1 pp 258ndash271 2013

[13] S Khan S Thakolsri E Steinbach and W Kellerer ldquoQoE-based cross-layer optimization for wireless multiuser systemsrdquoin Proceedings of the 18th ITC Specialist Seminar on Quality ofExperience 2008

[14] S Thakolsri S Khan E Steinbach and W Kellerer ldquoQoE-driven cross-layer optimization for high speed downlink packetaccessrdquo Journal of Communications vol 4 no 9 pp 669ndash6802009

[15] P Ameigeiras J J Ramos-Munoz J Navarro-Ortiz PMogensen and J M Lopez-Soler ldquoQoE oriented cross-layer design of a resource allocation algorithm in beyond 3GsystemsrdquoComputer Communications vol 33 no 5 pp 571ndash5822010

[16] I Taboada F Liberal J O Fajardo and U Ayesta ldquoQoE-aware optimization of multimedia flow schedulingrdquo ComputerCommunications vol 36 no 15-16 pp 1629ndash1638 2013

[17] U Ayesta M Erausquin and P Jacko ldquoResource-sharing in asingle server with time-varying capacityrdquo in Proceeding of the49th Annual Allerton Conference on Communication Controland Computing (Allerton rsquo11) pp 377ndash383 Monticello Ill USASeptember 2011

[18] K Thompson G J Miller and R Wilder ldquoWide-area internettraffic patterns and characteristicsrdquo IEEE Network vol 11 no 6pp 10ndash23 1997

[19] M Fiedler T Hossfeld and P Tran-Gia ldquoA generic quantitativerelationship between quality of experience and quality ofservicerdquo IEEE Network vol 24 no 2 pp 36ndash41 2010

[20] A Richards M Antoniades V Witana and G Rogers ldquoMap-ping user level QoS from a single parameterrdquo in Proceedingsof the International Conference on Multimedia Networks andServices Citeseer 1998

[21] E Ibarrola F Liberal I Taboada and R Ortega ldquoWebQoE evaluation in multi-agent networks validation of ITU-TG1030rdquo in Proceedings of the 5th International Conference onAutonomic and Autonomous Systems (ICAS rsquo09) pp 289ndash294Valencia Spain April 2009

[22] S Aalto and U Ayesta ldquoOptimal scheduling of jobs with aDHR tail in the MG1 queuerdquo in Proceedings of the 3rd Inter-national Conference on Performance Evaluation Methodologiesand Tools (ValueTools rsquo08) article 50 Institute for ComputerSciences Social-Informatics and Telecommunications Engi-neering Athens Greece 2008

[23] V Visweswaran ldquoDecomposition techniques for MILPlagrangian relaxationrdquo in Encyclopedia of Optimization pp632ndash638 Springer New York NY USA 2009

[24] I Taboada J O Fajardo and F Liberal ldquoPerformance analysisof scheduling algorithms for web QoE optimization in wirelessnetworksrdquo Network Protocols and Algorithms vol 4 pp 27ndash432012

[25] J A van Mieghem ldquoDynamic scheduling with convex delaycosts the generalized 119888120583 rulerdquoTheAnnals of Applied Probabilityvol 5 no 3 pp 809ndash833 1995

[26] S Sesia I Toufik and M Baker LTE-The UMTS Long TermEvolution FromTheory to Practice Wiley 2011

[27] P Reichl B Tuffin and R Schatz ldquoLogarithmic laws inservice quality perception where microeconomics meets psy-chophysics and quality of experiencerdquo Telecommunication Sys-tems vol 52 no 2 pp 587ndash600 2013

[28] J C Ikuno M Wrulich and M Rupp ldquoSystem level simulationof LTE networksrdquo in Proceedings of the IEEE 71st VehicularTechnology Conference Taipei Taiwan May 2010

Submit your manuscripts athttpwwwhindawicom

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical Problems in Engineering

Hindawi Publishing Corporationhttpwwwhindawicom

Differential EquationsInternational Journal of

Volume 2014

Applied MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical PhysicsAdvances in

Complex AnalysisJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

OptimizationJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Operations ResearchAdvances in

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Function Spaces

Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of Mathematics and Mathematical Sciences

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Algebra

Discrete Dynamics in Nature and Society

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Decision SciencesAdvances in

Discrete MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Stochastic AnalysisInternational Journal of

Abstract and Applied Analysis 11

03 04 05 06 07 08 09

05

06

07

08

09

ASPIMMRPF

cAS120583GM

120588

E[M

OS]

2E

[MO

S]1

Figure 7 E[MOS] for abrupt MOS scenario

which theMOS function in theQoE degradation range showsa great slope Thus in this setting the error of the MOSapproximation (15) is high since the gradient of MOS amongconsecutive delay levels becomes significant In Figure 7we compare the results of this second scenario and thebasic scenario From the illustrated graph we could say thatour QoE-aware proposal adapts to any MOS function Thisis whereas for the rest of disciplines (excluding GM) thenegative effects due to the abrupt degradation of the QoEutility function are appreciable ASPIMmanages to avoid thisadapting to any MOS function

53 Scenario 3 Higher Error Case due to Rate During thesimplification process carried out for obtaining the ASPIMindex rule in Section 41 the 120583 approximation used (see (14))introduces a rate dependant error Note that increasing therate in the good channel state makes bigger the error of the 120583approximation used Therefore we will analyze the obtainedperformance for our new QoE-aware proposal under thishigher error due to rate In the previous scenarios we haveassumed that the transmission rate in the good channelcondition is twice the rate in the bad channel state In thissetting we consider that the transmission rate in the goodchannel is four times the transmission rate in the bad channelThis way according to the QoE results presented in Figure 8in spite of a higher error caused by rate ASPIM is superior tothe rest of policies

54 Scenario 4 High Error Case due to Size Besides it isknown that for a distribution with a decreasing hazard ratesuch as Pareto that the gradient of completion probabilityincreases as long as the mean size decreases which makesthe error of the 120583 approximation (14) for the ASPIM ruleachievement bigger In this way in this scenario we verifythat ASPIM gives satisfactory behavior under a more notableerror due to size where we consider a mean size ten times

03 04 05 06 07 08 09

3

35

4

45

5

E[M

OS]

ASPIMMRPF

cAS120583GM

120588

Figure 8 E[MOS] under higher error due to rate

03 04 05 06 07 08 093

35

4

45

5E

[MO

S]

ASPIMMRPF

cAS120583GM

120588

Figure 9 E[MOS] under high error due to size

lower We conclude from Figure 9 that even under a higherror caused by size that ASPIM shows good performancebeing the best alternative

55 Scenario 5 A Real Network Case This last scenarioreflects a real 4G wireless network context but with thesimplification that a single user transmits in each TTI To thatend we use CQI traces obtained from a system-level radioaccess simulator [28] considering users moving at 5 kmhin the cell For this setting with dynamic users we definechannel propagation by the Extended Pedestrian Amodel Asin current 4G networks we consider 16 channel conditionsThe state probabilities of these channel states are collectedin Table 3 which are decreasing with the improvement ofchannel quality In this scenario we consider two classes ofusers which differ in size QoE and cost The mean size

12 Abstract and Applied Analysis

03 04 05 06 07 08 09

1

2

3

4

5

E[M

OS]

120588

(a)

03 04 05 06 07 08 091

2

3

4

5

E[M

OS]

ASPIMMRPF

cAS120583GM

120588

(b)

03 04 05 06 07 08 091

2

3

4

5

E[M

OS]

ASPIMMRPF

cAS120583GM

120588

(c)

Figure 10 E[MOS] for 4G scenario aggregate (a) class 1 (b) class 2 (c)

in class 2 is ten times bigger than in class 1 and we givepriority to the small-sized class by having bigger cost andQoErequirement

Under this mixture of classes and realistic channel mod-els in Figure 10(a) we observe that scheduling strategiesshow the same tendency as in the previous scenarios It isremarkable that the difference between ASPIM and GM isbigger especially for high 120588 values Moreover under thehighest network load ASPIM achieves MOS values aboveQoE satisfaction threshold (MOS = 4) Furthermore averageMOS values forMR and PF dramatically decrease comparingwith the previous settings

Concerning class behaviour we show per-class resultsin Figures 10(b) and 10(c) As can be concluded fromthe presented results QoE-aware policies guarantee fairnessamong classes and ASPIM is the best choice also inside classAside from that for the cost dependant cAS120583 discipline thedeterioration of the class with no priority is significant

6 Conclusions

This paper goes into detail about obtaining a schedulingalgorithmaimed atmaximizing usersrsquo perception of quality inchannels with time-varying capacity Furthermore this workconsiders traffic flowswith nonexponential size distributionscontrary to previous approaches that assume memorylessdistributions

As first contribution we provide a MDP-based modelfor the scheduling problem presented This model combinesQoE-awareness size-awareness and channel-awarenessNev-ertheless since this model can not be solved either analyt-ically or computationally from the proposed MDP modelwe derive a simple tractable and implementable well-performing scheduling index rule by applying amethodologybased on Whittle approach

The proposed scheduling solution still combines QoE-size- and channel-awareness Our scheduling proposal givespriority to users in their best channel condition that are

Abstract and Applied Analysis 13

out of the service unavoidable range As verified fromsimulation results the proposed scheduling strategy showssuitable subjective quality performance in a wide range ofscenarios including the case of a simplified 4G network withheterogeneous users

Therefore the results of this work present a relevantmathematical basis for developing scheduling algorithmsaimed at maximizing QoE in current and future networkswith time-varying channel capacity This way the proposedchannel-aware ASPIM discipline will be useful for networkoperators in order to guarantee to their customers servicesatisfaction in time-varying wireless networks Apart fromthat the approximations and simplifications used in Whittlemethod are applicable to any area when the gradient ofsystem components is considerably small among consecutivedecision slots

As future research we will extend our work to amultiuserapproach considering the simultaneous transmission ofmul-tiple flows per TTI

Appendix

Proof of Proposition 4

LemmaA1 Suppose that states (119886 119889+1119898 le 119899)cup(119886+119903119899 119889+

1119898 le 119899) are passive and that states (119886 119889 + 1119898 gt 119899) cup (119886 +

119903119899 119889 + 1119898 gt 119899) are active and

Vlowast(119886119889119899)

=120573120583(119886119899)

( MOS (119889 + 1) minus 119877119886119889)

1 minus 120573120583(119886119899)

119882119886119889

(A1)

Along with this if we assume that 119877119886119889

= sum119898119902119898119877(119886119889119898)

asymp

sum119898119902119898119877(119886+119903119897 119889+1119898)

and 119882119886119889

= sum119898119902119898119882(119886119889119898)

asymp

sum119898119902119898119882(119886+119903119897 119889+1119898)

for any 119897 = 1 2 119873 value for theundiscounted case

Vlowast(119886119889119899)

= lim120573rarr1

120573120583(119886119899)

sdot MOS (119889 + 1) (1 minus 120573)1 minus 120573 + 120573sum

119898gt119899119902119898(120583(119886119898)

minus 120583(119886119899)

) (A2)

Proof Referring to work elements using (12)

119882(119886119889119898le119899)

= 120573sum

119898

1199021015840

119898119882(119886119889+1119898

1015840)= 120573119882

119886119889

119882(119886119889119898gt119899)

= 1 + 120573 (1 minus 120583(119886119898)

)sum

119898

1199021015840

119898119882(119886+119903119898 119889+1119898

1015840)

= 1 + 120573 (1 minus 120583(119886119898)

)119882119886119889

(A3)

And by (A3) expressions

119882119886119889= (1 minus sum

119898gt119899

119902119898)120573119882

119886119889+ sum

119898gt119899

119902119898(1 + 120573 (1 minus 120583

(119886119898))119882119886119889)

(A4)

This way isolating119882119886119889

119882119886119889=

sum119898gt119899

119902119898

1 minus 120573 + 120573sum119898gt119899

119902119898120583(119886119898)

(A5)

Analogously for reward elements using (11)

119877(119886119889119898le119899)

= 120573sum

119898

1199021015840

119898119877(119886119889+1119898

1015840)= 120573119877119886119889

119877(119886119889119898gt119899)

= 120573[(1 minus 120583(119886119898)

)sum

119898

1199021015840

119898119877(119886+119903119898 119898

1015840)+ 120583(119886119898)

sdot MOS (119889 + 1)]

= 120573 [(1 minus 120583(119886119898)

) 119877119886119889+ 120583(119886119898)

sdot MOS (119889 + 1)] (A6)

And thus from (A6)

119877119886119889= (1 minus sum

119898gt119899

119902119898)120573119877119886119889

+ sum

119898gt119899

119902119898(120573 [(1 minus 120583

(119886119898)) 119877119886119889+ 120583(119886119898)

sdot MOS (119889 + 1)])

(A7)

Isolating 119877119886119889

119877119886119889=120573sum119898gt119899

119902119898120583(119886119898)

MOS (119889 + 1)1 minus 120573 + 120573sum

119898gt119899119902119898120583(119886119898)

(A8)

Substituting (A5) and (A8) in (A1) and by simplifyingwe obtain

Vlowast(119886119889119899)

= (120573120583(119886119899)

(MOS (119889 + 1)

minus120573sum119898gt119899

119902119898120583(119886119898)

MOS (119889 + 1)1 minus 120573 + 120573sum

119898gt119899119902119898120583(119886119898)

))

times (1 minus 120573120583(119886119899)

sum119898gt119899

119902119898

1 minus 120573 + 120573sum119898gt119899

119902119898120583(119886119898)

)

minus1

= (120573120583(119886119899)

(MOS (119889 + 1) (1 minus 120573 + 120573sum119898gt119899

119902119898120583(119886119898)

)

minus120573sum

119898gt119899

119902119898120583(119886119898)

MOS (119889 + 1)))

times (1 minus 120573 + 120573sum

119898gt119899

119902119898120583(119886119898)

minus 120573120583(119886119899)

sum

119898gt119899

119902119898)

minus1

=120573120583(119886119899)

sdot MOS (119889 + 1) (1 minus 120573)1 minus 120573 + 120573sum

119898gt119899119902119898(120583(119886119898)

minus 120583(119886119899)

)

(A9)

And therefore for the undiscounted case the 120573 rarr 1

limit for expression (A9) is (A2)

Conflict of Interests

The authors declare that there is no conflict of interestsregarding the publication of this paper

14 Abstract and Applied Analysis

Acknowledgment

The research leading to these results has received fundingfrom the European Union Seventh Framework Programme(FP72007ndash2013) under Grant agreement 284863 (FP7 SECGERYON)

References

[1] ITU-T P10G100 Amendment 1 (0107) New Appendix I-definition of Quality of Experience (QoE) 2006

[2] S Aalto and P Lassila ldquoFlow-level stability and performance ofchannel-aware priority-based schedulersrdquo in Proceeedings of the6th Euro NF Conference on Next Generation Internet (NGI rsquo10)IEEE June 2010

[3] L KleinrockQueueing Systems vol 2 ofComputer ApplicationsJohn Wiley amp Sons 1976

[4] J Gittins ldquoBandit processes and dynamic allocation indicesrdquoJournal of the Royal Statistical Society B Methodological vol 41no 2 pp 148ndash177 1979

[5] K Avrachenkovt U Ayesta P Brown and E Nyberg ldquoDif-ferentiation between short and long TCP ows predictabilityof the response timerdquo in Proceedings of the 23rd Annual JointConference of the IEEE Computer and Communications Societies(INFOCOM rsquo04) vol 2 p 762 IEEE 2004

[6] U Ayesta M Erausquin and P Jacko ldquoA modeling frameworkfor optimizing the flow-level scheduling with time-varyingchannelsrdquo Performance Evaluation vol 67 no 11 pp 1014ndash10292010

[7] S Aalto U Ayesta and R Righter ldquoProperties of the Gittinsindex with application to optimal schedulingrdquo Probability in theEngineering and Informational Sciences vol 25 no 3 pp 269ndash288 2011

[8] M Puterman ldquoMarkov decision processesrdquo in Handbooks inOperations Research and Management Science vol 2 pp 331ndash434 1990

[9] C H Papadimitriou and J N Tsitsiklis ldquoThe complexityof Markov decision processesrdquo Mathematics of OperationsResearch vol 12 no 3 pp 441ndash450 1987

[10] J Nino-Mora ldquoDynamic priority allocation via restless banditmarginal productivity indicesrdquo TOP vol 15 no 2 pp 161ndash1982007

[11] P Whittle ldquoRestless bandits activity allocation in a changingworldrdquo Journal of Applied Probability vol 25 pp 287ndash298 1988

[12] U Ayesta M Erausquin M Jonckheere and I M VerloopldquoScheduling in a random environment stability and asymptoticoptimalityrdquo IEEEACM Transactions on Networking vol 21 no1 pp 258ndash271 2013

[13] S Khan S Thakolsri E Steinbach and W Kellerer ldquoQoE-based cross-layer optimization for wireless multiuser systemsrdquoin Proceedings of the 18th ITC Specialist Seminar on Quality ofExperience 2008

[14] S Thakolsri S Khan E Steinbach and W Kellerer ldquoQoE-driven cross-layer optimization for high speed downlink packetaccessrdquo Journal of Communications vol 4 no 9 pp 669ndash6802009

[15] P Ameigeiras J J Ramos-Munoz J Navarro-Ortiz PMogensen and J M Lopez-Soler ldquoQoE oriented cross-layer design of a resource allocation algorithm in beyond 3GsystemsrdquoComputer Communications vol 33 no 5 pp 571ndash5822010

[16] I Taboada F Liberal J O Fajardo and U Ayesta ldquoQoE-aware optimization of multimedia flow schedulingrdquo ComputerCommunications vol 36 no 15-16 pp 1629ndash1638 2013

[17] U Ayesta M Erausquin and P Jacko ldquoResource-sharing in asingle server with time-varying capacityrdquo in Proceeding of the49th Annual Allerton Conference on Communication Controland Computing (Allerton rsquo11) pp 377ndash383 Monticello Ill USASeptember 2011

[18] K Thompson G J Miller and R Wilder ldquoWide-area internettraffic patterns and characteristicsrdquo IEEE Network vol 11 no 6pp 10ndash23 1997

[19] M Fiedler T Hossfeld and P Tran-Gia ldquoA generic quantitativerelationship between quality of experience and quality ofservicerdquo IEEE Network vol 24 no 2 pp 36ndash41 2010

[20] A Richards M Antoniades V Witana and G Rogers ldquoMap-ping user level QoS from a single parameterrdquo in Proceedingsof the International Conference on Multimedia Networks andServices Citeseer 1998

[21] E Ibarrola F Liberal I Taboada and R Ortega ldquoWebQoE evaluation in multi-agent networks validation of ITU-TG1030rdquo in Proceedings of the 5th International Conference onAutonomic and Autonomous Systems (ICAS rsquo09) pp 289ndash294Valencia Spain April 2009

[22] S Aalto and U Ayesta ldquoOptimal scheduling of jobs with aDHR tail in the MG1 queuerdquo in Proceedings of the 3rd Inter-national Conference on Performance Evaluation Methodologiesand Tools (ValueTools rsquo08) article 50 Institute for ComputerSciences Social-Informatics and Telecommunications Engi-neering Athens Greece 2008

[23] V Visweswaran ldquoDecomposition techniques for MILPlagrangian relaxationrdquo in Encyclopedia of Optimization pp632ndash638 Springer New York NY USA 2009

[24] I Taboada J O Fajardo and F Liberal ldquoPerformance analysisof scheduling algorithms for web QoE optimization in wirelessnetworksrdquo Network Protocols and Algorithms vol 4 pp 27ndash432012

[25] J A van Mieghem ldquoDynamic scheduling with convex delaycosts the generalized 119888120583 rulerdquoTheAnnals of Applied Probabilityvol 5 no 3 pp 809ndash833 1995

[26] S Sesia I Toufik and M Baker LTE-The UMTS Long TermEvolution FromTheory to Practice Wiley 2011

[27] P Reichl B Tuffin and R Schatz ldquoLogarithmic laws inservice quality perception where microeconomics meets psy-chophysics and quality of experiencerdquo Telecommunication Sys-tems vol 52 no 2 pp 587ndash600 2013

[28] J C Ikuno M Wrulich and M Rupp ldquoSystem level simulationof LTE networksrdquo in Proceedings of the IEEE 71st VehicularTechnology Conference Taipei Taiwan May 2010

Submit your manuscripts athttpwwwhindawicom

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical Problems in Engineering

Hindawi Publishing Corporationhttpwwwhindawicom

Differential EquationsInternational Journal of

Volume 2014

Applied MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical PhysicsAdvances in

Complex AnalysisJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

OptimizationJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Operations ResearchAdvances in

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Function Spaces

Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of Mathematics and Mathematical Sciences

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Algebra

Discrete Dynamics in Nature and Society

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Decision SciencesAdvances in

Discrete MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Stochastic AnalysisInternational Journal of

12 Abstract and Applied Analysis

03 04 05 06 07 08 09

1

2

3

4

5

E[M

OS]

120588

(a)

03 04 05 06 07 08 091

2

3

4

5

E[M

OS]

ASPIMMRPF

cAS120583GM

120588

(b)

03 04 05 06 07 08 091

2

3

4

5

E[M

OS]

ASPIMMRPF

cAS120583GM

120588

(c)

Figure 10 E[MOS] for 4G scenario aggregate (a) class 1 (b) class 2 (c)

in class 2 is ten times bigger than in class 1 and we givepriority to the small-sized class by having bigger cost andQoErequirement

Under this mixture of classes and realistic channel mod-els in Figure 10(a) we observe that scheduling strategiesshow the same tendency as in the previous scenarios It isremarkable that the difference between ASPIM and GM isbigger especially for high 120588 values Moreover under thehighest network load ASPIM achieves MOS values aboveQoE satisfaction threshold (MOS = 4) Furthermore averageMOS values forMR and PF dramatically decrease comparingwith the previous settings

Concerning class behaviour we show per-class resultsin Figures 10(b) and 10(c) As can be concluded fromthe presented results QoE-aware policies guarantee fairnessamong classes and ASPIM is the best choice also inside classAside from that for the cost dependant cAS120583 discipline thedeterioration of the class with no priority is significant

6 Conclusions

This paper goes into detail about obtaining a schedulingalgorithmaimed atmaximizing usersrsquo perception of quality inchannels with time-varying capacity Furthermore this workconsiders traffic flowswith nonexponential size distributionscontrary to previous approaches that assume memorylessdistributions

As first contribution we provide a MDP-based modelfor the scheduling problem presented This model combinesQoE-awareness size-awareness and channel-awarenessNev-ertheless since this model can not be solved either analyt-ically or computationally from the proposed MDP modelwe derive a simple tractable and implementable well-performing scheduling index rule by applying amethodologybased on Whittle approach

The proposed scheduling solution still combines QoE-size- and channel-awareness Our scheduling proposal givespriority to users in their best channel condition that are

Abstract and Applied Analysis 13

out of the service unavoidable range As verified fromsimulation results the proposed scheduling strategy showssuitable subjective quality performance in a wide range ofscenarios including the case of a simplified 4G network withheterogeneous users

Therefore the results of this work present a relevantmathematical basis for developing scheduling algorithmsaimed at maximizing QoE in current and future networkswith time-varying channel capacity This way the proposedchannel-aware ASPIM discipline will be useful for networkoperators in order to guarantee to their customers servicesatisfaction in time-varying wireless networks Apart fromthat the approximations and simplifications used in Whittlemethod are applicable to any area when the gradient ofsystem components is considerably small among consecutivedecision slots

As future research we will extend our work to amultiuserapproach considering the simultaneous transmission ofmul-tiple flows per TTI

Appendix

Proof of Proposition 4

LemmaA1 Suppose that states (119886 119889+1119898 le 119899)cup(119886+119903119899 119889+

1119898 le 119899) are passive and that states (119886 119889 + 1119898 gt 119899) cup (119886 +

119903119899 119889 + 1119898 gt 119899) are active and

Vlowast(119886119889119899)

=120573120583(119886119899)

( MOS (119889 + 1) minus 119877119886119889)

1 minus 120573120583(119886119899)

119882119886119889

(A1)

Along with this if we assume that 119877119886119889

= sum119898119902119898119877(119886119889119898)

asymp

sum119898119902119898119877(119886+119903119897 119889+1119898)

and 119882119886119889

= sum119898119902119898119882(119886119889119898)

asymp

sum119898119902119898119882(119886+119903119897 119889+1119898)

for any 119897 = 1 2 119873 value for theundiscounted case

Vlowast(119886119889119899)

= lim120573rarr1

120573120583(119886119899)

sdot MOS (119889 + 1) (1 minus 120573)1 minus 120573 + 120573sum

119898gt119899119902119898(120583(119886119898)

minus 120583(119886119899)

) (A2)

Proof Referring to work elements using (12)

119882(119886119889119898le119899)

= 120573sum

119898

1199021015840

119898119882(119886119889+1119898

1015840)= 120573119882

119886119889

119882(119886119889119898gt119899)

= 1 + 120573 (1 minus 120583(119886119898)

)sum

119898

1199021015840

119898119882(119886+119903119898 119889+1119898

1015840)

= 1 + 120573 (1 minus 120583(119886119898)

)119882119886119889

(A3)

And by (A3) expressions

119882119886119889= (1 minus sum

119898gt119899

119902119898)120573119882

119886119889+ sum

119898gt119899

119902119898(1 + 120573 (1 minus 120583

(119886119898))119882119886119889)

(A4)

This way isolating119882119886119889

119882119886119889=

sum119898gt119899

119902119898

1 minus 120573 + 120573sum119898gt119899

119902119898120583(119886119898)

(A5)

Analogously for reward elements using (11)

119877(119886119889119898le119899)

= 120573sum

119898

1199021015840

119898119877(119886119889+1119898

1015840)= 120573119877119886119889

119877(119886119889119898gt119899)

= 120573[(1 minus 120583(119886119898)

)sum

119898

1199021015840

119898119877(119886+119903119898 119898

1015840)+ 120583(119886119898)

sdot MOS (119889 + 1)]

= 120573 [(1 minus 120583(119886119898)

) 119877119886119889+ 120583(119886119898)

sdot MOS (119889 + 1)] (A6)

And thus from (A6)

119877119886119889= (1 minus sum

119898gt119899

119902119898)120573119877119886119889

+ sum

119898gt119899

119902119898(120573 [(1 minus 120583

(119886119898)) 119877119886119889+ 120583(119886119898)

sdot MOS (119889 + 1)])

(A7)

Isolating 119877119886119889

119877119886119889=120573sum119898gt119899

119902119898120583(119886119898)

MOS (119889 + 1)1 minus 120573 + 120573sum

119898gt119899119902119898120583(119886119898)

(A8)

Substituting (A5) and (A8) in (A1) and by simplifyingwe obtain

Vlowast(119886119889119899)

= (120573120583(119886119899)

(MOS (119889 + 1)

minus120573sum119898gt119899

119902119898120583(119886119898)

MOS (119889 + 1)1 minus 120573 + 120573sum

119898gt119899119902119898120583(119886119898)

))

times (1 minus 120573120583(119886119899)

sum119898gt119899

119902119898

1 minus 120573 + 120573sum119898gt119899

119902119898120583(119886119898)

)

minus1

= (120573120583(119886119899)

(MOS (119889 + 1) (1 minus 120573 + 120573sum119898gt119899

119902119898120583(119886119898)

)

minus120573sum

119898gt119899

119902119898120583(119886119898)

MOS (119889 + 1)))

times (1 minus 120573 + 120573sum

119898gt119899

119902119898120583(119886119898)

minus 120573120583(119886119899)

sum

119898gt119899

119902119898)

minus1

=120573120583(119886119899)

sdot MOS (119889 + 1) (1 minus 120573)1 minus 120573 + 120573sum

119898gt119899119902119898(120583(119886119898)

minus 120583(119886119899)

)

(A9)

And therefore for the undiscounted case the 120573 rarr 1

limit for expression (A9) is (A2)

Conflict of Interests

The authors declare that there is no conflict of interestsregarding the publication of this paper

14 Abstract and Applied Analysis

Acknowledgment

The research leading to these results has received fundingfrom the European Union Seventh Framework Programme(FP72007ndash2013) under Grant agreement 284863 (FP7 SECGERYON)

References

[1] ITU-T P10G100 Amendment 1 (0107) New Appendix I-definition of Quality of Experience (QoE) 2006

[2] S Aalto and P Lassila ldquoFlow-level stability and performance ofchannel-aware priority-based schedulersrdquo in Proceeedings of the6th Euro NF Conference on Next Generation Internet (NGI rsquo10)IEEE June 2010

[3] L KleinrockQueueing Systems vol 2 ofComputer ApplicationsJohn Wiley amp Sons 1976

[4] J Gittins ldquoBandit processes and dynamic allocation indicesrdquoJournal of the Royal Statistical Society B Methodological vol 41no 2 pp 148ndash177 1979

[5] K Avrachenkovt U Ayesta P Brown and E Nyberg ldquoDif-ferentiation between short and long TCP ows predictabilityof the response timerdquo in Proceedings of the 23rd Annual JointConference of the IEEE Computer and Communications Societies(INFOCOM rsquo04) vol 2 p 762 IEEE 2004

[6] U Ayesta M Erausquin and P Jacko ldquoA modeling frameworkfor optimizing the flow-level scheduling with time-varyingchannelsrdquo Performance Evaluation vol 67 no 11 pp 1014ndash10292010

[7] S Aalto U Ayesta and R Righter ldquoProperties of the Gittinsindex with application to optimal schedulingrdquo Probability in theEngineering and Informational Sciences vol 25 no 3 pp 269ndash288 2011

[8] M Puterman ldquoMarkov decision processesrdquo in Handbooks inOperations Research and Management Science vol 2 pp 331ndash434 1990

[9] C H Papadimitriou and J N Tsitsiklis ldquoThe complexityof Markov decision processesrdquo Mathematics of OperationsResearch vol 12 no 3 pp 441ndash450 1987

[10] J Nino-Mora ldquoDynamic priority allocation via restless banditmarginal productivity indicesrdquo TOP vol 15 no 2 pp 161ndash1982007

[11] P Whittle ldquoRestless bandits activity allocation in a changingworldrdquo Journal of Applied Probability vol 25 pp 287ndash298 1988

[12] U Ayesta M Erausquin M Jonckheere and I M VerloopldquoScheduling in a random environment stability and asymptoticoptimalityrdquo IEEEACM Transactions on Networking vol 21 no1 pp 258ndash271 2013

[13] S Khan S Thakolsri E Steinbach and W Kellerer ldquoQoE-based cross-layer optimization for wireless multiuser systemsrdquoin Proceedings of the 18th ITC Specialist Seminar on Quality ofExperience 2008

[14] S Thakolsri S Khan E Steinbach and W Kellerer ldquoQoE-driven cross-layer optimization for high speed downlink packetaccessrdquo Journal of Communications vol 4 no 9 pp 669ndash6802009

[15] P Ameigeiras J J Ramos-Munoz J Navarro-Ortiz PMogensen and J M Lopez-Soler ldquoQoE oriented cross-layer design of a resource allocation algorithm in beyond 3GsystemsrdquoComputer Communications vol 33 no 5 pp 571ndash5822010

[16] I Taboada F Liberal J O Fajardo and U Ayesta ldquoQoE-aware optimization of multimedia flow schedulingrdquo ComputerCommunications vol 36 no 15-16 pp 1629ndash1638 2013

[17] U Ayesta M Erausquin and P Jacko ldquoResource-sharing in asingle server with time-varying capacityrdquo in Proceeding of the49th Annual Allerton Conference on Communication Controland Computing (Allerton rsquo11) pp 377ndash383 Monticello Ill USASeptember 2011

[18] K Thompson G J Miller and R Wilder ldquoWide-area internettraffic patterns and characteristicsrdquo IEEE Network vol 11 no 6pp 10ndash23 1997

[19] M Fiedler T Hossfeld and P Tran-Gia ldquoA generic quantitativerelationship between quality of experience and quality ofservicerdquo IEEE Network vol 24 no 2 pp 36ndash41 2010

[20] A Richards M Antoniades V Witana and G Rogers ldquoMap-ping user level QoS from a single parameterrdquo in Proceedingsof the International Conference on Multimedia Networks andServices Citeseer 1998

[21] E Ibarrola F Liberal I Taboada and R Ortega ldquoWebQoE evaluation in multi-agent networks validation of ITU-TG1030rdquo in Proceedings of the 5th International Conference onAutonomic and Autonomous Systems (ICAS rsquo09) pp 289ndash294Valencia Spain April 2009

[22] S Aalto and U Ayesta ldquoOptimal scheduling of jobs with aDHR tail in the MG1 queuerdquo in Proceedings of the 3rd Inter-national Conference on Performance Evaluation Methodologiesand Tools (ValueTools rsquo08) article 50 Institute for ComputerSciences Social-Informatics and Telecommunications Engi-neering Athens Greece 2008

[23] V Visweswaran ldquoDecomposition techniques for MILPlagrangian relaxationrdquo in Encyclopedia of Optimization pp632ndash638 Springer New York NY USA 2009

[24] I Taboada J O Fajardo and F Liberal ldquoPerformance analysisof scheduling algorithms for web QoE optimization in wirelessnetworksrdquo Network Protocols and Algorithms vol 4 pp 27ndash432012

[25] J A van Mieghem ldquoDynamic scheduling with convex delaycosts the generalized 119888120583 rulerdquoTheAnnals of Applied Probabilityvol 5 no 3 pp 809ndash833 1995

[26] S Sesia I Toufik and M Baker LTE-The UMTS Long TermEvolution FromTheory to Practice Wiley 2011

[27] P Reichl B Tuffin and R Schatz ldquoLogarithmic laws inservice quality perception where microeconomics meets psy-chophysics and quality of experiencerdquo Telecommunication Sys-tems vol 52 no 2 pp 587ndash600 2013

[28] J C Ikuno M Wrulich and M Rupp ldquoSystem level simulationof LTE networksrdquo in Proceedings of the IEEE 71st VehicularTechnology Conference Taipei Taiwan May 2010

Submit your manuscripts athttpwwwhindawicom

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical Problems in Engineering

Hindawi Publishing Corporationhttpwwwhindawicom

Differential EquationsInternational Journal of

Volume 2014

Applied MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical PhysicsAdvances in

Complex AnalysisJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

OptimizationJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Operations ResearchAdvances in

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Function Spaces

Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of Mathematics and Mathematical Sciences

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Algebra

Discrete Dynamics in Nature and Society

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Decision SciencesAdvances in

Discrete MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Stochastic AnalysisInternational Journal of

Abstract and Applied Analysis 13

out of the service unavoidable range As verified fromsimulation results the proposed scheduling strategy showssuitable subjective quality performance in a wide range ofscenarios including the case of a simplified 4G network withheterogeneous users

Therefore the results of this work present a relevantmathematical basis for developing scheduling algorithmsaimed at maximizing QoE in current and future networkswith time-varying channel capacity This way the proposedchannel-aware ASPIM discipline will be useful for networkoperators in order to guarantee to their customers servicesatisfaction in time-varying wireless networks Apart fromthat the approximations and simplifications used in Whittlemethod are applicable to any area when the gradient ofsystem components is considerably small among consecutivedecision slots

As future research we will extend our work to amultiuserapproach considering the simultaneous transmission ofmul-tiple flows per TTI

Appendix

Proof of Proposition 4

LemmaA1 Suppose that states (119886 119889+1119898 le 119899)cup(119886+119903119899 119889+

1119898 le 119899) are passive and that states (119886 119889 + 1119898 gt 119899) cup (119886 +

119903119899 119889 + 1119898 gt 119899) are active and

Vlowast(119886119889119899)

=120573120583(119886119899)

( MOS (119889 + 1) minus 119877119886119889)

1 minus 120573120583(119886119899)

119882119886119889

(A1)

Along with this if we assume that 119877119886119889

= sum119898119902119898119877(119886119889119898)

asymp

sum119898119902119898119877(119886+119903119897 119889+1119898)

and 119882119886119889

= sum119898119902119898119882(119886119889119898)

asymp

sum119898119902119898119882(119886+119903119897 119889+1119898)

for any 119897 = 1 2 119873 value for theundiscounted case

Vlowast(119886119889119899)

= lim120573rarr1

120573120583(119886119899)

sdot MOS (119889 + 1) (1 minus 120573)1 minus 120573 + 120573sum

119898gt119899119902119898(120583(119886119898)

minus 120583(119886119899)

) (A2)

Proof Referring to work elements using (12)

119882(119886119889119898le119899)

= 120573sum

119898

1199021015840

119898119882(119886119889+1119898

1015840)= 120573119882

119886119889

119882(119886119889119898gt119899)

= 1 + 120573 (1 minus 120583(119886119898)

)sum

119898

1199021015840

119898119882(119886+119903119898 119889+1119898

1015840)

= 1 + 120573 (1 minus 120583(119886119898)

)119882119886119889

(A3)

And by (A3) expressions

119882119886119889= (1 minus sum

119898gt119899

119902119898)120573119882

119886119889+ sum

119898gt119899

119902119898(1 + 120573 (1 minus 120583

(119886119898))119882119886119889)

(A4)

This way isolating119882119886119889

119882119886119889=

sum119898gt119899

119902119898

1 minus 120573 + 120573sum119898gt119899

119902119898120583(119886119898)

(A5)

Analogously for reward elements using (11)

119877(119886119889119898le119899)

= 120573sum

119898

1199021015840

119898119877(119886119889+1119898

1015840)= 120573119877119886119889

119877(119886119889119898gt119899)

= 120573[(1 minus 120583(119886119898)

)sum

119898

1199021015840

119898119877(119886+119903119898 119898

1015840)+ 120583(119886119898)

sdot MOS (119889 + 1)]

= 120573 [(1 minus 120583(119886119898)

) 119877119886119889+ 120583(119886119898)

sdot MOS (119889 + 1)] (A6)

And thus from (A6)

119877119886119889= (1 minus sum

119898gt119899

119902119898)120573119877119886119889

+ sum

119898gt119899

119902119898(120573 [(1 minus 120583

(119886119898)) 119877119886119889+ 120583(119886119898)

sdot MOS (119889 + 1)])

(A7)

Isolating 119877119886119889

119877119886119889=120573sum119898gt119899

119902119898120583(119886119898)

MOS (119889 + 1)1 minus 120573 + 120573sum

119898gt119899119902119898120583(119886119898)

(A8)

Substituting (A5) and (A8) in (A1) and by simplifyingwe obtain

Vlowast(119886119889119899)

= (120573120583(119886119899)

(MOS (119889 + 1)

minus120573sum119898gt119899

119902119898120583(119886119898)

MOS (119889 + 1)1 minus 120573 + 120573sum

119898gt119899119902119898120583(119886119898)

))

times (1 minus 120573120583(119886119899)

sum119898gt119899

119902119898

1 minus 120573 + 120573sum119898gt119899

119902119898120583(119886119898)

)

minus1

= (120573120583(119886119899)

(MOS (119889 + 1) (1 minus 120573 + 120573sum119898gt119899

119902119898120583(119886119898)

)

minus120573sum

119898gt119899

119902119898120583(119886119898)

MOS (119889 + 1)))

times (1 minus 120573 + 120573sum

119898gt119899

119902119898120583(119886119898)

minus 120573120583(119886119899)

sum

119898gt119899

119902119898)

minus1

=120573120583(119886119899)

sdot MOS (119889 + 1) (1 minus 120573)1 minus 120573 + 120573sum

119898gt119899119902119898(120583(119886119898)

minus 120583(119886119899)

)

(A9)

And therefore for the undiscounted case the 120573 rarr 1

limit for expression (A9) is (A2)

Conflict of Interests

The authors declare that there is no conflict of interestsregarding the publication of this paper

14 Abstract and Applied Analysis

Acknowledgment

The research leading to these results has received fundingfrom the European Union Seventh Framework Programme(FP72007ndash2013) under Grant agreement 284863 (FP7 SECGERYON)

References

[1] ITU-T P10G100 Amendment 1 (0107) New Appendix I-definition of Quality of Experience (QoE) 2006

[2] S Aalto and P Lassila ldquoFlow-level stability and performance ofchannel-aware priority-based schedulersrdquo in Proceeedings of the6th Euro NF Conference on Next Generation Internet (NGI rsquo10)IEEE June 2010

[3] L KleinrockQueueing Systems vol 2 ofComputer ApplicationsJohn Wiley amp Sons 1976

[4] J Gittins ldquoBandit processes and dynamic allocation indicesrdquoJournal of the Royal Statistical Society B Methodological vol 41no 2 pp 148ndash177 1979

[5] K Avrachenkovt U Ayesta P Brown and E Nyberg ldquoDif-ferentiation between short and long TCP ows predictabilityof the response timerdquo in Proceedings of the 23rd Annual JointConference of the IEEE Computer and Communications Societies(INFOCOM rsquo04) vol 2 p 762 IEEE 2004

[6] U Ayesta M Erausquin and P Jacko ldquoA modeling frameworkfor optimizing the flow-level scheduling with time-varyingchannelsrdquo Performance Evaluation vol 67 no 11 pp 1014ndash10292010

[7] S Aalto U Ayesta and R Righter ldquoProperties of the Gittinsindex with application to optimal schedulingrdquo Probability in theEngineering and Informational Sciences vol 25 no 3 pp 269ndash288 2011

[8] M Puterman ldquoMarkov decision processesrdquo in Handbooks inOperations Research and Management Science vol 2 pp 331ndash434 1990

[9] C H Papadimitriou and J N Tsitsiklis ldquoThe complexityof Markov decision processesrdquo Mathematics of OperationsResearch vol 12 no 3 pp 441ndash450 1987

[10] J Nino-Mora ldquoDynamic priority allocation via restless banditmarginal productivity indicesrdquo TOP vol 15 no 2 pp 161ndash1982007

[11] P Whittle ldquoRestless bandits activity allocation in a changingworldrdquo Journal of Applied Probability vol 25 pp 287ndash298 1988

[12] U Ayesta M Erausquin M Jonckheere and I M VerloopldquoScheduling in a random environment stability and asymptoticoptimalityrdquo IEEEACM Transactions on Networking vol 21 no1 pp 258ndash271 2013

[13] S Khan S Thakolsri E Steinbach and W Kellerer ldquoQoE-based cross-layer optimization for wireless multiuser systemsrdquoin Proceedings of the 18th ITC Specialist Seminar on Quality ofExperience 2008

[14] S Thakolsri S Khan E Steinbach and W Kellerer ldquoQoE-driven cross-layer optimization for high speed downlink packetaccessrdquo Journal of Communications vol 4 no 9 pp 669ndash6802009

[15] P Ameigeiras J J Ramos-Munoz J Navarro-Ortiz PMogensen and J M Lopez-Soler ldquoQoE oriented cross-layer design of a resource allocation algorithm in beyond 3GsystemsrdquoComputer Communications vol 33 no 5 pp 571ndash5822010

[16] I Taboada F Liberal J O Fajardo and U Ayesta ldquoQoE-aware optimization of multimedia flow schedulingrdquo ComputerCommunications vol 36 no 15-16 pp 1629ndash1638 2013

[17] U Ayesta M Erausquin and P Jacko ldquoResource-sharing in asingle server with time-varying capacityrdquo in Proceeding of the49th Annual Allerton Conference on Communication Controland Computing (Allerton rsquo11) pp 377ndash383 Monticello Ill USASeptember 2011

[18] K Thompson G J Miller and R Wilder ldquoWide-area internettraffic patterns and characteristicsrdquo IEEE Network vol 11 no 6pp 10ndash23 1997

[19] M Fiedler T Hossfeld and P Tran-Gia ldquoA generic quantitativerelationship between quality of experience and quality ofservicerdquo IEEE Network vol 24 no 2 pp 36ndash41 2010

[20] A Richards M Antoniades V Witana and G Rogers ldquoMap-ping user level QoS from a single parameterrdquo in Proceedingsof the International Conference on Multimedia Networks andServices Citeseer 1998

[21] E Ibarrola F Liberal I Taboada and R Ortega ldquoWebQoE evaluation in multi-agent networks validation of ITU-TG1030rdquo in Proceedings of the 5th International Conference onAutonomic and Autonomous Systems (ICAS rsquo09) pp 289ndash294Valencia Spain April 2009

[22] S Aalto and U Ayesta ldquoOptimal scheduling of jobs with aDHR tail in the MG1 queuerdquo in Proceedings of the 3rd Inter-national Conference on Performance Evaluation Methodologiesand Tools (ValueTools rsquo08) article 50 Institute for ComputerSciences Social-Informatics and Telecommunications Engi-neering Athens Greece 2008

[23] V Visweswaran ldquoDecomposition techniques for MILPlagrangian relaxationrdquo in Encyclopedia of Optimization pp632ndash638 Springer New York NY USA 2009

[24] I Taboada J O Fajardo and F Liberal ldquoPerformance analysisof scheduling algorithms for web QoE optimization in wirelessnetworksrdquo Network Protocols and Algorithms vol 4 pp 27ndash432012

[25] J A van Mieghem ldquoDynamic scheduling with convex delaycosts the generalized 119888120583 rulerdquoTheAnnals of Applied Probabilityvol 5 no 3 pp 809ndash833 1995

[26] S Sesia I Toufik and M Baker LTE-The UMTS Long TermEvolution FromTheory to Practice Wiley 2011

[27] P Reichl B Tuffin and R Schatz ldquoLogarithmic laws inservice quality perception where microeconomics meets psy-chophysics and quality of experiencerdquo Telecommunication Sys-tems vol 52 no 2 pp 587ndash600 2013

[28] J C Ikuno M Wrulich and M Rupp ldquoSystem level simulationof LTE networksrdquo in Proceedings of the IEEE 71st VehicularTechnology Conference Taipei Taiwan May 2010

Submit your manuscripts athttpwwwhindawicom

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical Problems in Engineering

Hindawi Publishing Corporationhttpwwwhindawicom

Differential EquationsInternational Journal of

Volume 2014

Applied MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical PhysicsAdvances in

Complex AnalysisJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

OptimizationJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Operations ResearchAdvances in

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Function Spaces

Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of Mathematics and Mathematical Sciences

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Algebra

Discrete Dynamics in Nature and Society

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Decision SciencesAdvances in

Discrete MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Stochastic AnalysisInternational Journal of

14 Abstract and Applied Analysis

Acknowledgment

The research leading to these results has received fundingfrom the European Union Seventh Framework Programme(FP72007ndash2013) under Grant agreement 284863 (FP7 SECGERYON)

References

[1] ITU-T P10G100 Amendment 1 (0107) New Appendix I-definition of Quality of Experience (QoE) 2006

[2] S Aalto and P Lassila ldquoFlow-level stability and performance ofchannel-aware priority-based schedulersrdquo in Proceeedings of the6th Euro NF Conference on Next Generation Internet (NGI rsquo10)IEEE June 2010

[3] L KleinrockQueueing Systems vol 2 ofComputer ApplicationsJohn Wiley amp Sons 1976

[4] J Gittins ldquoBandit processes and dynamic allocation indicesrdquoJournal of the Royal Statistical Society B Methodological vol 41no 2 pp 148ndash177 1979

[5] K Avrachenkovt U Ayesta P Brown and E Nyberg ldquoDif-ferentiation between short and long TCP ows predictabilityof the response timerdquo in Proceedings of the 23rd Annual JointConference of the IEEE Computer and Communications Societies(INFOCOM rsquo04) vol 2 p 762 IEEE 2004

[6] U Ayesta M Erausquin and P Jacko ldquoA modeling frameworkfor optimizing the flow-level scheduling with time-varyingchannelsrdquo Performance Evaluation vol 67 no 11 pp 1014ndash10292010

[7] S Aalto U Ayesta and R Righter ldquoProperties of the Gittinsindex with application to optimal schedulingrdquo Probability in theEngineering and Informational Sciences vol 25 no 3 pp 269ndash288 2011

[8] M Puterman ldquoMarkov decision processesrdquo in Handbooks inOperations Research and Management Science vol 2 pp 331ndash434 1990

[9] C H Papadimitriou and J N Tsitsiklis ldquoThe complexityof Markov decision processesrdquo Mathematics of OperationsResearch vol 12 no 3 pp 441ndash450 1987

[10] J Nino-Mora ldquoDynamic priority allocation via restless banditmarginal productivity indicesrdquo TOP vol 15 no 2 pp 161ndash1982007

[11] P Whittle ldquoRestless bandits activity allocation in a changingworldrdquo Journal of Applied Probability vol 25 pp 287ndash298 1988

[12] U Ayesta M Erausquin M Jonckheere and I M VerloopldquoScheduling in a random environment stability and asymptoticoptimalityrdquo IEEEACM Transactions on Networking vol 21 no1 pp 258ndash271 2013

[13] S Khan S Thakolsri E Steinbach and W Kellerer ldquoQoE-based cross-layer optimization for wireless multiuser systemsrdquoin Proceedings of the 18th ITC Specialist Seminar on Quality ofExperience 2008

[14] S Thakolsri S Khan E Steinbach and W Kellerer ldquoQoE-driven cross-layer optimization for high speed downlink packetaccessrdquo Journal of Communications vol 4 no 9 pp 669ndash6802009

[15] P Ameigeiras J J Ramos-Munoz J Navarro-Ortiz PMogensen and J M Lopez-Soler ldquoQoE oriented cross-layer design of a resource allocation algorithm in beyond 3GsystemsrdquoComputer Communications vol 33 no 5 pp 571ndash5822010

[16] I Taboada F Liberal J O Fajardo and U Ayesta ldquoQoE-aware optimization of multimedia flow schedulingrdquo ComputerCommunications vol 36 no 15-16 pp 1629ndash1638 2013

[17] U Ayesta M Erausquin and P Jacko ldquoResource-sharing in asingle server with time-varying capacityrdquo in Proceeding of the49th Annual Allerton Conference on Communication Controland Computing (Allerton rsquo11) pp 377ndash383 Monticello Ill USASeptember 2011

[18] K Thompson G J Miller and R Wilder ldquoWide-area internettraffic patterns and characteristicsrdquo IEEE Network vol 11 no 6pp 10ndash23 1997

[19] M Fiedler T Hossfeld and P Tran-Gia ldquoA generic quantitativerelationship between quality of experience and quality ofservicerdquo IEEE Network vol 24 no 2 pp 36ndash41 2010

[20] A Richards M Antoniades V Witana and G Rogers ldquoMap-ping user level QoS from a single parameterrdquo in Proceedingsof the International Conference on Multimedia Networks andServices Citeseer 1998

[21] E Ibarrola F Liberal I Taboada and R Ortega ldquoWebQoE evaluation in multi-agent networks validation of ITU-TG1030rdquo in Proceedings of the 5th International Conference onAutonomic and Autonomous Systems (ICAS rsquo09) pp 289ndash294Valencia Spain April 2009

[22] S Aalto and U Ayesta ldquoOptimal scheduling of jobs with aDHR tail in the MG1 queuerdquo in Proceedings of the 3rd Inter-national Conference on Performance Evaluation Methodologiesand Tools (ValueTools rsquo08) article 50 Institute for ComputerSciences Social-Informatics and Telecommunications Engi-neering Athens Greece 2008

[23] V Visweswaran ldquoDecomposition techniques for MILPlagrangian relaxationrdquo in Encyclopedia of Optimization pp632ndash638 Springer New York NY USA 2009

[24] I Taboada J O Fajardo and F Liberal ldquoPerformance analysisof scheduling algorithms for web QoE optimization in wirelessnetworksrdquo Network Protocols and Algorithms vol 4 pp 27ndash432012

[25] J A van Mieghem ldquoDynamic scheduling with convex delaycosts the generalized 119888120583 rulerdquoTheAnnals of Applied Probabilityvol 5 no 3 pp 809ndash833 1995

[26] S Sesia I Toufik and M Baker LTE-The UMTS Long TermEvolution FromTheory to Practice Wiley 2011

[27] P Reichl B Tuffin and R Schatz ldquoLogarithmic laws inservice quality perception where microeconomics meets psy-chophysics and quality of experiencerdquo Telecommunication Sys-tems vol 52 no 2 pp 587ndash600 2013

[28] J C Ikuno M Wrulich and M Rupp ldquoSystem level simulationof LTE networksrdquo in Proceedings of the IEEE 71st VehicularTechnology Conference Taipei Taiwan May 2010

Submit your manuscripts athttpwwwhindawicom

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical Problems in Engineering

Hindawi Publishing Corporationhttpwwwhindawicom

Differential EquationsInternational Journal of

Volume 2014

Applied MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical PhysicsAdvances in

Complex AnalysisJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

OptimizationJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Operations ResearchAdvances in

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Function Spaces

Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of Mathematics and Mathematical Sciences

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Algebra

Discrete Dynamics in Nature and Society

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Decision SciencesAdvances in

Discrete MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Stochastic AnalysisInternational Journal of

Submit your manuscripts athttpwwwhindawicom

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical Problems in Engineering

Hindawi Publishing Corporationhttpwwwhindawicom

Differential EquationsInternational Journal of

Volume 2014

Applied MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical PhysicsAdvances in

Complex AnalysisJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

OptimizationJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Operations ResearchAdvances in

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Function Spaces

Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of Mathematics and Mathematical Sciences

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Algebra

Discrete Dynamics in Nature and Society

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Decision SciencesAdvances in

Discrete MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Stochastic AnalysisInternational Journal of