User-centric optimization of caching andrecommendations in edge cache networks

Dimitra Tsigkari and Thrasyvoulos SpyropoulosEURECOM, France, first.last@eurecom.fr

Abstract—In platforms such as Youtube, Netflix, and Spotify,recommendations influence a large share of content consumption.In this context, user experience depends on both the quality ofthe recommendations (QoR) and the quality of service (QoS) ofthe delivered content. Nevertheless, network decisions (such ascaching) affecting QoS are usually made without explicit knowl-edge of the recommender’s actions. Similarly, recommendationdecisions are made without considering the potential deliveryquality of the recommended content. In this paper, we proposeto jointly optimize both caching and recommendations on top ofa generic network of caches, towards maximizing the quality ofexperience (QoE). This is in line with the recent trend for largecontent providers to also act as CDN owners. We formulate thisjoint optimization problem and prove that it can be approximatedup to a constant. To the best of our knowledge, this is the firstpolynomial algorithm to achieve a constant approximation ratiofor the joint problem.

I. INTRODUCTION

A. Motivation

Caching has received the research community’s attention notonly in wired networks (in the context of Content DeliveryNetworks) but also in wireless networks (in the context of5G and beyond). Placing content in caches close to the userensures a better delivery with higher bitrate, shorter delay,etc. [1]. Moreover, it can alleviate the backhaul link traffic. Inplatforms of video streaming services, low bitrate can lead toincrease in abandonment rate [2]. Therefore, to ensure a long-term engagement of the user to the platform, content providers(CPs) need to pay attention to the overall quality of experience(QoE) of a delivered content. It is increasingly understood thatthis QoE largely depends on both the interest of the user inthe content delivered (a minimum assumed requirement forany CP) and the QoS of the actual delivery [3].

At the same time, recommendation systems greatly affectuser requests: in Netflix, 80% of requests come from therecommendations that appeared to the user [4]. Similarly, theimpact of recommendations on Youtube (through the relatedvideo section) has been shown to be significant [5]. Thetraditional role of a recommendation system has been tomake personalized recommendations to the user based solelyon content interest, i.e., bringing forward items from a vastcatalogue that best match her interests. However, the QoS,e.g., where the content is cached and whether this allows therecommended content to be delivered at low or high streamingquality, does not usually factor into the decision of mostrecommenders.

At first glance, content caching and recommendation sys-tems appear to be independent, since they are usually governedby two different entities (CP and network provider, or CPand 3rd party CDNs like Akamai). However, major CPs likeNetflix and Google started partnering with Internet ServiceProviders (ISPs) to implement their own CDN solutions insidethe network: Netflix Open Connect and Google Global Cache.This brings together content caching and recommendations, asnow the same entity can control and coordinate both, towardsensuring high user experience and minimizing delivery costsand network congestion.

As a result, some recent works propose caching and rec-ommendation policies that take into account their interplay[6], [7], [8], [9], [10], [11], [12]. Nevertheless, many of theseworks still focus on one side of the problem, e.g., network-friendly recommendations [7], [12], or recommendation-awarecaching policies [8]. Some works that do try to modify boththe caching and recommendation policies are usually basedon heuristics [10], [11]. Hence, a formal joint treatment of thetwo problems is largely missing.

B. Our approach and contributions

To this end, the main goal of this paper is to formulateand analytically study the problem of optimizing both sets ofcontrol variables jointly: (i) what content to store at each ofa generic network of caches (this can correspond to a “femto-caching” wireless setup, or a standard CDN-like hierarchicalnetwork), and (ii) what content to recommend to each user,based on the location of the user in relation to the cachingnetwork topology and the user’s predicted preferences forcontents. In this direction, the main contributions of this workare the following:

• We introduce a simple yet generic metric of quality ofexperience (QoE) for a recommendation driven contentapplication, that depends on both where content is stored(a proxy of QoS [1]) and the quality of recommendationsgiven to the users (the traditional goal of any recom-mender). Based on this model, we formulate the problemof optimally choosing both sets of variables towardsmaximizing average user QoE.

• While such joint caching and recommendation problemshave been shown before to be NP-hard, to the best of ourknowledge we provide the first polynomial algorithm forthe problem that has an approximation guarantee (in facta constant one) for both equal and variable sized contents.

• We investigate the performance of our algorithm throughboth synthetic and real data, and compare it with thestate-of-the-art. Our results both validate our analysis, aswell as demonstrate that significant performance gainscan be achieved with respect to both “baseline” policies(e.g., separate optimization, as is the case today) as wellas compared to the best existing heuristics for the jointproblem.

II. PROBLEM SETUP

A. Caching Network

We consider a set of C caches with capacity Cj , j =1, . . . , C and a content catalogue K. We assume that Cj � |K|,j = 1, . . . , C, as it is common in most caching setups. We willconsider both equal-sized contents (as it is commonly assumedin literature, e.g. in [13], [14]) and contents of different sizes(as in [10]). In the second case, we denote by σi the size ofcontent i.

Definition 1 (Caching variable). We let xij be the binaryvariable, where xij = 1 when the content i is cached in cachej, and xij = 0 otherwise. We let matrix X = {xij}i,j .

We consider a set U of users, each of which has access to asubset of caches. We denote this set as C(u) for user u ∈ U . Arequest for content i by user u is served by one of the cachesbelonging to C(u) where the requested content is stored, i.e.,by one of the caches of the set

{j : j ∈ C(u) and xij = 1}.

The access to a cache could be over multiple links (as in thecase of hierarchical caching or ICNs) or direct (e.g. wirelessconnectivity to a nearby small cell [13]). For the purposesof our analysis, all such networks can be represented as ageneric bipartite graph between users and (associated) caches,as shown in Fig. 1. Specifically, we assume that every edgeof this graph has a weight suj , which denotes the expectedstreaming rate that can be supported between user u and cachej. This rate might differ from cache to cache, and mightdepend on channel quality, number of hops, scheduling policy,congestion level, etc.1

Finally, we assume that there is a large cache C0 that fitsall the contents, i.e. xi0 = 1 for all i ∈ K, and is accessibleby all users, i.e. C0 ∈ C(u), for all u ∈ U . This could bea large cache deep(er) in the network. For this reason andw.l.o.g., we let su0 < suj , for all j = 1, . . . , C and u ∈ U ,as it is commonly assumed (e.g. in [13], [17]). This setup isgeneric and could capture a variety of caching networks, suchas femto-caching framework [13], hierarchical CDN networks[18] etc.

1All these factors are important for deciding the specific streaming rates,and have been studied in [15] and [16]. However, these are orthogonal to ourmodel, provided that they vary at a different, longer time scale. We will getback to this point later in the paper.

B. RecommendationsA list of Nu recommended contents appears to the user

u ∈ U . This number may vary from user to user dependingon the device used, which is the case in Netflix [4], forexample. The recommendations are personalized and mightdiffer from one user to the other. The chosen recommendationsmight depend on various factors such as user ratings (e.g.,via collaborative filtering), past user behavior, viewing times,etc. ([19], [20]). State-of-the-art recommenders usually firstassign an interest value/utility or “score” (or “rank”) to eachcontent for each user u, and then select the Nu items withthe highest scores [20], [21], [22]. Our model uses theseinterest values/utilities, denoted as rui ∈ [0, 1], as input to ourproblem. For example, this could be the (normalized) predictedscore through collaborative filtering of content i for user u.

Definition 2 (Recommendation variable). We let yui ∈ {0, 1}denote the binary variable for content i being recommendedto user u (yui = 1) or not (yui = 0). We denote by Y the|U| × |K| matrix of yui.

Then, the equation∑i∈K yui = Nu, for all u ∈ U , captures

the fact that Nu contents are recommended.

C. User modelWe assume that the user makes content requests, affected

by the aforementioned recommendations, according to thefollowing model:• with probability αu the user will request a recommended

content. For simplicity, we assume that each of the Nurecommended items will be chosen with equal probabilityby the user;

• with probability (1−αu) the user will ignore the recom-mendation system and request a content i of the cataloguewith probability pui.

Essentially, αu captures the percentage of time a user u tendsto follow recommendations. For example, it is estimated thatau = 0.8, on average, on the Netflix platform [4], but it canof course differ among users. Assuming prior knowledge ofuser’s disposition to follow the recommendation is commonin related works (e.g., [7], [10]) and also in other works onrecommendation systems (e.g., [23]). In practice, au mightchange over longer time intervals both because of intrinsicchanges to user behavior or due to decreasing/increasing trustin the recommender. Nevertheless, in this work we assume thatour optimization happens at a smaller time scale, for whichwe can assume that the users au parameter is roughly constant(but can be recalibrated at longer intervals)

Furthermore, the assumption that each recommended con-tent will be clicked with equal probability 1/Nu is alsocommon in related works (e.g., [7], [10]), and might holdin scenarios where the recommended items are “unknown”to the user, and hence cannot evaluate their interest value,before requesting them. However, in other scenarios clickprobabilities might be position-based [24] or further dependon the rui values. We defer the investigation of such elaboratemodels to future work.

As for the probabilities pui, they capture the probability ofuser u requesting the content i outside of recommendations(e.g. through the search bar). This could be an arbitrarydistribution over the catalogue (e.g., with probability massonly on content the user already “knows”). Alternatively, givenour earlier definition of interest values rui a reasonable choicecould also be the normalized values:

pui = rui/∑k∈K

ruk. (1)

D. Example

To better elucidate our model thus far, we present here asmall-scale example together with Fig. 1 that illustrates thevariables and the parameters defined above. We consider anetwork of C = 2 caches of capacity 2, and a large cacheC0 containing the entire catalogue that consists of |K| = 9equal-sized contents. As shown, cache 1 contains contents 1and 4 and therefore x11, x14 = 1, while x1j = 0 for any othercontent j. There are 3 users present in the network. An edgebetween a user u and a cache j means that user u can fetch acontent from cache j. For example, for user 1, we have thatC(1) = {0, 1}, while for user 2, we have C(2) = {0, 1, 2}.Note again that such an edge might actually correspond to apath of multiple physical links. The rate at which the user canfetch a content is indicated as an edge weight.

We assume, in this example, that the recommendationsystem recommends Nu = 1 content to every user (illus-trated with a dashed-line arrow). For example, content 4 isrecommended to user 1, which means that y14 = 1. If user1 does indeed request the content 4, then it can be streamedfrom cache 1 at rate s11. However, if user 1 requests, say, thecontent 2, then it will be fetched from cache 0 at a (lower)rate s10. Lastly, arrows from users to recommended contentsdisplay the probabilities au.

Fig. 1. Illustration of the example in Section II-D.

E. Quality of experience (QoE)

In the context of streaming contents in platforms such asNetflix or Spotify, it has been traditionally assumed that theuser will be satisfied by the provided service if the recommen-dation system is tailored to his/her tastes. However, it has beenincreasingly observed that quality of service (e.g., low stream-ing rates, rebufferings, etc.) greatly affects user experience and,most importantly (for CPs), retention/abandonement rate ([25],[2]). In fact, some recent experimental evidence suggests thatusers might be willing to tradeoff (some) content relevance for(better) QoS [26]. In this direction, we model the user qualityof experience as a twofold quantity: one part relates to thequality of recommendations shown to the user related to therecommended content utility rui; the second part relates tothe streaming rate experience, which depends on where thecontent is cached (xij) and what the expected streaming rate2

is from that cache (suj).

Definition 3 (Quality of Recommendations - QoR). Thequality of recommendations, as perceived by user u, is equalto∑i∈K yuiϕ(rui), where ϕ is any non-decreasing function.

The function ϕ represents the impact of a content’s rel-evance rui in the overall experience. It could be a lin-ear function, or, more commonly, a concave function (e.g.log(rui)) to capture diminishing returns beyond a minimumcontent relevance. Moreover, we can also demand a minimumquality rmin for any recommended content i featuring therecommendation list3. We can then define ϕ as follows

ϕ(rui) =

{log(rui) if rui > rmin,

−∞ otherwise.(2)

Regarding the impact of streaming rate, the actual experi-enced rate depends on which cache it is streamed from. Weassume, as in [13], that a content i requested by user u willbe fetched by the “best” connected cache that stores it. Toformalize this, we use the following definition:

Definition 4 (Ordered streaming rates). If C(u) is the set ofcaches that user u has access to, we let su(1) = max{suj , j ∈C(u)} denote the maximum rate for user u. Similarly, su(2)

denotes the second highest rate for u, and so forth4.

By definition, su|C(u)| = su0, for every u ∈ U , since weassumed that su0 < suj , for all j = 1, . . . , C.

Based on this, we can prove the following lemma that givesthe streaming rate per content for user u and the expectedstreaming rate, as a function of the caching policy (xij), therecommendation policy (yui) and the users preferences (rui).

2While we call, for simplicity, the value suj the “streaming rate” this couldalso correspond to any measure of streaming experience that includes initialbuffering delay, rebufferings and other phenomena ([27], [28]).

3This quantity can serve as an additional safeguard to support the earlierassumptions about the user model: e.g., if any recommended content’s interestis above rmin then the user’s trust in recommendations au will not becompromised.

4As the rates suj are sorted for every user, the notation su(k)u , k =1, . . . , |C(u)|, would be more appropriate. For simplicity, we drop the sub-index u here.

Lemma 1 (Quality of Service - QoS). The rate at which useru ∈ U will download content i ∈ K upon request (for a givencache allocation X) is equal to:

su(X, i) :=

|C(u)|∑j=1

[su(j)xi(j)

j−1∏l=1

(1− xi(l))], (3)

where xi(j) are similarly the caching variables assuming arate-based ordering. Moreover, the expected streaming rate(that measures the QoS) for a user u is equal to:

su = αu∑i∈K

yuiNu

|C(u)|∑j=1

[su(j)xi(j)

j−1∏l=1

(1− xi(l))]

+(1− αu)∑i∈K

pui

|C(u)|∑j=1

[su(j)xi(j)

j−1∏l=1

(1− xi(l))]. (4)

Proof. For a requested content i ∈ K, the term∏j−1l=1 (1 −

xi(l))xi(j) captures the fact that i will be retrieved by thecache (j) (i.e. the cache with the j-th highest rate) for lack ofany other cache with higher rate in C(u) where the content iscached (i.e. xi(l) = 0, l < j). Then, this request will be servedto the user at rate su(j). Of course, if i is not cached in anycache, then it will be retrieved by the large cache C0, whichis ranked last, resulting in low streaming rate. Essentially,su(X, i) is the highest rate associated to content i for useru among all the locations where i is cached.

Then, the formula of the expected streaming rate easilyfollows by taking into account the two possible ways of howthe user makes a request, as explained in Section II-C.

Remark 1. When estimating the QoS, instead of suj we canconsider ψ(suj) for any non-decreasing function ψ of suj , aswe did for the recommendation quality. Thus, we can calculatesu by replacing su(j) with ψ(su(j)). Without loss of generality,we assumed here that ψ is the identity function, i.e. ψ(suj) =suj . Note also that if we let

ψ(suj) =

{1 if j ∈ C(u) \ C0,

0 otherwise,(5)

then the formula in (4) will estimate the cache hit rate for the(small) caches: upon a request, it counts 1 if the content iscached and therefore, retrieved from a small cache nearby.

Definition 5 (QoE function). The quality of experience foruser u ∈ U as a function of the caching and recommendationvariables is defined as

su + βu∑i∈K

yuiϕ(rui),

where su is given by (4) and βu > 0 is a tuning parameter.Then the aggregate quality of experience over all users is equalto:

f(X,Y ) :=∑u∈U

[su + βu

∑i∈K

yuiϕ(rui)]. (6)

Modeling QoE in this fashion implies a tradeoff betweenQoS and QoR, as evidenced in the earlier discussed works.

The value of βu, which might differ from user to user, capturesthe importance of each factor. High βu means user u is moresensitive to recommendation quality, while low βu that (s)heis more sensitive to streaming quality. It is beyond the scopeof this paper to investigate good choices for βu or ϕ (or ψ inRemark 1). Instead, our focus is to propose efficient algorithmsfor any values and conforming functions.

Moreover, assuming βu = β, f could be seen as a Paretooptimality formulation capturing two objectives (QoS andQoR). We will see such plots in Section IV. Similarly, onecould imagine more complex models (e.g., based on machinelearning and real data). Such more advanced models go beyondthe scope of this paper. Such more advanced models go beyondthe scope of this paper. We believe that the above modelis generic enough, as a first step, to capture a number ofinteresting tradeoffs.

TABLE INOTATION SUMMARY

Notation DescriptionK catalogue of contentsU set of users in the networkC0 large cache containing the entire catalogueC number of caches in the network (C0 is excluded)Cj capacity of cache j, j = 0, · · · , CC(u) set of caches that user u communicates withrui utility of content i for user usuj streaming rate between user u and cache jσi size of content iNu number of recommended contents for the user uαu prob. that user u follows the recommendationspui prob. that user u requests content i while not

following the recommendationsxij caching variable, xij = 1 when content i is cached

in cache j, and xij = 0 otherwiseyui recommendation variable, yui = 1 when content i

is recommended to user u, and yui = 0 otherwise

F. Joint recommendation and caching

Within the above problem setup, we ask the followingquestion: How can we make caching and recommendationdecisions in order to maximize QoE?

To better understand the tradeoffs involved, let us look at atoy example involving only a single cache, depicted in Fig. 2,and let us consider two “naive” policies:

Policy C, for “Conservative”. This policy caches the Cj mostpopular contents (among all users connected to the cache j);it then recommends to each user u the Nu contents with thehighest utility for this user, regardless of whether they arecached or not. This policy captures in fact today’s status quo.

Policy A, for “Aggressive”. This policy has the same cachingpolicy as policy C, but recommends only cached contents (themost relevant to user u among them). It is closer to cache-friendly recommendation policies like the one proposed in [8].

Note that both policies take the caching and recommenda-tion decisions separately. In this example, we will attempt toshow the benefits of a policy that takes jointly these decisions.

Fig. 2. Toy example presented in Section II-F. On the left: illustration of thenetwork together with the caching and recommendation decisions made bythe A, C and J policies. On the right: the matrix of content utilities per user.

Referring to Fig. 2, suppose we have a catalogue of 4 equal-sized contents and 3 users, all connected to the large cache C0

(not shown in the figure, for simplicity) that contains all filesand a smaller cache C1 of capacity 1. All users can downloada content from C1 with rate 3 Mbps while the rate from C0 is2 Mbps. We assume that the number of recommended itemsis Nu = 1 and αu = 1 for all users, i.e. user requests arebased exclusively on recommendations. We depict the utilitiesrui for each content i and user u on the right side.

Both A and C policies would cache the item with the highestaggregate utility, i.e. content 1. Policy A would then recom-mend this single cached item to all users. Policy C wouldinstead recommend the item with highest utility per user,namely contents 2, 3 and 4 respectively. It is easy to see thatpolicy C would lead to better recommendation quality, whilepolicy A would lead to better streaming rate. Nevertheless,neither policy is optimal with respect to maximizing the QoE(eq. (6)).

A better option would be to cache content 2, observing thatthis would then facilitate the recommender. More precisely,it allows one to recommend content 2 to both users 1 and 3,achieving cache hits for them with maximum or close to max-imum recommendation quality (for user 1 and 3 respectively).Instead, for user 2, the content 3 is recommended (with utilityr23 = 0.5), since content 2 would seriously degrade the user’sQoR (r22 = 0.05 only). This policy, which we refer to as “J”for Joint in Fig. 2, outperforms both A and C in this examplein terms of QoE (for any beta and conforming φ functions).

In this toy example, it is easy to guess how to outperformthe simple policies A and C (or even find the optimal one).However, this task becomes significantly harder for biggerscenarios. To this end, in the next section, we formulate thisproblem rigorously, analyze its complexity, and propose, tothe best of our knowledge, the first guaranteed approximationalgorithm for this problem.

III. PROBLEM FORMULATION AND ANALYSIS

The optimization problem we are targeting is the following:

QoE problem.

maximizeX,Y

f(X,Y )

subject to∑i∈K

σixij ≤ Cj for every j = 1, . . . , C; (7)∑i∈K

yui = Nu for every u ∈ U ; (8)

xij , yui ∈ {0, 1}, (9)

where, according to Eq. (3), (4) and (6), f(X,Y ) is equal to∑u∈U

∑i∈K

[αuyuiNu

su(X, i)+(1−αu) pui su(X, i)+βuyuiϕ(rui)]

and su(X, i) :=∑|C(u)|j=1

[su(j) xi(j)

∏j−1l=1 (1 − xi(l))

]. The

constraints in (7) are the capacity constraints for every cache.In the case of equal-sized contents, (7) suggest that no morethan Cj items can fit in cache j, and the constraints in (8)suggests that each user receives Nu recommendations. Finally,as expressed in (9), xij and yui are binary/decision variables.

The following Lemma shows that the QoE problem easilyreduces to the standard femto-caching problem [13], so it’s atleast as hard.

Lemma 2. The QoE problem is NP-hard.

Proof. We consider a special case of the QoE problem where:• The recommendation variable Y is fixed. In this way, the

caching variable is the only optimization variable and theconstraint (8) is dropped;

• αu = 0, for all u ∈ U , i.e. there are no recommendationsor the user always ignores them;

• suj = s1 > su0 for all u ∈ U and j ∈ C(u) \ C0, i.e.the rate from the caches that each user can connect is thesame except for the rate from the large cache C0;

• The contents are equal-sized, i.e. σi = 1, for all i ∈ K;• The probability of requesting a content while not follow-

ing the recommendations is the same for all users, i.e.pui = pi.

It suffices to prove that the corresponding decision problemis NP-complete [29]. The decision problem can be stated asfollows: Given the input parameters of the above QoE problemand a real number D ≥ 0, is there a feasible cache placementX such that f(X,Y ) ≥ D? This problem is essentially thehelper decision problem as defined in [13], which is shown tobe NP-complete.

A. Intuition on joint optimization

As we saw in Lemma 2, even just the caching part (i.e. max-imizing in variable X) of the QoE problem is hard to solve. Forthis simpler problem, the authors in [13] propose algorithmswith approximation guarantees by exploiting submodularityproperties of the objective. However, these algorithms do notaccount for the recommendation part of the QoE problem(variable Y ) and therefore the approximation guarantees donot extend to the joint problem.

One could be tempted to extend the methodology in [13]by using both sets of variables X and Y as the ground set.However, the authors of [10] prove that a subcase of the QoEproblem (when βu = 0) is not submodular in X and Y .

Furthermore, the authors of [8] consider a number of prob-lem variants where the caching decision is ”recommendation-aware”. They show that this problem is hard even for onecache, but manage to retrieve submodularity properties anduse the methodology of [13] to derive efficient algorithmswith approximation guaranties. However, their objective andproblem setup does not contain recommendation variables(among other things, the recommender’s actions are fixed, andthe caching policy simply knows what the recommender woulddo). It is thus significantly different than the QoE problem.Finally, a brief qualitative comparison of these works can befound in Table II.

TABLE IISTATE-OF-THE-ART WORKS ON CACHING AND/OR RECOMMENDATION

SYSTEMS OPTIMIZATION

Related Variables How many Approx.Works Caching Recomm. caches? guarantees[13] 3 7 Network 3[8] 3 7∗ Network 3[30] 3 3 Single cache 7[10] 3 3 Network 7

This work 3 3 Network 3∗In [8], although the problem formulation does not contain any recommendationvariable, the caching variable is “recommendation-aware”.

This discussion raises the question of whether the QoEproblem can be efficiently approximated and how. In thenext section, we prove that this is indeed the case. By firstconsidering something akin to a primal decomposition [31] ofthe original problem: rather than handling variables X and Yat the same time as the ground set, we show that:

(i) for the problem on variables Y , i.e., fixing variables X(“inner” problem), the global maximizer can be foundefficiently;

(ii) the problem on variables X (“outer” problem), given theglobal maximizer of Y (for any X), is in fact submodular.

This property will allow us to devise a combined algorithm forthe joint problem that is polynomial in the problem size and,somewhat surprisingly, retains the approximation guaranteesof the much simpler ”caching-only” problems considered in[13] and [8].

B. Towards efficient algorithms

The key to our methodology is the following lemma.

Lemma 3. The QoE problem is equivalent to the problem:

Outer problem.

maximizeX

f∗(X) := f(X, argmaxY

f(X,Y )) (10)

subject to (7), (8), and (9).

The equivalence of the two problems follows straightfor-wardly from the well known identity [32]:

maxX,Y

f(X,Y ) = maxX

(maxY

f(X,Y )). (11)

1) Inner problem and algorithm: The first step would be tofind a closed-form expression for f∗ for any cache allocation,i.e. matrix X . Hence, given the cache allocation X , theproblem of choosing the recommendation policy (matrix Y ) isthe problem of finding f∗(X), as defined in (10). We formulatethe following problem:

Inner problem.

maximizeY

f(X,Y )

subject to (8) and yui ∈ {0, 1}.

The following lemma will help us tackle the inner problem.

Lemma 4. If F ∗u (X) := maxY

(su + βu

∑i∈K yuiϕ(rui)

), for

any u ∈ U and any cache allocation X , then

f∗(X) =∑u∈U

F ∗u (X).

Proof. The inner problem can be decoupled into |U| prob-lems since, given a cache placement X , the recommendationdecisions (variable Y ) for a user do not interfere with thedecisions of the other users. Note also that the constraint in(8) is decoupled for every user.

Given Eq. (3) in Lemma 1, we can write F ∗u (X) as follows.

F ∗u (X) = maxY

(∑i∈K

yui( αuNu

su(X, i) + βuϕ(rui)))

+ (1− αu)∑i∈K

su(X, i)pui. (12)

Note that in the above expression, the summand with coef-ficient (1 − αu) does not contain the variable Y . Therefore,in order to find a closed-form expression for f∗(X) givena caching placement X , we need to solve the followingmaximization problem:

maximizeY

∑i∈K

yui[ αuNu

su(X, i) + βuϕ(rui)]

subject to∑i∈K

yui = Nu,

yui ∈ {0, 1}.

As a first step, we introduce the notion of V-value of acontent, which is its value in terms of the inner problem.

Definition 6 (V-value and ordered V-values). We define, asV-value of a content i ∈ K for user u ∈ U and for a givencache allocation X , the quantity

Vui(X) :=αuNu

su(X, i) + βuϕ(rui), (13)

where su(X, i) is defined in (3). Similar to Def. 4, we definethe ordered Vui (sorted in decreasing order) as the orderedsequence {Vu[k]}k∈K5.

Lemma 5. For a given cache allocation X , we consider thematrix Y ′ such that y′u[k] = 1 for k = 1, . . . , Nu, and y′u[k] =0 otherwise, where [k] is the content index associated to thek-th highest V-value for any user u ∈ U . Then

F ∗u (X) =

Nu∑k=1

Vu[k](X) + (1− αu)∑i∈K

(su(X, i)pui

)and f∗(X) = f(X,Y ′) =

∑u∈U

F ∗u (X).

In words, the above lemma states that the optimal solutionfor the inner problem is to recommend to every user u theNu contents with the highest V-value associated to the cacheplacement.

Proof. It is straightforward to prove the result above throughcontradiction, i.e., assuming some content m with lower V-value than the Vu[Nu] should have been included instead.

Based on the previous lemma, here is a summary of thealgorithm that finds the solution for the Inner Problem:

Inner algorithm (subroutine)Input: U , K, Nu, X , {βu}, ϕ, {αu}, {rui}, {suj}

1 Start with empty matrix Y2 for every user u ∈ U do3 for every content i ∈ K do4 Calculate Vui;5 end6 Sort Vui in decreasing order : {Vu[k]}

|K|k=1 ;

7 Set yu[k] = 1 for k = 1, · · · , Nu;8 end9 Return Y

2) Complexity of the inner algorithm: For a given place-ment of cached items X , f∗(X) can be calculated in com-plexity O(|U| · |K| · log |K|). The internal for loop (lines 3−5)consists of |K| calculations. Next, the worst-case complexityfor the sorting step is O(|K| · log |K|) and the complexity ofthe assignment step (line 7) is O(Nu), where Nu � |K|. Sincethese steps are repeated for every user, the total complexity ofthe inner algorithm is at most O(|U| · |K| · log |K|).

3) Outer problem and submodularity: We proved that theoptimal Y can be found efficiently for the inner problem, givenany cache allocation X . We will now prove some interestingproperties of the outer problem (defined in Lemma 3) that willlead us to an algorithm (with approximation guarantees) forthe QoE problem.

First, we need to extend f∗ as a set function. To do so, wedefine the ground set that corresponds to X . More precisely,

5We do not use the same notation as in Def. 4 because the ordering hereis done with respect to the V-value and not the streaming rate. In general,Vu(k)(X) 6= Vu[k](X), for all u ∈ U and k = 1, . . . |K|.

for any matrix X we define the corresponding placement PXof cached items in the network by

PX := {(i, j) : xij = 1, i ∈ K, j = 1, . . . , C}. (14)

Essentially, PX consists of the pairs (content, cache) of allthe contents cached in the (small) caches in the network. Notethat, since, by definition, the large cache C0 contains the entirecatalogue (xi0 = 1, for all i ∈ K), X is considered as a |K|×Cmatrix. In other words, PX belongs to the set

P := P (K × {1, . . . , C}), (15)

where P (K×{1, . . . , C}) is the powerset of K×{1, . . . , C}.Inversely, given a placement P , we can define the correspond-ing matrix XP such that xij is equal to 1, for every pair (i, j)in P , and 0 otherwise. Hence, from now on, X and P will beused interchangeably to denote the content allocation acrossthe network of caches. We can also define the subsets of aplacement P representing the storage of the cache m:

P (m) := {(i,m) ∈ P}, (16)

or in terms of the associated matrix X:

P (m) = {(i,m) ∈ P : xim = 1}. (17)

We can thus extend the definitions of F ∗u , f∗, su and Vuito the ground set P . We first prove that F ∗u is monotoneincreasing.

Lemma 6. The set function F ∗u is monotone increasing for allu ∈ U .

Proof. We consider two placements P and Q such that P ⊆Q ⊆ P and we will prove that F ∗u (P ) ≤ F ∗u (Q). Since P ⊆Q, Q contains at least all the pairs (content, cache), that Pcontains. This implies that the contents cached in P are alsoavailable in Q with the same or better streaming rate, i.e.,

su(P, i) ≤ su(Q, i), for all i ∈ K. (18)

To see this, we assume by contradiction that there exista content η such that su(P, η) > su(Q, η). Then by thedefinition of su(P, η) (Eq. (3)), it follows that there exist acache l such that (η, l) ∈ P , (η, l) 6∈ Q and sul > suj for allj in {j : (η, j) ∈ Q}. But this would imply that P 6⊆ Q.

Next, by Def. 6, the following inequalities are true

Vui(P ) ≤ Vui(Q), (19)Vu[k](P ) ≤ Vu[k](Q), for all i, k ∈ K. (20)

Finally, from (12), it follows that F ∗u (P ) ≤ F ∗u (Q).

Next, we define the marginal gain of F ∗u and we state animmediate consequence of Lemma 6.

Corollary 1 (Marginal gain). For a given placement of cacheditems P , and a pair (i, j) ∈ (K×{1, . . . , C}) such that (i, j) 6∈P , we denote by

∆F ∗u (P, (i, j)) = f∗(P ′)− f∗(P ),

where P ′ := P ∪ {(i, j)}, the marginal gain of F ∗u at P withrespect to (i, j). Then,

∆F ∗u (P, (i, j)) ≥ 0.

Lemma 7. The set function F ∗u is submodular6 for all u ∈ U .

Proof. In order to prove submodularity we consider twoplacements A and B such that A ⊆ B ⊆ P and (i, j) ∈ P\B.We then need to prove that

∆F ∗u (A, (i, j)) ≥ ∆F ∗u (B, (i, j)). (21)

In other words, the marginal benefit of adding content i to thecache j in A is greater than or equal to the marginal benefitin B. This means that the function F ∗u has the diminishingreturns property.

The detailed proof can be found in Appendix A.

Lemma 8. The set function f∗, as defined in (10), is monotoneincreasing and submodular.

Proof. We proved in Lemma 4 that f∗(X) =∑u∈U F

∗u (X).

It is easy to prove that monotonicity and submodularityare properties that are preserved under non-negative linearcombinations [34]. Therefore, the result is an immediateconsequence of Lemmas 6 and 7.

C. QoE algorithms and guaranteesIn the previous section, we managed to prove through the

decomposition

maxX,Y

f(X,Y ) = maxX

f∗(X) = maxX

f(X, argmaxY

f(X,Y ))

that f∗(X) is submodular for any cache allocation X . Thetheory on submodularity optimization suggests that differentgreedy algorithm variants give constant approximations for theouter problem, and thus for the QoE problem (by Lemma 3).In fact, the factor of approximation depends on the type ofconstraints. In particular, in the QoE problem formulation, theconstraints in (7) (cache capacity constraints) lead to differentalgorithms and approximation guarantees when the contentsare equal-sized, i.e., σi = 1 for all i ∈ K .

1) The case of equal-sized contents: Taking into consider-ation the submodularity of f∗, we define a greedy algorithmthat we call the QoE algorithm. This algorithm starts with aplacement P consisting of empty caches (except for the largecache that contains the entire catalogue) and greedily fills oneby one all the available shots. In every round of selection, itcalculates the marginal gain of f∗ at P with respect to at most|K| contents by solving the Inner Algorithm (as subroutine).It then selects and adds to P the content that maximizes thegain (ties broken arbitrarily), before the next selection roundbegins. The algorithm is summarized below.

Theorem 1. If we let OPT denote the optimal solution ofthe QoE problem with equal-sized contents, and X∗, Y ∗ theoutput of the QoE algorithm, then

f(X∗, Y ∗) ≥ 1

2OPT. (22)

6For definition, see [33] or [34].

QoE algorithm (for equal-sized contents)Input: C, {Cj},U , K, {Nu}, {suj}, {rui}, {βu}, {αu}

1 Start with empty caches, i.e. P = ∪Cj=1P(j), where

P (j) = ∅, for all j = 1, . . . , C2 Outer algorithm:3 for every cache j = 1, . . . , C, do4 while j is not full, i.e. |P (j)| < Cj , do5 for every content i ∈ K such that (i, j) /∈ P (j),

do6 Estimate ∆f∗ (P, (i, j)) by calling Inner

Algorithm(X).7 end8 Sort ∆f∗ (P, (i, j)) and

I := argmaxi∆f∗ (P, (i, j)).

9 Then add I to the cache j, i.e.P (j) ← P (j) ∪ (I, j).

10 end11 end12 Return X∗ ↔ P, Y ∗ = f∗(X∗)

Proof. When σi = 1, for all i ∈ K, the constraints in (7) arematroid constraints, as in [13]. Then, the theory on submod-ular optimization [33] suggests that a 1/2 approximation isachievable.

We should note here that a better than 1/2-approximationcan be achieved in some special cases (of submodular max-imization with matroid constraints) or with a larger runningtime, see for example [35] or [36].

2) The general case: different-sized contents: The fun-damental difference between the two cases is the capacityconstraints. The constraints in (7) in the general case areknapsack constraints. However, the QoE algorithm as definedabove is oblivious of content size. The following greedyalgorithm is an adaptation of the QoE algorithm that takessize into account. More precisely, it adds to the cache, inevery round of selection, the content that maximizes the ratioof marginal gain to the content size. It is summarized below.

However, in the case of different-sized contents, both QoEand s-QoE algorithms can perform arbitrarily badly [37].According to the following theorem, it suffices to choose thebest outcome between the two algorithms in order to achieveconstant approximation guarantees.

Theorem 2. If we let OPTs denote the optimal solution ofthe QoE problem in the general case (different-sized contents),(X∗, Y ∗) the output of the QoE algorithm, and (Xs, Ys) theoutput of the s-QoE algorithm, then

max{f(X∗, Y ∗), f(Xs, Ys)} ≥1− 1/e

2OPTs. (23)

The theorem above is given in [37]. It is worth mentioningthat, in [38], the author describes an algorithm with betterapproximation ratio but at the cost of higher computationalcomplexity. We elaborate on the complexity of the algorithmsin the next section.

s-QoE algorithm (size-aware)Input: C, {Cj},U , K, {Nu}, {suj}, {rui}, {βu},

{αu}, {σi}1 Start with P = ∪Cj=1P

(j), where P (j) = ∅, for allj = 1, . . . , C;

2 Outer algorithm:3 for every cache j = 1, . . . , C, do4 while j is not full, i.e.

∑k∈P (j) σk < Cj , do

5 for every content i ∈ K such that (i, j) /∈ P (j)

and σi ≤ Cj −∑k∈P (j) σk, do

6 Estimate δf∗(P, (i, j)) := ∆f∗(P,(i,j))σi

7 by calling Inner Algorithm(X).8 end9 Sort δf∗ (P, (i, j)) and

I := argmaxiδf∗ (P, (i, j)).

10 Then add I , i.e., P (j) ← P (j) ∪ (I, j).11 end12 end13 Return X∗ ↔ P, Y ∗ = f∗(X∗)

3) Complexity and implementation speed-ups.: It is easyto see that the complexity of both the QoE and the s-QoEalgorithms is the same.

The algorithms need to run C ·∑j∈C |Cj | times (at most,

in the general case) in order to fill all helper caches. On eachiteration, they evaluate the marginal gain of |K| contents whileretaining the maximum of them. For every content, they callthe inner algorithm of complexity O(|U|·|K|·log(|K|)). There-fore, the total complexity of the QoE and s-QoE algorithmsis

O(|U| · |K|2 · C ·∑j∈C|Cj | · log |K|).

In practice, this can be reduced to O(|U|·|K|2·C ·∑j∈C |Cj |) if

the values Vui are calculated offline for all contents cached inthe large cache C0 and sorted. Then, every time a new contentis placed in a cache, its Vui is updated and it is inserted in asorted list in at most O(|K|) steps.

Implementation-wise, there is a way to speed up bothalgorithms by using the so-called lazy evaluations method([39], [37]). This method takes advantage of the monotonic-ity and submodularity of the objective function in order toavoid unnecessary calculations in the selection process ofthe caching placement. More precisely, the marginal value∆f∗(Pk, (e, j)) is bounded above by ∆f∗(Pk−1, (e, j)) forall e ∈ K where Pk−1 ⊂ Pk. Therefore, a sorted list of themarginal value of every content is kept at each iteration andif ∆f∗(Pk, (e, j)) ≥ ∆f∗(Pk−1, (e

′, j)) for all e′ 6= e, thenthere is no need to calculate ∆f∗(Pk, (e

′, j)) for e′ 6= e andthe element e will be added in cache j.

Furthermore, we note that distributed techniques can beapplied for submodular maximization with matroid constraints[40]. These techniques achieve the same approximation guar-antees.

D. The case C = 1

In this section, we study the case where C = 1, i.e.apart from the large cache C0, there is only one cache. Weprove that the QoE problem in this case can be transformedinto an Integer Linear Program (ILP) problem and, therefore,common optimization methods can be applied to find theoptimal solution. This will be particularly useful in the nextsection since it will allow us to compare the performance ofour QoE algorithm with the optimal joint policy.

Let us assume that C = 1. A user u ∈ U retrieves a contenti with rate su1 if this content is cached in the cache, otherwiseit is retrieved by the large cache with rate su0 with su0 < su1.

Then, we introduce the new variable {zui}i∈K,u∈U such thatzui = xiyui. The objective of the QoE problem can be thuswritten as

g(X,Y, Z) =∑u∈U

∑i∈K

[αuN

((su1 − su0) zui + su0yui)

+ (1− αu) pui ((su1 − su0)xi + su0) + βuyuiφ(rui)]. (24)

Therefore, the QoE problem for C = 1 is equivalent to theproblem:

Z Problem.

maximizeX,Y,Z

g(X,Y, Z) (25)

subject to (7), (8),zui = xiyui; (26)xij , yui, zui ∈ {0, 1}. (27)

The equivalence comes from the fact that a pair (X, Y ),where X = {x}i and Y = {y}u,i, is optimal for the QoEproblem if and only if (X, Y , Z) with Z = {z}ui such thatzui = xiyui, for every u ∈ U and i ∈ K, is optimal for the Zproblem.

Notice that, although g(X,Y, Z) is linear in the variablesX,Y and Z, the constraints (26) are nonlinear. Howeverwe will prove that these constraints can be replaced by thefollowing inequalities:

zui ≤ xi, (28)zui ≤ yui, for all u ∈ U , i ∈ K. (29)

Lemma 9. The QoE problem for C = 1 (i.e. one cache andone large cache) is equivalent to the following ILP problem:

QoE ILP problem.

maximizeX,Y,Z

g(X,Y, Z)

subject to (7), (8), (27)− (29).

Proof. It suffices to prove that a solution for the Z problem isalso a solution for the QoE ILP problem and the inverse. Let usassume that (X, Y , Z) is a solution for the Z problem. SinceX and Y are binary variables, the expression zui = xiyuiimplies the inequalities (28) and (29). Hence, (X, Y , Z) isalso a solution for the QoE ILP problem.

Inversely, let us assume that (X, Y , Z) is a solution for theQoE ILP problem. It suffices to prove that zui = xiyui forevery u ∈ U and i ∈ K. For the u and i such that xi = 0or yui = 0, then the inequality constraints imply that zui =0. For the u and i such that xi = 1 and yui = 1, we willnecessarily have that zui = 1 since the coefficient of zui inthe objective function g is strictly positive in a maximizationproblem. Hence, considering that all variables are binary, itfollows that zui = xiyui, and this concludes the proof.

IV. PERFORMANCE EVALUATION

The main objectives of this section are to compare theproposed policy (QoE algorithm) with other policies, to in-vestigate how it outperforms these policies by looking atthe QoS-QoR tradeoffs, and, lastly, to validate its theoreticalapproximation guarantees. To do so, we consider a variety ofscenarios.

A. Scenario 1

As a first step, we compare the performance of our algo-rithm with the optimal one (oracle). For this, we considera scenario with a single cache and the large cache C0 thatcontains the entire catalogue. As shown in Section III-D, theQoE problem for the case C = 1 can be transformed into anILP problem. We use the standard MATLAB solver to solvethis ILP problem.

In our scenario, we consider 25 users connected to thecache and a catalogue of 150 equal-sized contents. The smallsize of the scenario is necessary to be able to calculate theoracle performance. We will consider much larger scenariossubsequently. We assume that the cache can fit 2% of thecatalogue size. Every user u receives one recommendationand its impact is determined by αu. We chose αu to followa uniform distribution between 0.6 and 0.9 (in line with thestatistics gathered on Netflix [4]). We first consider a smallsynthetic scenario where the content popularities pui are thenormalized rui according to equation (1). The values of ruiare chosen randomly, but so that their aggregate values over allusers (i.e. the content popularities) follow a Zipf distribution(with parameter 0.6), in line with the findings of [41] and [42].

In this scenario, we measure the QoS as cache hits (asexplained in Remark 1) and the QoR (Def. 3) by consideringϕ(rui) = log(rui). For a variety of values of βu = β > 0,we queried the oracle and we calculated the QoE given by theproposed algorithm. Table III shows the approximation ratiothat our policy achieves for some β.

TABLE IIIAPPROXIMATION RATIO (OPT/f(X∗, Y ∗))

Parameter β 0.01 0.2 1.2 5 lower bound:Approx. ratio 1 0.9968 0.9923 1 0.5

According to Theorem 1, the ratio (OPT/f(X∗, Y ∗))cannot be lower than 1/2. We observe that, in practice, theachieved ratio is much higher than 1/2, close to 1. In fact,

among all the different values we considered (44 in total) thelowest observed approximation ratio is equal to 0.9923.

Observation 1. Our numerical results validate the theoreticalapproximation guarantees of our policy and also suggest amuch better approximation ratio in practice.

Next, we investigate if this close-to-optimal performance isreflected in the QoS-QoR tradeoffs. At the same time, we willcompare with the tradeoff curve achievable by a state-of-the-art scheme for a similar problem [10]. We describe briefly thisalgorithm.Cache-aware recommendations (CAwR):. CAwR[10] takescaching and recommendation decisions at every cache inde-pendently. It decomposes the problem in two steps: the cachingand recommendation steps. First, given the content preferencedistribution for every user (equivalent to the content popularitydistribution pui or content utilities rui of our model) and theweight every user gives to recommendations (the αu of ourmodel), the aggregate request probability of every content iscalculated. Then, the cache allocation decisions are made bysolving a 0−1 knapsack problem, where the “value” of everycontent is this probability. Then, in the recommendation step,the recommendations are made partially by cached contentsand by non-cached contents that are of high utility for theparticular user. The balance between cached and non-cachedcontents is determined by a so-called distortion parameterrd ∈ [0, 1), which is similar to the β parameter of our model.

Fig. 3. Scenario 1, cache capacity equal to 2% of the catalogue size, N = 1.

Figure 3 depicts the QoS-QoR tradeoffs given by the oracle,our policy and CAwR as points in the plane. We obtainedthese tradeoffs for a variety of values of β and the distortionparameter. We remind the reader that each of these pointscorresponds to a different objective tradeoff, between QoS andQoR, that a content provider might have, i.e., these curvescould also be interpreted as Pareto curves. We normalized thex-axis by considering as minimum QoR the one achieved bythe recommendation decision of policy A (i.e., to recommendthe most popular contents over all users) and as maximumQoR the one achieved by policy C (i.e., to recommend thecontent with the highest utility per user).

Observation 2. The tradeoff curve of our policy dominatesthat of CAwR: We observe that our policy outperforms CAwR

A B C

Fig. 4. A. Scenario 2 (100 users and 6000 equal-sized contents), cache capacity equal to 1% of the catalogue, N = 5. B. Scenario 2 with contents ofvariable sizes, cache capacity equal to 1.5% of the total catalogue, N = 10, C. Scenario 3 (multi-cache): 9 helpers, 100 users and and 6000 equal-sizedcontents, helper’s cache size equal to 1.5%, rates between 0.5 and 15 Mbps, N = 5.

in terms of at least QoS or QoR (or both).

For example, for a similar desired value of QoR of 75%,CaWR achieves 46% QoS and our policy 70%. More impor-tantly, most of the tradeoffs of our policy (e.g., around 90%QoR and and 60% QoS) are not achievable by any tuningof the CAwR algorithm. Points in the extreme right lead tomaximum QoR and both policies recommend the same items,i.e. the ones with the highest utility per user.

Observation 3. Our policy’s tradeoff curve almost coincideswith the optimal. This confirms the findings of Table III.

B. Scenario 2

We proceed with simulating a larger scenario in order tocompare our performance with CAwR. For this, we consider asingle cache with 100 connected users. The catalogue consistsof 6000 equal-sized contents7. The probabilities αu are chosenrandomly in [0.7, 0.9], for all u ∈ U , in line with the statisticsgathered on Netflix [4]. For this scenario, we use a real datasetfor the matrix of utilities rui:MovieLens dataset:. The MovieLens dataset [44] is a collec-tion of 5-star movie ratings collected on MovieLens, an onlinemovie recommendation service. These data have been used forother state-of-the-art works on caching and recommendations(like in [8], [10]). Here, we used a variety of subsets of thetotal 20000263 ratings available in the original dataset. It iscommonly assumed that the utility of a content for a user isthe predicted rating of this user for the content [22]. Therefore,we interpret the rating as the content utility. Since the rangeof ratings is 0.5−5 with 0.5 increments, we map every ratingr to a random number in the interval (r/5 − 0.1, r/5]. Asis common, this matrix is quite sparse. To obtain the missingrui ratings we perform matrix completion through the TFOCSsoftware [45]. TFOCS performs nuclear norm minimization inorder to find the missing entries of a low-rank matrix.

In this scenario, we assume that the cache size is equal to1% of the catalogue size, the contents are of equal size, N = 5

7Note that according to [43], the total number of titles (movies and TVshows) available on Netflix in USA is equal to 5848.

and pui are the normalized rui. As before, we measure the QoSas cache hits and the QoR as the logarithm of rui. In Figure4A, we plot the tradeoffs achieved by CAwR and our policyfor different values of β and CawR’s distortion parameter.

Observation 4. The QoS-QoR tradeoff curve of our policydominates that of CAwR even in larger, more realistic scenar-ios, driven by real datasets.

We notice, for example, that for QoS of value 81%, CAwRalgorithm achieves 16% QoR while our algorithm 42%. Sim-ilar gains are noticeable in terms of QoS for different valuesof QoR. This is an encouraging finding that suggests that thetheoretical gains could also be experienced in practice.

Contents with different sizes: So far, we have consideredscenarios with equal sized content (e.g., chunks), as oftenassumed in related work [13]. Here, we turn our attention toa scenario with different sized contents, analyzed in SectionIII-C2. The sizes of contents are in the interval [1, 4] 8 and weadjust the cache capacity to fit up to 1.5% of the catalogue. ForN = 10 and in the same setup as before, we plot in Figure 4Bthe tradeoffs achieved by the two policies. In this context andaccording to Theorem 2, our policy runs both QoE and s-QoEalgorithms and selects the maximum of them. As expected, thedifference between the tradeoff curves is similar to the one inFig. 4A.

Observation 5. The gains of our proposed algorithm areconsistent for all the different scenarios and parameters con-sidered, including the scenario of variable content sizes.

C. Scenario 3

In the final scenario, we evaluate the performance of theproposed policy in a more generic context. For this, weconsider a cellular network in a square area of 500 m2 with 9small-cell BS (helpers) and a macro-cell base station (the largecache of our scenario). Users are placed in the area accordingto a homogeneous Poisson point process. (in line with the

8Note that the sizes are independent of any other property of the contents.The correlation between size and popularity/access frequency has been inves-tigated in [41] and only a weak correlation has been found.

related works [13], [17], [8]), while helpers are placed in agrid. Helpers’ communication range is set to 200 m whichresults in an average of 3.5 helpers per user.

Without loss of generality, we assume that the streamingrate from the core cache (or macro-cell cache) C0 is 0.5 Mbps,while the suj rates for edge caches are chosen randomly be-tween 2 and 15 Mbps9. In fact, the required Internet connectionspeed to use both YouTube [46] and Netflix [47] is 0.5 Mbpsand the recommended connection speed to watch a video inHD quality is 5 Mbps.

In this scenario, we consider a subset of 6000 contents ofMovielens dataset and αu as in the Scenario 2. Every userreceives N = 5 recommendations. We finally set the helper’scapacity equal to 1.5% of the catalogue size.

We will compare the performance of our policy in terms ofQoE with the following baseline policies.

A-femto and C-femto policies. These policies generalize poli-cies A and C described in Section II-F in a network ofcaches. They make the caching and recommendation decisionsseparately. First, they both cache the most popular contentsbased on the femto-caching policy proposed in [13] thattakes into account the fact that users have access to multiplecaches in the network. Then, they make the recommendationsdecisions based on the policies A and C (recommend thecached items and the ones with the highest utility per userrespectively).

For different values of β > 0, the achieved QoE (QoS+ β∗QoR) of our policy, A-femto and C-femto policy areshown in Fig. 4C. We observe that, for β close to 0 (priorityis given to the QoS), the performance of A-femto and ourpolicy coincide. This is because both policies make the samecaching and recommendation decisions (i.e., cache the mostpopular and recommend these items). The QoE achieved byC-femto policy is lower since caching and recommendationsdecisions where made independently. More precisely, althoughit provides the best QoR, the recommended items are notnecessarily among the cached ones and thus need to beretrieved by the large cache at the cost of lower QoS.

As β increases, the priority moves towards QoR, andhence the performance of A-femto starts to worsen until it isdominated by the one of C-femto policy. Our policy continuesto perform better than both of them as a result of caching andrecommendation decisions made in harmony for the best QoE.Furthermore, the distance between the performance curve ofour policy and C-femto is constant for large β. This is becauseC-femto policy makes the caching decision without accountingfor the impact of recommendations on user requests and thusits QoS is much lower than the one our policy achieves. Infact, this is illustrated in very small scale in our toy examplepresented in Sec. II-F.

Observation 6. The proposed joint policy outperforms base-line policies in terms of achieved QoE as a result of caching

9As we are interested in capturing both wired (CDN) and wireless (femto-caching) the physical layer details are beyond the scope of this analysis.

and recommendations orchestration.

V. RELATED WORK

Hierarchical caching: Optimization of hierarchical caching(e.g., CDNs or ICNs) has been widely explored both in thecontext of wired [18] and wireless networks [13]. Variousaspects of this problem have been explored such as approx-imation algorithms for different variants [48], iterative anddistributed algorithms [49], [50], caching for different videostreaming qualities [17], and economic aspects [51]. See forexample a survey on the recent JSAC issue on caching [52].Nevertheless, these works are oblivious to the impact of therecommender, beyond the simple (usually IRM) popularitymodel used as input.

Caching-recommendation interplay: In an early work inthis direction [53], the authors propose heuristic algorithms forrecommendations in P2P networks that take into considerationboth service cost and user preferences. In [7], the authorspropose a recommendation algorithm that tries to bias requeststowards content in the (single) cache, using a Markov chain tomodel sequential user requests. In a similar spirit, [6] proposesa reordering of the videos appearing in YouTube related videossection by “pushing” on top of the list the cached items. This isshown to increase the cache hit rate in a small-scale experimentthat the authors realized. However, the caching policy is inthese works is fixed.

Considering now various setups with many caches, [54] and[8] introduce the concept of “soft cache hits” that allows theuser to choose an alternative cached content if the initiallyrequested is not locally cached. The concept of satisfying arequest with alternative messages has also proposed in thecontext of “pliable index coding”, targeting physical layerperformance [12]. For example, the authors of [8] focus onrevisiting the optimal caching problem (e.g., in the contextof femto-caching [8]). While these latter caching policiesare recommendation-aware, they don’t explicitly control therecommendations. The recommender in [8]) comes after thecaching decisions, and offers the most relevant alternativecontents in the cache, in case of a cache miss. It thus canbe seen as a simple decomposition algorithm, similar to theaggressive “policy A” of Sec.II-F. A simple decompositionof the joint problem is also proposed in [10] (single cache)and [30] (multi-cache), for a problem setup closer to our work.Targeting cache hit rate maximization, their policy first decideson caching, accounting for the impact of recommendations,and then adjusts the list of recommendations in order to favorcached items. However, no performance guarantees are offeredfor this algorithm. Finally, the authors in [11] formulate a jointproblem in the somewhat different context of prefetching con-tent over a time-varying channel. The formulate the problemas a mixed-integer linear program, but only propose “globaloptimization” methods (e.g., brand and bound) that don’t scalewell with problem size.

Joint optimization theory: Submodularity-based proofs forvarious caching-related problems have flourished since theseminal paper of [13], where the focus is on one set of

variables (caching). Additionally, [10] proves that submod-ularity cannot be straightforwardly extended to both sets ofvariables (caching and recommendation). Joint optimization ina discrete setup has been widely considered in many caching-related and other networking problems. For example, routingor user association variables (subject to capacity constraints)are treated jointly with caching in [48]. It is shown thatthese problems can be mapped to variants of the facilitylocation problem, and related approximation algorithms can beadapted to the problem. However, facility location algorithmsare not applicable to the joint recommendation and cachingproblem at hand. While largely different problems, the iterative(“primal”) decomposition method and submodularity of theouter routine, followed in our work, is similar in spirit to thecombinatorial methods in [55], and [56]. Finally, we note thatiterative methods based on Benders decomposition have alsobeen considered recently for jointly optimizing the economiccosts of caching policies [51]. Alternative decompositionsor methods to find better approximations (e.g., continuousrelaxation) might also be applicable to our model, but arebeyond the scope of this paper.

VI. CONCLUSION

In this paper, we studied the problem of QoE maximizationthrough joint caching and recommendation decisions in anetwork of caches. This is a problem of great interest asentities like Netflix can now manage both caching and recom-mendations in their network. We formulated the problem as amaximization problem where every user’s QoE is a balancedsum of QoS and QoR. The model we considered is genericsince QoS can be replaced by any caching gain/profit. Weapplied the method of primal decomposition and then showedthat the resulting function is submodular in the caching vari-able. Thanks to this property, we propose the QoE algorithmthat has 1

2 -approximation guarantees (or 1−1/e2 ≈ 0.32 in

the case of variable content sizes). Moreover, this algorithmallows us to explore the QoS-QoR tradeoffs of the problem.Our numerical results showed that our algorithm outperformsother proposed schemes in the literature. An interesting di-rection for future work is to investigate if an algorithm withapproximation guarantees can be found when the user-cacheassociations is also variable of the problem.

REFERENCES

[1] T. V. Doan, L. Pajevic, V. Bajpai, and J. Ott, “Tracing the path toyoutube: A quantification of path lengths and latencies toward contentcaches,” IEEE Communications Magazine, vol. 57, no. 1, pp. 80–86,2019.

[2] H. Nam, K.-H. Kim, and H. Schulzrinne, “Qoe matters more than qos:Why people stop watching cat videos,” in IEEE INFOCOM 2016-The35th Annual IEEE International Conference on Computer Communica-tions. IEEE, 2016, pp. 1–9.

[3] P. Sermpezis, S. Kastanakis, J. I. Pinheiro, F. Assis,D. Menasche, and T. Spyropoulos, “Toward qos recommendations,”https://arxiv.org/abs/1907.06392, 2019.

[4] C. A. Gomez-Uribe and N. Hunt, “The netflix recommender system:Algorithms, business value, and innovation,” ACM Transactions onManagement Information Systems (TMIS), vol. 6, no. 4, p. 13, 2016.

[5] R. Zhou, S. Khemmarat, and L. Gao, “The impact of youtube recom-mendation system on video views,” in Proceedings of the 10th ACMSIGCOMM conference on Internet measurement. ACM, 2010, pp. 404–410.

[6] D. K. Krishnappa, M. Zink, C. Griwodz, and P. Halvorsen, “Cache-centric video recommendation: an approach to improve the efficiencyof youtube caches,” ACM Transactions on Multimedia Computing,Communications, and Applications (TOMM), vol. 11, no. 4, p. 48, 2015.

[7] T. Giannakas, P. Sermpezis, and T. Spyropoulos, “Show me the cache:Optimizing cache-friendly recommendations for sequential content ac-cess,” in 19th IEEE International Symposium on ”A World of Wireless,Mobile and Multimedia Networks”, WoWMoM 2018, Chania, Greece,June 12-15, 2018, 2018, pp. 14–22.

[8] P. Sermpezis, T. Giannakas, T. Spyropoulos, and L. Vigneri, “Soft cachehits: Improving performance through recommendation and delivery ofrelated content,” IEEE Journal on Selected Areas in Communications,vol. 36, no. 6, pp. 1300–1313, 2018.

[9] L. E. Chatzieleftheriou, M. Karaliopoulos, and I. Koutsopoulos,“Caching-aware recommendations: Nudging user preferences towardsbetter caching performance,” in IEEE INFOCOM 2017-IEEE Confer-ence on Computer Communications. IEEE, 2017, pp. 1–9.

[10] ——, “Jointly optimizing content caching and recommendations in smallcell networks,” IEEE Trans. Mob. Comput., vol. 18, no. 1, pp. 125–138,2019.

[11] Z. Lin and W. Chen, “Joint pushing and recommendation for susceptibleusers with time-varying connectivity,” in 2018 IEEE Global Communi-cations Conference (GLOBECOM), 2018, pp. 1–6.

[12] L. Song and C. Fragouli, “Making recommendations bandwidth aware,”IEEE Trans. Information Theory, vol. 64, no. 11, pp. 7031–7050, 2018.

[13] K. Shanmugam, N. Golrezaei, A. G. Dimakis, A. F. Molisch, andG. Caire, “Femtocaching: Wireless content delivery through distributedcaching helpers,” IEEE Transactions on Information Theory, vol. 59,no. 12, pp. 8402–8413, 2013.

[14] M. Leconte, G. Paschos, L. Gkatzikis, M. Draief, S. Vassilaras, andS. Chouvardas, “Placing dynamic content in caches with small popu-lation,” in IEEE INFOCOM 2016-The 35th Annual IEEE InternationalConference on Computer Communications. IEEE, 2016, pp. 1–9.

[15] M. Marchese, QoS over heterogeneous networks. John Wiley & Sons,2007.

[16] A. Bentaleb, B. Taani, A. C. Begen, C. Timmerer, and R. Zimmermann,“A survey on bitrate adaptation schemes for streaming media over http,”IEEE Communications Surveys & Tutorials, vol. 21, no. 1, pp. 562–585,2018.

[17] K. Poularakis, G. Iosifidis, A. Argyriou, and L. Tassiulas, “Videodelivery over heterogeneous cellular networks: Optimizing cost andperformance,” in IEEE INFOCOM 2014-IEEE Conference on ComputerCommunications. IEEE, 2014, pp. 1078–1086.

[18] S. Borst, V. Gupta, and A. Walid, “Distributed caching algorithms forcontent distribution networks,” in 2010 Proceedings IEEE INFOCOM,2010, pp. 1–9.

[19] J. Bobadilla, F. Ortega, A. Hernando, and A. Gutierrez, “Recommendersystems survey,” Knowledge-based systems, vol. 46, pp. 109–132, 2013.

[20] G. Adomavicius and Y. Kwon, “Improving aggregate recommendationdiversity using ranking-based techniques,” IEEE Transactions on Knowl-edge and Data Engineering, vol. 24, no. 5, pp. 896–911, 2011.

[21] P. Covington, J. Adams, and E. Sargin, “Deep neural networks foryoutube recommendations,” in Proceedings of the 10th ACM Conferenceon Recommender Systems. ACM, 2016, pp. 191–198.

[22] X. Amatriain, “Building industrial-scale real-world recommender sys-tems,” in Proceedings of the sixth ACM conference on Recommendersystems. ACM, 2012, pp. 7–8.

[23] M. Bressan, S. Leucci, A. Panconesi, P. Raghavan, and E. Terolli, “Thelimits of popularity-based recommendations, and the role of social ties,”in Proceedings of the 22nd ACM SIGKDD International Conferenceon Knowledge Discovery and Data Mining, ser. KDD ’16, 2016, p.745–754.

[24] T. Giannakas, T. Spyropoulos, and P. Sermpezis, “The order of things:Position-aware network-friendly recommendations in long viewing ses-sions,” 2019.

[25] S. S. Krishnan and R. K. Sitaraman, “Video stream quality impactsviewer behavior: inferring causality using quasi-experimental designs,”IEEE/ACM Transactions on Networking, vol. 21, no. 6, pp. 2001–2014,2013.

[26] P. Sermpezis, S. Kastanakis, J. I. Pinheiro, F. Assis, D. Menasche, andT. Spyropoulos, “Towards qos-aware recommendations,” 2019.

[27] M. Seufert, S. Egger, M. Slanina, T. Zinner, T. Hoßfeld, and P. Tran-Gia,“A survey on quality of experience of http adaptive streaming,” IEEECommunications Surveys & Tutorials, vol. 17, no. 1, pp. 469–492, 2014.

[28] Y. Chen, K. Wu, and Q. Zhang, “From qos to qoe: A tutorial on videoquality assessment,” IEEE Communications Surveys & Tutorials, vol. 17,no. 2, pp. 1126–1165, 2014.

[29] T. H. Cormen, C. E. Leiserson, R. L. Rivest, and C. Stein, Introductionto algorithms, 2009.

[30] L. E. Chatzieleftheriou, G. Darzanos, M. Karaliopoulos, and I. Kout-sopoulos, “Joint user association, content caching and recommendationsin wireless edge networks,” SIGMETRICS Performance EvaluationReview, vol. 46, no. 3, pp. 12–17, 2018.

[31] S. Boyd, L. Xiao, A. Mutapcic, and J. Mattingley. (2007)Notes on decomposition methods. Notes for EE364B, StanfordUniversity. [Online]. Available: https://see.stanford.edu/materials/lsocoee364b/08-decomposition notes.pdf

[32] S. Boyd and L. Vandenberghe, Convex optimization. Cambridgeuniversity press, 2004.

[33] G. L. Nemhauser, L. A. Wolsey, and M. L. Fisher, “An analysis ofapproximations for maximizing submodular set functions—i,” Mathe-matical programming, vol. 14, no. 1, pp. 265–294, 1978.

[34] A. Krause and D. Golovin, “Submodular function maximization,”Tractability: Practical Approaches to Hard Problems, vol. 3, no. 19,p. 8, 2012.

[35] G. Calinescu, C. Chekuri, M. Pal, and J. Vondrak, “Maximizing amonotone submodular function subject to a matroid constraint,” SIAMJournal on Computing, vol. 40, no. 6, pp. 1740–1766, 2011.

[36] Y. Filmus and J. Ward, “Monotone submodular maximization over amatroid via non-oblivious local search,” SIAM Journal on Computing,vol. 43, no. 2, pp. 514–542, 2014.

[37] J. Leskovec, A. Krause, C. Guestrin, C. Faloutsos, J. VanBriesen, andN. Glance, “Cost-effective outbreak detection in networks,” in Proc.ACM SIGKDD, 2007, pp. 420–429.

[38] M. Sviridenko, “A note on maximizing a submodular set function subjectto a knapsack constraint,” Operations Research Letters, vol. 32, no. 1,pp. 41–43, 2004.

[39] M. Minoux, “Accelerated greedy algorithms for maximizing submodularset functions,” in Optimization Techniques. Springer Berlin Heidelberg,1978, pp. 234–243.

[40] A. Clark, B. Alomair, L. Bushnell, and R. Poovendran, “Scalable anddistributed submodular maximization with matroid constraints,” in 201513th International Symposium on Modeling and Optimization in Mobile,Ad Hoc, and Wireless Networks (WiOpt). IEEE, 2015, pp. 435–442.

[41] L. Breslau, P. Cao, L. Fan, G. Phillips, S. Shenker et al., “Webcaching and zipf-like distributions: Evidence and implications,” in IEEEINFOCOM, vol. 1, no. 1, 1999, pp. 126–134.

[42] P. Gill, M. Arlitt, Z. Li, and A. Mahanti, “Youtube traffic characteriza-tion: A view from the edge,” in Proceedings of the 7th ACM SIGCOMMConference on Internet Measurement, ser. IMC ’07, 2007, pp. 15–28.

[43] Flixable. (2019) Netflix Museum. [Online]. Available: https://flixable.com/netflix-museum/

[44] F. M. Harper and J. A. Konstan, “The movielens datasets: History andcontext,” ACM Transactions on Interactive Intelligent Systems (TiiS),vol. 5, no. 4, p. 19, 2016.

[45] S. R. Becker, E. J. Candes, and M. C. Grant, “Templates for convex coneproblems with applications to sparse signal recovery,” MathematicalProgramming Computation, vol. 3, no. 3, p. 165, Jul 2011.

[46] YouTube Help. (2019) System Requirements. [Online]. Available:https://support.google.com/youtube/answer/78358?hl=en

[47] Netflix Help Center. (2019) Internet connection speed recommendations.[Online]. Available: https://help.netflix.com/en/node/306

[48] K. Poularakis, G. Iosifidis, and L. Tassiulas, “Approximation algorithmsfor mobile data caching in small cell networks,” IEEE Transactions onCommunications, vol. 62, no. 10, pp. 3665–3677, 2014.

[49] E. Leonardi and G. Neglia, “Implicit coordination of caches in small cellnetworks under unknown popularity profiles,” IEEE Journal on SelectedAreas in Communications, vol. 36, no. 6, pp. 1276–1285, 2018.

[50] K. Avrachenkov, J. Goseling, and B. Serbetci, “A low-complexity ap-proach to distributed cooperative caching with geographic constraints,”Proc. ACM Meas. Anal. Comput. Syst., vol. 1, no. 1, Jun. 2017.

[51] J. Krolikowski, A. Giovanidis, and M. D. Renzo, “A decompositionframework for optimal edge-cache leasing,” IEEE Journal on SelectedAreas in Communications, vol. 36, no. 6, pp. 1345–1359, 2018.

[52] G. S. Paschos, G. Iosifidis, M. Tao, D. Towsley, and G. Caire, “Therole of caching in future communication systems and networks,” IEEEJournal on Selected Areas in Communications, vol. 36, no. 6, pp. 1111–1125, 2018.

[53] D. Munaro, C. Delgado, and D. S. Menasche, “Content recommendationand service costs in swarming systems,” in Proc. IEEE ICC, 2015.

[54] T. Spyropoulos and P. Sermpezis, “Soft cache hits and the impact ofalternative content recommendations on mobile edge caching,” in Proc.ACM Workshop on Challenged Networks (CHANTS), 2016, pp. 51–56.

[55] E. M. Craparo, J. P. How, and E. Modiano, “Throughput optimization inmobile backbone networks,” IEEE Transactions on Mobile Computing,vol. 10, no. 4, pp. 560–572, Apr. 2011.

[56] T. Lukovszki, M. Rost, and S. Schmid, “Approximate and incrementalnetwork function placement,” Journal of Parallel and Distributed Com-puting, vol. 120, pp. 159–169, 2018.

APPENDIX

In order to prove Lemma 7, we need a few intermediateresults. All the results and proofs given here are true forany u ∈ U , so, for simplicity, the index u will be omittedthroughout the appendix.

Lemma A.1. Given P , a placement of cached items, and apair (i, j) ∈ (K × {1, . . . , C}) such that (i, j) 6∈ P , then

s(P ′, i) = max{sj , s(P, i)},

where P ′ = P ∪ (i, j).

Proof. The content i is present in P since it is cached bydefault in the large cache C0. However, it might be cached, inaddition to C0, in other caches different than j. Before adding(i, j), upon request, i will arrive at the user with rate s(P, i),which is the highest rate among all caches where it is cached(as defined in (3)).

After adding (i, j) to the placement, if sj is higher thans(P, i), then i will be retrieved from cache j. In other words,s(P ′, i) = sj . On the other hand, if cache j does not offera better rate for the user than before, the rate associated to iwill stay the same, i.e. s(P ′, i) = s(P, i).

Lemma A.2. Given P , a placement of cached items, anda pair (i, j) ∈ (K × {1, . . . , C}) such that (i, j) 6∈ P , thefollowing statements are true:

a. ∆F ∗(P, (i, j)) = 0 if and only if sj ≤ s(P, i).b. ∆F ∗(P, (i, j)) > 0 if and only if sj > s(P, i).

In the second case, the marginal gain is equal to

∆F ∗(P, (i, j)) = (1− α) pi (sj − s(P, i)) +{0, if Vi(P

′) ≤ V[N ](P ),

Vi(P′)−M, if Vi(P

′) > V[N ](P ),

where P ′ = P ∪ (i, j) and M = max{Vi(P ), V[N ](P )}.

Essentially, Lemma A.2 states that adding (i, j) to theplacement P will lead to positive marginal gain of F ∗ if andonly if the cache j has a higher streaming rate for the userthan all the other caches where i was already cached in P .

Proof. Lemma 5 implies that ∆F ∗(P, (i, j)) > 0 if and onlyif

N∑k=1

(V[k](P

′)− V[k](P ))> 0 (30)

or∑k∈K

(s(P ′, k)− s(P, k)) > 0. (31)

Considering that the only difference between P and P ′ isthe content i cached in cache j, the rates of the contents otherthan i remain the same as the addition of (i, j) does not affectthem. As a result, the inequality in (31) is true if and only ifs(P ′, i) > s(P, i). By Lemma A.1, this means that s(P ′, i) =sj and, therefore, (31) is equivalent to sj > s(P, i). Next,when the inequality (30) holds, then Vi(P ′) > Vi(P ) because,otherwise, V[k](P

′) = V[k](P ) for every k ∈ K. By (13), theinequality Vi(P ′) > Vi(P ) is equivalent to

s(P ′, i) = sj > s(P, i).

Hence, we proved that the inequality in (30) implies sj >s(P, i), which is equivalent to (31). Therefore, we provedthat the statement (b) holds. Since ∆F ∗(P, (i, j)) ≥ 0 (byCorollary 1), it follows that ∆F ∗(P, (i, j)) = 0 if and only ifsj ≤ s(P, i).

Next, we will calculate the marginal gain of F ∗ in the casewhere sj > s(P, i). First, note that, as we mentioned above, inthe expression F ∗(P ′)− F ∗(P ) given by (14), the summandwith coefficient (1− α) pi is equal to∑k∈K

(s(P ′, k)− s(P, k)) = s(P ′, i)− s(P, i) = (sj − s(P, i)).

In order to calculate the other summand, we need to compareVi(P

′) with the V-values of the recommended items in theplacement P before adding (i, j), i.e. the values V[k](P ) fork = 1, . . . , N .

If the V-value of (i, j) is less than the [N ]-th highest V-valuein P , i.e. Vi(P ′) < V[N ](P ), content i will not feature in therecommendations list after adding it in cache j. If Vi(P ′) =V[N ](P ), then content i may make it to the recommendationslist by replacing the [N ]-th item in the list, assuming that tiesare broken arbitrarily in the selection process. In both cases,nothing changes in terms of the [N ] highest V-values in P ′,which means that

N∑k=1

(V[k](P

′)− V[k](P ))

= 0.

On the other hand, if the V-value of (i, j) is greater thanthe [N ]-th highest V-value in P , i.e. Vi(P ′) > V[N ](P ), thismeans that i will definitely feature in the recommendationslist after adding it in cache j. This will lead to the inequalityin (30). We consider the two following subcases:

• i was already among the recommendations in P evenbefore adding it in cache j, i.e. Vi(P ) ≥ V[N ](P ). In thiscase, since the streaming rate is better at j, a part of the

marginal gain will come from the difference in V-valueof i before and after adding (i, j). This means that

N∑k=1

(V[k](P

′)− V[k](P ))

= Vi(P′)− Vi(P ).

• i was not recommended before adding it in cache j, i.e.Vi(P ) < V[N ](P ). Since Vi(P

′) > V[N ](P ) > Vi(P ),content i gets into the recommendations list by taking theplace of the N -th recommendation. Hence, a part of themarginal gain will come from the difference of the newV-value of i and the V-value of the [N ]-th recommendeditem in P , i.e.

N∑k=1

(V[k](P

′)− V[k](P ))

= Vi(P′)− V[N ](P ).

Combining these two subcases, we obtainN∑k=1

(V[k](P

′)− V[k](P ))

= Vi(P′)−max{Vi(P ), V[N ](P )}.

Then, the result follows by replacing the findings above in theexpression F ∗(P ′)− F ∗(P ).

We can now prove Lemma 7.

Proof of Lemma 7. We consider two placements A and Bsuch that A ⊆ B ⊆ P and a pair (i, j) ∈ P\B. As previously,A′ and B′ will denote the sets A ∪ (i, j) and B ∪ (i, j)respectively.

In line with Lemma A.2, we will examine the followingcases:

1) ∆F ∗(A, (i, j)) = 0, and2) ∆F ∗(A, (i, j)) > 0.The first case is equivalent to sj ≤ s(A, i), according to

Lemma A.2. But since A ⊆ B, the fact that F ∗ is monotoneincreasing (Lemma 6) implies that

s(A, i) ≤ s(B, i). (32)

By combining the two inequalities, we obtain sj ≤ s(B, i).We invoke once again Lemma A.2 and we get that

∆F ∗(B, (i, j)) = ∆F ∗(A, (i, j)) = 0.

Concerning the second case, it is equivalent to sj > s(A, i)and ∆F ∗(A, (i, j)) is given by the formula in Lemma A.2.We consider three subcases:• sj ≤ s(B, i);• sj > s(B, i) and Vi(B′) ≤ V[N ](B);• sj > s(B, i) and Vi(B′) > V[N ](B).In the first subcase, Lemma A.2 implies that

∆F ∗(B, (i, j)) = 0 and, therefore,

∆F ∗(A, (i, j)) > ∆F ∗(B, (i, j)) = 0.

Next, sj > s(B, i) is equivalent to ∆F ∗(B, (i, j)) > 0. Sincesj > s(A, i) as well, we have

sj = s(A′, i) = s(B′, i). (33)

We also note that from (32) it follows that

sj − s(A, i) ≥ sj − s(B, i). (34)

If Vi(B′) ≤ V[N ](B), then, by Lemma A.2 and (34), it holdsthat

∆F ∗(B, (i, j)) = (1− α) pi (sj − s(B, i)) (35)≤ (1− α)) pi (sj − s(A, i)) (36)≤ ∆F ∗(A, (i, j)). (37)

If Vi(B′) > V[N ](B), it holds that

∆F ∗(B, (i, j)) = (1− α) pi (sj − s(B, i)) +

Vi(B′)−max{Vi(B), V[N ](B)}.(38)

Furthermore, the inequality (20) in the proof of Lemma 6implies that

V[N ](B) ≥ V[N ](A). (39)

By the equation (33), it follows that

Vi(A′) = Vi(B

′). (40)

Note that this is only true in this subcase. By combining theinequalities above we get that

Vi(A′) = Vi(B

′) > V[N ](B) ≥ V[N ](A).

Consequently, the marginal gain ∆F ∗(A, (i, j)) is given bythe same formula as ∆F ∗(B, (i, j)) in (38), by replacing Bby A.

Gathering together (38) and the inequalities (34) and (39),we only need to prove that

max{Vi(A), V[N ](A)} ≤ max{Vi(B), V[N ](B)} (41)

in order to prove that ∆F ∗(A, (i, j)) ≥ ∆F ∗(B, (i, j)). By(32) we have that Vi(A) ≤ Vi(B), and therefore

Vi(A) ≤ max{Vi(B), V[N ](B)}. (42)

Moreover, by (39) we obtain that

V[N ](A) ≤ max{Vi(B), V[N ](B)}. (43)

Combining (42) and (43), we obtain (41) and this concludesthe proof.