Received July 2, 2020, accepted July 20, 2020, date of publication August 5, 2020, date of current version August 27, 2020.

Digital Object Identifier 10.1109/ACCESS.2020.3014379

Active Content Popularity Learningand Caching Optimization WithHit Ratio GuaranteesSRIKANTH BOMMARAVENI , (Student Member, IEEE),THANG X. VU , (Member, IEEE),SYMEON CHATZINOTAS , (Senior Member, IEEE),AND BJÖRN OTTERSTEN , (Fellow, IEEE)Interdisciplinary Centre for Security, Reliability and Trust (SnT), University of Luxembourg, 1511 Luxembourg City, Luxembourg

Corresponding author: Srikanth Bommaraveni (srikanth.bommaraveni@uni.lu)

This work was supported in part by the National Research Fund, Luxembourg, through the Project AGNOSTICunder Grant 742648, in part by the FNR Bilateral Project LARGOS under Grant 12173206, and in part by theFNR CORE ProCAST under Grant C17/IS/11691338.

ABSTRACT Edge caching is an effective solution to reduce delivery latency and network congestionby bringing contents close to end-users. A deep understanding of content popularity and the principlesunderlying the content request sequence are required to effectively utilize the cache. Most existing worksdesign caching policies based on global content requests with very limited consideration of individualcontent requests which reflect personal preferences. To enable the optimal caching strategy, in this article,we propose an Active learning (AL) approach to learn the content popularities and design an accurate contentrequest prediction model. We model the content requests from user terminals as a demand matrix and thenemploy AL-based query-by-committee (QBC) matrix completion to predict future missing requests. Themain principle of QBC is to query the most informative missing entries of the demand matrix. Based onthe prediction provided by the QBC, we propose an adaptive optimization caching framework to learnpopularities as fast as possible while guaranteeing an operational cache hit ratio requirement. The proposedframework is model-free, thus does not require any statistical knowledge about the underlying trafficdemands. We consider both the fixed and time-varying nature of content popularities. The effectivenessof the proposed learning caching policies over the existing methods is demonstrated in terms of root meansquare error, cache hit ratio, and cache size on a simulated dataset.

INDEX TERMS Edge caching, active learning, matrix completion, content popularity.

I. INTRODUCTIONThe proliferation of numerous cellular devices and theirdemand for data-hungry services result in exponential growthin mobile data traffic. According to Cisco annual report[2], the data traffic is predicted to increase by sevenfoldbetween 2017 and 2022. This explosive growth in data traf-fic challenges the capabilities of current network architec-tures. As the data traffic is primarily dominated by repeatedrequests for a few popular contents [3], a promising solutionto overcome this challenge is to offload network traffic andstore repeatedly requested contents at the network edge [4].Moreover, appropriate caching at the edge results in overallreduced the latency and downloading time. This motivates the

The associate editor coordinating the review of this manuscript andapproving it for publication was Ghufran Ahmed.

need for developing new algorithms to effectively cache thecontents at the edge to avoid the duplicate transmissions.

Typically content caching is divided into two phases: acontent placement and a content delivery. The content place-ment phase depends on the historic requests of the users andit is limited by the cache storage size. On the other hand,the content delivery depends on serving the request of a userupon the arrival of the request at the edge. Some of theworks that are based on joint design of content placementand content delivery are [4]–[10]. All these works focus onimproving the utilization of the cache assuming the users’requests to the contents are known in advance. Therefore,the performance of these methods depends on the accu-racy of users’ content requests (UCR). Although, in prac-tice, the UCR is unknown and needs to be estimated [11].Several works focused on prediction of UCR based

151350 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ VOLUME 8, 2020

S. Bommaraveni et al.: Active Content Popularity Learning and Caching Optimization

on machine learning methods [11]–[16]. However, in areal-world application, the popularity of the contents isnon-stationary, and predicting the UCR is therefore a chal-lenging task. In the edge network, only a small set of requestsare observed in a given window of time. However, to improvethe long term average performance of cache, the number ofrequests to the not-yet requested contents (missing requests)needs to be known. As mentioned earlier, it is difficult tocollect the UCR, hence, machine learning methods have beenadopted to accurately model and estimate UCR probabilityfor missing requests in recommendation systems [11], [17],[18]. For non-stationary UCR, it is important to design aniterative recommendation model to predict future contentpopularity. In this aspect, active learning (AL) is an indispens-able tool to collect the data and steers the learning processtowards achieving the accuracy goal by actively selectingthe most useful data points to query. It is also referred toas query learning or optimal experimental design [19]. Themain idea behind AL in edge caching is that the mobileedge computing (MEC) server interacts with the end usersby posing queries or recommendations. This way the serverobtains knowledge about the end users’ preferences from timeto time and predicts their future content requests. From theobserved and estimated content requests the popularity of thecontents is determined, which then is used as an effectivemeasure for making caching decisions. We see the problemof estimating the content requests as an active learning matrixcompletion problem where the entries of the matrix are UCR.

A. LITERATURE REVIEWDuring recent years, several papers investigated caching atedge network under various objectives such as minimizinglatency, network congestions, maximizing user’s quality ofexperience (QoE), or energy efficiency. The following worksassume that content popularity is known as a priory. In [5],the authors optimize the cache placement phase to minimizethe average latency and total throughput during the deliveryphase. Caching the most popular files in distributed small cellaccess points to minimize the total average delay of all theusers is proposed in [4]. In [6], the authors aim to maximizethe delivery phase’s rate by considering both placementand delivery phases jointly. A distributed caching policy isproposed in [7] for device-to-device systems based on socialawareness and the matching theory. The authors of [17],proposed a cache placement algorithm to reduce networkcongestion in the back-haul link and maximize the QoE.To minimize the download latency over multiple distributedcaches, the authors in [20] proposed an optimization prob-lem as a maximization of a submodular function subject tomatroid constraints for content placement. The performanceof caching under uncoded and coded strategies is analyzedin [21].

Content popularity prediction-based edge-caching hasbeen considered in [11]–[16], [22], [23]. The authors in [11]propose a proactive caching algorithm by leveraging socialnetworks and D2D communications, by assuming the content

requests follow Zipf distribution. To learn the local and globalspace-time popularities of contents a reinforcement learningframework is proposed in [13]. They also assume that thepopularities of contents follow Zipf distribution. A reinforce-ment learning framework for the dynamic content updateusing the Markov decision process is proposed [23] withcache and user requests as state space and content evictionand content retaining as action space. The online contentpopularity is learned based on the context information ofconnected users by modeling a caching policy as the multi-armed-bandit problem [14]. From the perspective of regionalusers, an online content popularity prediction algorithm isproposed based on content features to predict content pop-ularity in [12] using logistic regression. The authors in [15],used content features to improve the prediction accuracy inthe Bayesian framework. They assume that the demands forthe content follow the Poisson distribution and thus incorpo-rate bias in the process. A location-aware content predictionmodel using a linear model, ridge regression is proposedin [16] based on location and content features. A trans-fer learning-based caching mechanism is modeled based onlearning and transferring the hidden latent features whichare extracted from device to device interactions to maximizethe back-haul offloading gains in [22]. The aforementionedworks assume that content popularity does not change overtime or it changes very slowly. However, in practice, thecontent popularity is non-stationary [24] e.g, viral videos.

B. CONTRIBUTIONSIn this article, we consider an active learning framework topredict the UCR in edge caching systems. A novel adaptivecaching framework is proposed to learn the popularities asfast as possible while guaranteeing an operational cache hitratio. Our contributions are as follows:• Firstly, we propose an Active learning method to predict

the UCR in edge caching networks. We formulate thecontent caching as the matrix completion problem ofdemands observed at the small base stations (SBS).

• We then estimate the popularity of the contentsusing active learning based query-by-committee matrixcompletion algorithm. Three passive matrix completionalgorithms, i.e., singular value thresholding, uncon-strained nuclear norm minimization, and matrix fac-torization, are served as the committee members.Compared with collaborative filtering [25], which hasonly one matrix completion algorithm compared toQBC.

• Based on the predicted content popularity, we proposetwo caching policies which maximize the query andguarantee the minimum CHR requirements. The pro-posed caching algorithms are model-free and applica-ble to both static and time-varying content popularitymodels.

• The performance analysis of both caching polices overexistingmethod is established throughMonte Carlo sim-ulations. It is shown that the proposed caching schemes

VOLUME 8, 2020 151351

S. Bommaraveni et al.: Active Content Popularity Learning and Caching Optimization

FIGURE 1. The network model.

perform better than random and most popular cachingschemes.

The rest of the paper is organized as follows. Section IIdescribes the system model. In section III, we propose anactive learning matrix completion algorithm. Section IVpresents the caching and query strategies. Section V presentsthe simulation results and finally, conclusions are drawn inSection VI.Notation: Lower or upper case letters represent scalars,

boldface upper case for matrices, boldface lower case forvectors, [.]a,b represents the element in row a and columnb of a matrix, ‖.‖∗ represents nuclear norm, ‖.‖F representsthe frobenius norm, � is element-wise product or also calledas Hadamard product, ⊗ represents Kronecker product, (.)T

denotes the transpose operator, |.| represents the cardinalityof set and 1 represents a vector of all ones.

II. SYSTEM MODELA. SYSTEM MODELWe consider a heterogeneous cellular network scenario withsmall base stations connected to the mobile core networkover reliable back-haul links, as shown in Fig. 1. Each SBSis equipped with a mobile edge computing server to processthe content requests and finite storage memory to cache thecontents. Denote U = {t1, . . . , tK } as the set of the user ter-minals (UTs) connected to the serving SBS, which has accessto a library F = {f1, . . . , fF } of F contents at the contentserver. Let li be the size of file fi and denote l = [l1, . . . , lF ].Further, the coverage areas of the SBS are assumed to bedisjoint, thus a UT can only be connected to the closest SBSat a time. Each SBS cache can store up to D Gigabits (Gbits).Upon receiving users’ requests, the SBS first checks it’s localstorage. If the requested content is available in the SBS’scache, it can be served immediately. Otherwise, the requestedcontent is fetched from the server before being sent to the UT.

B. DEMAND MATRIXThe SBS keeps track of content demands from its usersvia a demand matrix L ∈ Z+K×F , whose rows andcolumns of L represent anonymous UT profiles and thecontents, respectively. An element [L]k,f ∈ Z+ represents

TABLE 1. Notations and definitions.

the number of requests for content f from UT k . In reality,each UT usually requests only a small number of contents,therefore the demand matrix is sparse and largely rectangular.This is because the number of contents at the content serveris much large compared to the UTs connected to SBS. Dueto the mobility of UTs the size of demand matrix changesover time, but the preferences of UT remain the same sinceeach UT has its profile history. Tomaximize the total requestsserved by the SBS’s cache, the demands of content are esti-mated by predicting themissing entries of the demandmatrix.In the next section, we propose an active learning-basedmatrix completion for the estimation of missing entries of thedemand matrix.

III. ACTIVE LEARNING BASED DEMAND MATRIXCOMPLETIONActive learning(AL) is a sub-field of machine learning andartificial intelligence: which is the study of computer systemsthat improve through experience and training. In short, it is aspecial case of semi-supervised learning in which a learningalgorithm can interactively query the oracle/human annotatorto acquire high-quality data for training [26]. In the contextof matrix completion, AL aims to find the most informativemissing entries of the matrix. In the considered caching prob-lem, each SBS has a demand matrix which represents thenumber of requests for contents from theUTs’. Since eachUTrequests only a small subset of contents of the total contentlibrary, which results in the demand matrix to be sparsewith a lot of missing entries. To estimate the missing entriesof the demand matrix we use AL-based matrix completionwhich is discussed in the III-B. The main advantage in ALis that it has the freedom to choose the data it wants to learnfrom through queries in achieving good performance results.This freedom of choice of data to train helps the learningmethod fast and helps to estimate the real-time requests. Thequeries/recommendations are generated based on informa-tiveness, whichmeans a very informativemissing entry is thatthe matrix completion method has a hard time determiningits actual true value. This is because if we ask the UT whichthen responds with its correct true value, knowing the correcttrue value of the missing entry would improve our matrixcompletion predictions for similar difficult missing entries.On the other hand, if the matrix completion method is veryconfident when predicting a missing entry, then knowing it’sactual true value is not very helpful since the predicted valuewill more than likely be correct.

151352 VOLUME 8, 2020

S. Bommaraveni et al.: Active Content Popularity Learning and Caching Optimization

A. QUERYINGA query is defined as the response received from the userterminal to the system. Queries are selected in serial meansone at a time or either in batches means several to be labeledat once. In the serial setting, the model/learner selects a singlemost informative data point to label from a pool of unlabelleddata points. It then adds the data point to the training setand retrains the model/learner. Due to this, it is not suitablefor many applications, since the process of inducing a modelfrom training data may be slow and expensive [27]. On theother hand in the batch setting, it is more natural to acquirelabels for many different data points at once. It is importantto notice that in the batch querying, the data points should bediverse (to avoid redundancy) as well as be informative to themodel/learner.

B. DEMANDS ESTIMATION BASED ON QBC MATRIXCOMPLETIONTo estimate the demand matrix missing entries we employAL-based Query-by-committee(QBC) [28] matrix comple-tion algorithm. The intuition of the QBC approach is that itmaintains a group of models that are all trained on the sametraining data. Eachmodel in the group then predicts on how tolabel potential input points. The informative missing entriesare selected for querying from the input points for whichthey disagree the most which result in high variance. Thefundamental idea of QBC is that the committee of models aimto minimize the version space (set of models consistent withthe training set) [29], which is the subset of all hypothesisspace1 that are consistent with the known entries. To constrainthe size of version space as small as possible, QBC uses theuncertainty of the predictions for each missing entry. Thechoice of models in the committee can be selected in manyways [26], e.g., using simple sampling [28].

A fundamental challenge in the estimation of the demandmatrix is that only a small subset of requests are observed.This result in the data sparsity and cold start problems [1].AL-based QBC tackles the data sparsity and cold start prob-lems at the root, by identifying the most informative anduseful data that better represents the UT preferences throughqueries [30]. This can be done in various forms, through serialsetting by requesting the UT to assess content-by-contentor through batch setting by requesting the UT to assessseveral contents at a time. Let the total number of passivematrix completion models in the committee be N . The pre-dicted matrix by the n-th committee model is representedby Mn for n ∈ [1, . . . ,N ]. In the following, we have usedthe three low-rank approximation variants of passive matrixcompletion algorithms as the members of the committee.

1) SINGULAR VALUE THRESHOLDINGSingular value thresholding (SVT) [31] is a standard low-rankapproximation matrix completion algorithm. In this method,the algorithm is iterative and produces a sequence of matrices

1set of possible approximations of true function that the algorithm cancreate.

at each step, and performs soft-thresholding on the singularvalues obtained at each step. It is inferred as the convexrelaxation of a rank minimization problem of a matrix. Theminimization problem is defined as,

minimizeMn

‖Mn‖∗

subject to [Mn]k,f = [L]k,f , ∀(k, f ) : [�]k,f = 1.

To compute the singular values and singular vectorsefficiently, we use PROPACK [32]. Unlike Matlab built-infunction for ’svds’, PROPACK uses the iterative Lanczosalgorithm to compute singular values and singular vectorswhich is efficient. It is about ten times faster than Matlabbuilt-in function for ’svds’.

2) UNCONSTRAINED NUCLEAR NORM MINIMIZATIONIn this method, fixed point and Bregman iterative algorithmsare used to solve the nuclear norm minimization of a matrix[33]. The homotopy approach is used together with anapproximate singular value decomposition (SVD) procedure,which results in a very fast, robust, and powerful algo-rithm called fixed-point continuation with approximate SVD(FPCA). The unconstrained nuclear norm minimization of amatrix is given as,

minimizeMn

µ‖Mn‖∗ + ‖�� (L−Mn)‖2F ,

whereµ is the regularization constant to avoid overfitting andis determined by cross-validation. Both the nuclear norm andthe Frobenius norm are convex functions which results in thewhole equation to a convex function. This is solved directlyusing CVX solver [34] in Matlab.

3) MATRIX FACTORIZATIONMatrix factorization [35] is a way of reducing a matrix into itslatent factor space. Theminimization of a regularized squarederror on the set of observed entries is given as,

minimizeX,Y

‖�� (L− XYT )‖2F + λ(‖X‖2F + ‖Y‖

2F ),

where X ∈ RK×r and Y ∈ RF×r are two latent fac-tor matrices such that Mn = XYT , r is the rank, and λis the regularization constant that is determined by cross-validation. We used an alternating least squares method tosolve the minimization problem. Since both the latent factorsare unknowns the minimization problem is not convex. But,the minimization problem becomes quadratic by fixing oneof the unknown and can be solved optimally. When one ofthe unknown is fixed the equation is solved as a least-squaresproblem with other unknown and vice versa, is continueduntil convergence.

C. POPULARITY AND UNCERTAINTY ESTIMATIONThe final estimated demand matrix, denoted by E, iscalculated as the average of the predicted matrices by thecommittee members as,

E =1N

N∑n=1

Mn. (1)

VOLUME 8, 2020 151353

S. Bommaraveni et al.: Active Content Popularity Learning and Caching Optimization

The caching at the SBS is done based on two factors:popularity and uncertainty. The popularity indicates the fre-quency in the demands of the contents across all UTs. Thepopularity of the content f by the committee member n,say pnf , is defined as,

pnf =K∑k=1

[Mn]k,f , n = 1, . . . ,N , f = 1, . . . ,F . (2)

However, each committee member predicts the matrices(i.e. {Mn}

Nn=1) differently, which results in different

popularity values (i.e., {pnf }Nn=1) associated with file f . The

uncertainty of file f , denoted by uf , is defined as therange of the predicted popularity values for a file across allcommittee member estimates associated with file f and ismathematically defined as,

uf = maxn=1,...,N

pnf − minn=1,...,N

pnf . (3)

With the defined notations, the popularity of the file f ,say pf , is defined as the mean of the popularities of estimatesby all the committee i.e.,

pf =1N

N∑n=1

pnf , f = 1, . . . ,F . (4)

Further, the popularity and uncertainty vectors are defined asp = [p1, . . . , pF ] and u = [u1, . . . , uF ] respectively. In thesequel, we jointly optimize the caching and query processes.

IV. CACHE AND QUERY STRATEGYIn this section, we propose two caching policies to maximizethe caching and query performance based on the predictedpopularities.

The storage at the SBS is used to store both the popularand uncertain (most informative contents). This allows thesystem to leverage the trade-off between exploration andexploitation. Here, the exploration is associated with theuncertainty of the contents, and exploitation is associatedwith the popularity of the contents. Therefore, the systemfinds more information about UT’s preferences by exploringthe uncertain contents and then exploits the popular contentto maximize the system performance, e.g., cache hit ratio(CHR). In the following, we propose two caching policies forstoring the contents at the cache.

A. FIXED-MEMORY CACHING POLICYIn this caching policy, the storage at the SBS is divided intotwo parts, one part stores the most popular contents whichis referred to as exploitation of contents, and the secondpart stores the uncertain contents which is referred to asexploration of contents. The number of uncertain contents tobe cached alongside with the popular contents in the cacheis given by query budget Q (< D) Gbits. The selection setand cache placement vector is defined by C and binary vectorx ∈ {0, 1}F×1. As described in the Algorithm 1, the topD−Qcontents of p and the topQ contents of u are selected to cacheplacement x. The stopping criteria of the algorithm is defined

Algorithm 1 Fixed-Memory CachingWith ActiveMatrixCompletion

1: Initialize: L, �,N ,D,Q2: repeat3: [M1,M2, . . . ,MN ] = QBC(L, �,N ),4: calculate p and u as given in III respectively5: Sorting and Indexing: [pvalue,pindex] = sort

(p, descend) [uvalue,uindex] = sort(u, descend)6: Selection set: C C = {Fi ∪ Fi† | i = pindex[1 :D− Q], i† = uindex[1 : Q]}

7: Placement vector: x ∈ {0, 1}F×1

[x]f =

{1, f ∈ C0, otherwise

(5)

8: Query generation: Get the uncertain entries of the Qplaced contents to query.

9: until: stopping criteria

FIGURE 2. System model which iterates between placing files in thecache and receiving feedback of the number of requests.

when the uncertainty vector becomes zeroswhich implies thatall the entries of demand matrix are perfectly known as aresult the placement vector does not change and will have amaximum cache hit ratio.

B. ADAPTIVE CACHING POLICYIn this subsection, we define an adaptive caching policy basedon optimization formulation for cache placement, shownin Fig. 2. Since we want to store both popular and uncertaincontents in the cachewe define two binary vectors which indi-cate 1 if the content is stored in cache and 0 if it is not stored.Let x1 ∈ {0, 1}F×1 and x2 ∈ {0, 1}F×1 be the binary vectorswhich define the cache placement of the contents. x1 includesfiles selected which are popular to achieve exploration whilex2 are uncertain contents to exploitation. We formulate anoptimization problem that maximizes the exploration underguaranteed cache hit ratio as,

P1 : maxx1,x2

(u� lT )x2

s.t. C1 : x1, x2 ∈ {0, 1}F×1,

C2 : x1 + x2 ≤ 1,

C3 : x = x1 + x2,

C4 : xT l ≤ D,

C5 :((p− u)� lT )x1

(p� lT )≥ θ. (6)

151354 VOLUME 8, 2020

S. Bommaraveni et al.: Active Content Popularity Learning and Caching Optimization

Remarks:

• Formally, x1 and x2 are two binary decision variablesfor caching the popular contents and to querying theuncertain contents respectively.

• The objective of the function P1 is responsible formaximizing the exploration of uncertain contents.

• The constraints C2 and C3 describes the files to cache inthe storage.

• C4 is the maximum storage constraint.• And the last constraint C5 denote the guaranteed cachehit ratio greater than or equal to a given CHR targetθ ∈ [0, 1]. Note that for known files, the correspondinguncertainty in u is zero.

The optimization problem P1 regulates the trade-off betweenexploration and exploitation by ensuring the guaranteedcache hit ratio (GCHR) greater than or equal to θ . The queriesare generated based on the binary decision variable x2.We used Mosek solver of CVX [34] to solve the optimizationproblem P1 in Matlab.

Due to the nature of optimization problem P1,whenever the variance of uncertain contents goes to zeroimplies when there is no exploration the storage will not beutilized completely. The size of remaining storage is given as,

Dnew = D− (xT l). (7)

Based on the remaining storage, we formulate an optimiza-tion problem that maximizes the CHR by storing the mostpopular contents in the remaining storage and is referred toas enhanced GCHR. So, when the uncertain vector is zero wefocus on the exploitation of the learned phase by utilizing thefull cache storage. This is mathematically formulated as,

P2 : maxy

(p� lT )y(p� lT )

s.t. C1 : y ∈ {0, 1}F×1,

C2 : x+ y ≤ 1,

C3 : yT l ≤ Dnew. (8)

Remarks:

• Where y is the binary decision variable for caching thepopular contents and x is the placement vector of theoptimization problem P1.

• C2 describes the files to cache.• C3 is the storage constraint.

The optimization problem P2 aims at maximizing thecache hit ratio by storing the most popular contents. The addi-tional contents to be stored after the optimization problemP1to fill the cache completely is given by y. The optimizationproblem P2 is solved using the Mosek solver of CVX [34].Note that, in both the caching policies the MEC server checkand skip the contents, if the contents are already in the cache.The popularity and uncertainty of the contents change overtime, so before making content eviction and replacement thepopularity and uncertainty of contents are calculated.

V. NUMERICAL RESULTSWe evaluate the performance of the proposed cachingpolicies through numerical results. First, we consider thestationary demand model and evaluate the performance ofthe fixed-memory caching policy. Then, we consider thenon-stationary demand model and evaluate the performanceof the adaptive caching policy. For both the scenarios, we con-sider the total number of contents (F) at the server is 100 andthe number of user terminals (K ) connected to SBS at agiven time is 30.2 The UTs connected to SBS changes fromtime due to mobility and as a result, the demand matrix alsochanges over time, for simulations we consider an averageof 30 UTs connected to SBS at any given time. The contentrequests from the UTs for the contents can follow any dis-tribution since our proposed caching polices are model free.However, we consider that the requests follow a Zipf-likedistribution denoted by,

Pk (f ) = ω/f α, (9)

where ω =( F∑f=1

1/f α)−1

and α is the Zipf skewness factor.

A. STATIONARY DEMAND MODELWe analyze the performance of fixed-memory caching policyas metrics of root mean square (RMSE), cache hit ratio(CHR), and back-haul load. For this, we generate a demandmatrix Ltrue with all entries and delete 98% of entries to eval-uate the performance of the active learning query approach infinding themissing entries. with the help of notations defined,the metrics are defined as

RMSE =1

‖Ltrue‖F‖Ltrue − L‖F , (10)

CHR =ptruexT

ptrue1T. (11)

The back-haul load is defined as the number of newcontents fetched from the content server. Note that, beforestoring the contents in the cache, the MEC server will checkand skip the contents if it is already stored. The performanceobtained by random query strategy and collaborative filtering[25] is used as the benchmark for comparison of the resultsand matrix factorization is used as the baseline without querystrategy. In figure 3, we evaluate the performance of thefixed-memory caching policy via the RMSE metric. Theaim is to study the rate of reduction of RMSE When moreand more contents are explored. Initially, the active learn-ing approach performs poorly since 98% of the entries aremissing. As a result, the random caching and collaborativefiltering scheme achieve smaller RMSE until iteration 25 forQ = 2. This effect is explained by the property called inco-herence property [36]. It is seen that after 25 iterations the rateof decrease in RMSE is faster with active learning comparedto random caching and collaborative filtering. This is due tothe fact that using active learning the contents to a query

2We limit the total UTs to nmax

VOLUME 8, 2020 151355

S. Bommaraveni et al.: Active Content Popularity Learning and Caching Optimization

FIGURE 3. RMSE performance comparison, storage size = 30, andskewness factor α = 0.8.

FIGURE 4. Impact of CHR, storage size = 30, and skewness factor α = 0.8.

FIGURE 5. Impact of back-haul load, storage size = 30, and skewnessfactor α = 0.8.

are selected in a way that yields a finite information gain.By increasing the query budget to Q = 5, AL-based cachingoutperforms the random caching policy after iteration 8.

Figure 4 presents the CHR as a function of the queryiterations. Similar to figure 3, the active learning with querybudget is 2 approach has poor performance until iteration

FIGURE 6. Impact of CHR, storage size = 20, and skewness factor α = 0.7.

FIGURE 7. The average over time CHR vs cache capacity.

FIGURE 8. Impact of CHR vs Skewness factor.

25 due to incoherence property. After 25 iterations CHRimproves drastically compared to the random caching policyfor the same reason mentioned in figure 3. However, the per-formance of active learning can be improved by increasing

151356 VOLUME 8, 2020

S. Bommaraveni et al.: Active Content Popularity Learning and Caching Optimization

FIGURE 9. Data distribution at time slot T = 1.

the query budget and is illustrated with the query budgetto 5 in figure 4. By increasing the query budget induceshigher back-haul load as shown in figure 5. The performanceof active learning with query budget 5 can be achievedwithout increasing the back-haul load with query budget2 with the number of iterations. This impact can be seenaround the iteration 45-50 in figure 4. The performance ofthe AL-based caching scheme in terms of the back-haulload is shown in figure 5. Active learning with query bud-get 2 requires lesser back-haul load compared to a randomquerying strategy. However, the active learning method withquery budget 2 imposes a slightly higher back-haul load after45 iterations and converges. This slight increase in back-haulload results in higher CHR as explained in figure 4.Moreover,increasing the query budget results in higher back-haul load,this is observed in figure 5.

B. NON-STATIONARY DEMAND MODELIn this sub-section, we consider the non-stationary onlinedemands for the contents, also we analyze the performanceof the adaptive caching policy. The relation between userterminals and contents at tth time slot aremodelled by demandmatrix represented as Lt ∈ Z+K×F . The entries of thedemand matrix represent the number of times the contentshave been requested by the user terminals. To capture theevolution of content requests over time, we assume that Ltfollows a first-order Markov process:

Lt = Lt−1 + Et ,

Lt = max(Lt , 0) t = 1 . . . T . (12)

where Et ∼ N (0,Q). Q is modeled as a correlation betweenuser terminals and contents given by Q = QF ⊗ QU . Thus,the dependencies are modeled as the Kronecker product ofcovariance matrices QF of contents and QU of user termi-nals. The covariance matrices generated are symmetric andpositive semi-definite, with unit diagonal and other elementsin the closed interval [−1, 1] [37].

We generate the entire demand matrix Ltrue,1 at the initialtime slot T = 1, we then use first-order Markov processto generate a sequence of demand matrices over the timefrom multivariate normal distribution as described above.We model the observed L1 at T = 1, by deleting the 50%of entries randomly. This implies only 50% of entries areobserved at an initial time slot. From the T = 2, the missingentry percentage depends on the contents placed in the cacheand forgetting factor, which is defined as, more importanceis given to recent observations and less importance to earlierdata [38]. In our simulations, we use fixed forgetting factorbased on the bootstrap method.

The effect of the skewness factor can be seen in figure 9.As seen in the figure the contents are uniformly distributedwhen the skewness factor is equal to 0.3 and are more skewedwhen it is equal to 0.7. In figure 6, the performance of theproposed active learning caching is compared with the mostpopular caching and random caching schemes. The cache hitratio is shown as the function of time. The goal of this studyshows that the proposed caching scheme always maintainsthe cache hit ratio above 0.55 which is a guaranteed cache hitratio. While the most popular and random caching schemesperform poorly than the proposed scheme.

In figure 7, we show the average cache hit ratio as afunction of varying cache size for different guarantee cachehit ratio (θ). As seen in the figure, as θ increases the averagecache hit ratio also increases. For greater θ values need highercache size, for example when θ = 0.3 the valid cache sizeis 10 Mbits below this the proposed optimization problemis infeasible. Higher θ value needs a higher cache size. Theimpact of the cache hit ratio as a function of the skewnessfactor is shown in figure 8. The cache hit ratio increase withthe skewness factor. This is because for higher skewnessfactor there are only a few most popular contents and ourproposed learning model effectively finds those contents andcache them for exploitation, as a result, higher skewnessfactor results in a higher cache hit ratio.

VOLUME 8, 2020 151357

S. Bommaraveni et al.: Active Content Popularity Learning and Caching Optimization

VI. CONCLUSIONIn this article, we studied a novel active learning-basedcaching framework at the edge node. By formulating thecontent caching problem as a matrix completion problem,an active learning query by committee approach was usedto predict the missing entries. The most informative missingentries are selected based on uncertainty measure to query.The interactive learning between the system and user ter-minals helps the user terminals become more self-aware oftheir own likes/dislikes while at the same time providing newinformation to the system which helps in better estimation ofthe popularity of contents. Our proposed caching frameworkis model-free, it can be used either for fixed or time-varyingpopularity learning situations. The superiority in the perfor-mance of both the caching policies over the state-of-the-artmethod is established through simulations.

ACKNOWLEDGMENTThis article was presented in part at the IEEE AsilomarConference on Signals, Systems, and Computers,3-6 November 2019, Pacific Grove, CA, USA.

REFERENCES[1] S. Bommaraveni, T. X. Vu, S. Vuppala, S. Chatzinotas, and

B. Ottersten, ‘‘Active content popularity learning via query-by-committeefor edge caching,’’ in Proc. 53rd Asilomar Conf. Signals, Syst., Comput.,Nov. 2019, pp. 301–305.

[2] Cisco, ‘‘Cisco visual networking index: Forecast and trends, 2017–2022,’’White Paper, Feb. 2019.

[3] B. Perabathini, E. Bastug, M. Kountouris, M. Debbah, and A. Conte,‘‘Caching at the edge: A green perspective for 5G networks,’’ inProc. IEEEInt. Conf. Commun. Workshop (ICCW), Jun. 2015, pp. 2830–2835.

[4] K. Shanmugam, N. Golrezaei, A. G. Dimakis, A. F. Molisch, andG. Caire, ‘‘FemtoCaching: Wireless content delivery through distributedcaching helpers,’’ IEEE Trans. Inf. Theory, vol. 59, no. 12, pp. 8402–8413,Dec. 2013.

[5] T. X. Vu, L. Lei, S. Vuppala, A. Kalantari, S. Chatzinotas, and B. Otter-sten, ‘‘Latency minimization for content delivery networks with wirelessedge caching,’’ in Proc. IEEE Int. Conf. Commun. (ICC), May 2018,pp. 1–6.

[6] M. A. Maddah-Ali and U. Niesen, ‘‘Cache-aided interference chan-nels,’’ IEEE Trans. Inf. Theory, vol. 65, no. 3, pp. 1714–1724,Mar. 2019.

[7] J. Li, M. Liu, J. Lu, F. Shu, Y. Zhang, S. Bayat, and D. N. K. Jayakody,‘‘On social-aware content caching for D2D-enabled cellular networks withmatching theory,’’ IEEE Internet Things J., vol. 6, no. 1, pp. 297–310,Feb. 2019.

[8] E. Bastug, M. Bennis, and M. Debbah, ‘‘Cache-enabled small cell net-works: Modeling and tradeoffs,’’ in Proc. 11th Int. Symp. Wireless Com-mun. Syst. (ISWCS), Aug. 2014, pp. 649–653.

[9] K. Poularakis, G. Iosifidis, and L. Tassiulas, ‘‘Approximation algorithmsfor mobile data caching in small cell networks,’’ IEEE Trans. Commun.,vol. 62, no. 10, pp. 3665–3677, Oct. 2014.

[10] M. A. Maddah-Ali and U. Niesen, ‘‘Fundamental limits of caching,’’ IEEETrans. Inf. Theory, vol. 60, no. 5, pp. 2856–2867, May 2014.

[11] E. Bastug, M. Bennis, and M. Debbah, ‘‘Living on the edge: The role ofproactive caching in 5Gwireless networks,’’ IEEE Commun. Mag., vol. 52,no. 8, pp. 82–89, Aug. 2014.

[12] Y. Jiang, M. Ma, M. Bennis, F.-C. Zheng, and X. You, ‘‘User preferencelearning-based edge caching for fog radio access network,’’ IEEE Trans.Commun., vol. 67, no. 2, pp. 1268–1283, Feb. 2019.

[13] A. Sadeghi, F. Sheikholeslami, and G. B. Giannakis, ‘‘Optimal and scal-able caching for 5G using reinforcement learning of space-time popular-ities,’’ IEEE J. Sel. Topics Signal Process., vol. 12, no. 1, pp. 180–190,Feb. 2018.

[14] S. Müller, O. Atan, M. van der Schaar, and A. Klein, ‘‘Context-awareproactive content caching with service differentiation in wireless net-works,’’ IEEE Trans. Wireless Commun., vol. 16, no. 2, pp. 1024–1036,Feb. 2017.

[15] S. Mehrizi, A. Tsakmalis, S. Chatzinotas, and B. Ottersten, ‘‘A feature-based Bayesian method for content popularity prediction in edge-cachingnetworks,’’ in Proc. IEEE Wireless Commun. Netw. Conf. (WCNC),May 2019, pp. 1–6.

[16] P. Yang, N. Zhang, S. Zhang, L. Yu, J. Zhang, and X. Shen, ‘‘Contentpopularity prediction towards location-aware mobile edge caching,’’ IEEETrans. Multimedia, vol. 21, no. 4, pp. 915–929, Apr. 2019.

[17] E. Bastug, J.-L. Guenego, and M. Debbah, ‘‘Proactive small cell net-works,’’ in Proc. ICT, May 2013, pp. 1–5.

[18] L. E. Chatzieleftheriou, M. Karaliopoulos, and I. Koutsopoulos, ‘‘Jointlyoptimizing content caching and recommendations in small cell net-works,’’ IEEE Trans. Mobile Comput., vol. 18, no. 1, pp. 125–138,Jan. 2019.

[19] B. Settles, ‘‘Active learning literature survey,’’ Univ. Wisconsin–Madison,Madison, WI, USA, Comput. Sci. Tech. Rep. 1648, 2009.

[20] N. Golrezaei, K. Shanmugam, A. G. Dimakis, A. F. Molisch, and G. Caire,‘‘FemtoCaching: Wireless video content delivery through distributedcaching helpers,’’ in Proc. IEEE INFOCOM, Mar. 2012, pp. 1107–1115.

[21] T. X. Vu, S. Chatzinotas, and B. Ottersten, ‘‘Edge-caching wireless net-works: Performance analysis and optimization,’’ IEEE Trans. WirelessCommun., vol. 17, no. 4, pp. 2827–2839, Apr. 2018.

[22] E. Bastug, M. Bennis, and M. Debbah, ‘‘A transfer learning approachfor cache-enabled wireless networks,’’ in Proc. 13th Int. Symp. Mod-eling Optim. Mobile, Ad Hoc, Wireless Netw. (WiOpt), May 2015,pp. 161–166.

[23] P. Wu, J. Li, L. Shi, M. Ding, K. Cai, and F. Yang, ‘‘Dynamic contentupdate for wireless edge caching via deep reinforcement learning,’’ IEEECommun. Lett., vol. 23, no. 10, pp. 1773–1777, Oct. 2019.

[24] A. Tatar, M. D. de Amorim, S. Fdida, and P. Antoniadis, ‘‘A survey onpredicting the popularity ofWeb content,’’ J. Internet Services Appl., vol. 5,no. 1, p. 8, Dec. 2014.

[25] E. Bastug, M. Bennis, E. Zeydan, M. A. Kader, I. A. Karatepe, A. S. Er,and M. Debbah, ‘‘Big data meets telcos: A proactive caching perspective,’’J. Commun. Netw., vol. 17, no. 6, pp. 549–557, Dec. 2015.

[26] B. Settles, Active Learning. San Rafael, CA, USA: Morgan & Claypool,2012.

[27] B. Settles, ‘‘From theories to queries: Active learning in practice,’’ inProc. Active Learn. Exp. Design Workshop Conjunct. (AISTATS), vol. 16,May 2011, pp. 1–18.

[28] H. S. Seung, M. Opper, and H. Sompolinsky, ‘‘Query by committee,’’in Proc. 5th Annu. workshop Comput. Learn. Theory (COLT), 1992,pp. 287–294.

[29] T. M. Mitchell, ‘‘Generalization as search,’’ Artif. Intell., vol. 18, no. 2,pp. 203–226, Mar. 1982.

[30] M. Elahi, F. Ricci, and N. Rubens, ‘‘Active learning in collaborativefiltering recommender systems,’’ in E-Commerce and Web Technolo-gies, M. Hepp and Y. Hoffner, Eds. Cham, Switzerland: Springer, 2014,pp. 113–124.

[31] J.-F. Cai, E. J. Candès, and Z. Shen, ‘‘A singular value threshold-ing algorithm for matrix completion,’’ SIAM J. Optim., vol. 20, no. 4,pp. 1956–1982, Jan. 2010.

[32] R. M. Larsen. PROPACK–Software for Large and Sparse SVD Calcula-tions. [Online]. Available: http://sun.stanford.edu/rmunk/PROPACK/

[33] S. Ma, D. Goldfarb, and L. Chen, ‘‘Fixed point and Bregman itera-tive methods for matrix rank minimization,’’ Math. Program., vol. 128,nos. 1–2, pp. 321–353, Jun. 2011.

[34] I. CVX Research. (Aug. 2012). CVX: MATLAB Software forDisciplined Convex Programming, Version 2.0. [Online]. Available:http://cvxr.com/cvx

[35] Y. Koren, R. Bell, and C. Volinsky, ‘‘Matrix factorization techniques forrecommender systems,’’ Computer, vol. 42, no. 8, pp. 30–37, Aug. 2009.

[36] E. J. Candes and B. Recht, ‘‘Exact low-rank matrix completion via convexoptimization,’’ in Proc. 46th Annu. Allerton Conf. Commun., Control,Comput., Sep. 2008, pp. 806–812.

[37] K. Numpacharoen and A. Atsawarungruangkit, ‘‘Generating correlationmatrices based on the boundaries of their coefficients,’’ PLoS ONE, vol. 7,no. 11, pp. 1–7, Nov. 2012, doi: 10.1371/journal.pone.0048902.

[38] T. J. Brailsford, J. H. W. Penm, and R. D. Terrell, ‘‘Selecting the forgettingfactor in subset autoregressivemodelling,’’ J. Time Ser. Anal., vol. 23, no. 6,pp. 629–649, Nov. 2002, doi: 10.1111/1467-9892.00283.

151358 VOLUME 8, 2020

S. Bommaraveni et al.: Active Content Popularity Learning and Caching Optimization

SRIKANTH BOMMARAVENI (Student Member,IEEE) received the bachelor’s degree in electronicsand communication engineering from Jawahar-lal Nehru Technological University Hyderabad,Hyderabad, India, in 2013, and the M.Sc. degreein computer and communications technologyfrom Saarland University, Saarbrücken, Germany,in 2017. He is currently pursuing the Ph.D. degreewith the Interdisciplinary Centre for Security, Reli-ability and Trust (SnT), University of Luxem-

bourg, Luxembourg. His research interests include data analysis, machinelearning, and its applications in the field of telecommunications.

THANG X. VU (Member, IEEE) was born in HaiDuong, Vietnam. He received the B.S. and M.Sc.degrees in electronics and telecommunicationsengineering from the VNU University of Engi-neering and Technology, Vietnam, in 2007 and2009, respectively, and the Ph.D. degree in electri-cal engineering from the University of Paris-Sud,France, in 2014. From 2007 to 2009, he was aResearch Assistant with the Department of Elec-tronics and Telecommunications, VNU University

of Engineering and Technology. From September 2010 to May 2014, he waswith the Laboratory of Signals and Systems, a joint Laboratory of CNRS,CentraleSupélec, and the University of Paris-Sud XI. From July 2014 toJanuary 2016, he was a Postdoctoral Researcher with the Information Sys-tems Technology and Design Pillar, Singapore University of Technology andDesign, Singapore. He is currently a Research Associate with the Interdisci-plinary Centre for Security, Reliability and Trust, University of Luxembourg.His research interests includewireless communications, with particular inter-ests in cache-assisted 5G, cloud radio access networks, resources alloca-tion and optimization, cooperative diversity, channel and network decoding,and iterative decoding. In 2010, he received the Allocation de RechercheFellowship to study the Ph.D. degree in France.

SYMEON CHATZINOTAS (Senior Member, IEEE)received theM.Eng. degree in telecommunicationsfrom the Aristotle University of Thessaloniki,Thessaloniki, Greece, in 2003, and the M.Sc. andPh.D. degrees in electronic engineering from theUniversity of Surrey, Surrey, U.K., in 2006 and2009, respectively. He was involved in numer-ous research and development projects with theNational Center for Scientific Research Demokri-tos, Institute of Informatics Telecommunications,

the Center of Research and Technology Hellas, Institute of Telematics andInformatics, and the Center of Communication Systems Research, Mobile

Communications Research Group, University of Surrey. He is currentlythe Deputy Head of the Interdisciplinary Centre for Security, Reliability,and Trust, SIGCOM Research Group, University of Luxembourg, Luxem-bourg, and a Visiting Professor with the University of Parma, Italy. He hasover 250 publications, 2000 citations, and H-index of 25 according toGoogle Scholar. His research interests include multiuser information theory,cooperative/cognitive communications, and wireless networks optimization.He was a co-recipient of the 2014 Distinguished Contributions to SatelliteCommunications Award and the CROWNCOM 2015 Best Paper Award.He serves on the Satellite and Space Communications Technical Committeeand the IEEE Communications Society.

BJÖRN OTTERSTEN (Fellow, IEEE) was bornin Stockholm, Sweden, in 1961. He received theM.S. degree in electrical engineering and appliedphysics from Linköping University, Linköping,Sweden, in 1986, and the Ph.D. degree in electricalengineering from Stanford University, Stanford,CA, USA, in 1990. He has held research posi-tions at the Department of Electrical Engineering,Linköping University, the Information SystemsLaboratory, Stanford University, the Katholieke

Universiteit Leuven, Leuven, Belgium, and the University of Luxembourg,Luxembourg. From 1996 to 1997, he was the Director of research withArrayComm, Inc., a start-up in San Jose, CA, USA, based on his patentedtechnology. In 1991, he was appointed as a Professor of signal processingwith the Royal Institute of Technology (KTH), Stockholm. From 1992 to2004, he was the Head of the Department for Signals, Sensors, and Systems,KTH. From 2004 to 2008, he was the Dean of the School of Electrical Engi-neering, KTH. He is currently the Director of the Interdisciplinary Centre forSecurity, Reliability and Trust, University of Luxembourg. He is a Fellowof EURASIP. He is a member of the Editorial Board of Signal Processing(EURASIP), the EURASIP Journal on Advances in Signal Processing, andFoundations and Trends in Signal Processing. He was a recipient of theIEEE Signal Processing Society Technical Achievement Award, in 2011,the European Research Council Advanced Research Grant, from 2009 to2013, and the European Research Council Advanced Research Grant, since2017. He has coauthored journal articles that received the IEEE SignalProcessing Society Best Paper Award, in 1993, 2001, 2006, and 2013, andseven IEEE conference Best Paper Awards. He has served on the EditorialBoard of IEEE Signal Processing Magazine. He has served as an AssociateEditor for the IEEE TRANSACTIONS ON SIGNAL PROCESSING.

VOLUME 8, 2020 151359