arX

iv:1

511.

0863

1v1

[cs.

NI]

27

Nov

201

51

Dynamic Clustering and ON/OFF Strategies for WirelessSmall Cell Networks

Sumudu Samarakoon,Student Member, IEEE, Mehdi Bennis,Senior Member, IEEE, Walid Saad,Senior Member,IEEE and Matti Latva-aho,Senior Member, IEEE

Abstract—In this paper, a novel cluster-based approach formaximizing the energy efficiency of wireless small cell networksis proposed. A dynamic mechanism is proposed to group locally-coupled small cell base stations (SBSs) into clusters basedonlocation and traffic load. Within each formed cluster, SBSs coor-dinate their transmission parameters to minimize a cost functionwhich captures the tradeoffs between energy efficiency and flowlevel performance, while satisfying their users’ quality-of-servicerequirements. Due to the lack of inter-cluster communications,clusters compete with one another in order to improve theoverall network’s energy efficiency. This inter-cluster competitionis formulated as a noncooperative game between clusters thatseek to minimize their respective cost functions. To solve thisgame, a distributed learning algorithm is proposed using whichclusters autonomously choose their optimal transmission strate-gies based on local information. It is shown that the proposedalgorithm converges to a stationary mixed-strategy distributionwhich constitutes an epsilon-coarse correlated equilibrium forthe studied game. Simulation results show that the proposedapproach yields significant performance gains reaching up to36% of reduced energy expenditures and up to41% of reducedfractional transfer time compared to conventional approaches.

Index Terms—Small cell networks; energy efficiency; learning;game theory; 5G.

I. I NTRODUCTION

In the past decade, wireless services have evolved fromtraditional voice and text messaging to advanced applica-tions such as video streaming, multimedia file sharing, andsocial networking [1]. Such bandwidth-intensive applicationsincrease the load of existing wireless cellular systems andpotentially lead to an increased energy consumption [2]–[4].The deployment of low-cost and high-capacity small cells overexisting cellular networks has been introduced as a promisingsolution to offload the macro cellular traffic to small cellnetworks [5]. However, a systematic deployment of small cellsmay cause severe inter-cell interference and can increase thenetwork’s overall energy consumption. Thus, it is important toanalyze and address the inter-cell interference in wireless small

This research was supported by the Finnish Funding Agency for Technol-ogy and Innovation (TEKES), Nokia, Anite, and Huawei Technologies, theSHARING project under the Finland grant 128010, and the U.S.NationalScience Foundation (NSF) under Grants CNS-1460333, CNS-1460316, andCNS-1513697.

Sumudu Samarakoon, Mehdi Bennis, and Matti Latva-aho are with Depart-ment of Communications Engineering, University of Oulu, Finland (email:{sumudu,bennis,matti.latva-aho}@ee.oulu.fi).

Walid Saad was a corresponding author, is with Wireless@VT,Bradley De-partment of Electrical and Computer Engineering, VirginiaTech, Blacksburg,VA. He was also International Scholar at Department of Computer Engineer-ing, Kyung Hee University, Seoul, South Korea (email: walids@vt.edu).

The authors would like to thank Dr. Kimmo Hiltunen, Ericsson, Finland,for his valuable comments which helped to improve the quality of this work.

cell networks [6]. In this respect, the development of energy-efficient resource management mechanisms for wireless smallcell networks has become a major research challenge in recentyears [2]–[4], [7]–[10].

Existing literature has studied energy efficiency enhance-ments of small cells under various scenarios. In [2], an optimaldeployment strategy is proposed for a two-tier network basedon power minimization subject to a target spectral efficiency.The work in [3] introduces a stochastic programing approachto minimize energy consumption by optimizing small cell basestation (SBS) locations. In [4], a greedy algorithm for turningbase stations (BSs) on or off is proposed to improve thetradeoff between energy efficiency and traffic delay. A proba-bilistic sleep-wake mechanism is presented in [7] to optimizeenergy efficiency of relays in conventional cellular networks.In [8], a new approach to find the optimal cell size andsleep/wake state is studied focusing on energy consumption. Arandom ON/OFF strategy for data networks is proposed in [9]for energy reduction by analyzing the distributions of delaytolerant and intolerant users. Although these interestingworksenable notable energy reductions in small cell networks, theydo not provide thorough analytical studies on the convergenceand the optimality of the solution obtained from the proposedalgorithms. In addition, some of practical problems of smallcells such as the selfish behavior of BSs, the inefficientBS switching between ON and OFF, the underutilized radiospectrum as well as the potential undesirable outages at BSsare not taken into account [8], [9], [11].

One promising solution to overcome these challenges isto enable the BSs to coordinate their transmission. In thisregard, different types of BS clustering methods [12]–[16]and cluster based coordination mechanisms [17]–[19] havebeen recently proposed. In [17], a clustering-based resourceallocation and interference management scheme is proposedfor femtocells based on exhaustive search. The authors in [18]introduce an inter-cluster interference coordination techniqueto improve the sum rate of a femtocell network. In [19], athreshold based BS sleep algorithm is proposed to improvethe energy efficiency of cellular networks by using a Delaunaytriangulation graph. Although these studies take advantage ofclustering, they are based on heuristics with no convergenceproofs and in which BS clustering is static and does not capturenetwork dynamics such as variation of BS spatio-temporalloads or user arrivals and departures.

The main contribution of this paper is to propose a newcluster-based ON/OFF strategy that allows SBSs to dynami-cally switch ON and OFF, depending on the current, slowlyvarying traffic load, user requirements, and network topology.

2

In small cell networks, performing dynamic approaches forswitching BSs ON and OFF may require the knowledge of theentire network to operate effectively which incurs significantoverhead. Therefore, coordination mechanisms with minimumoverhead are needed to group BSs into clusters within whichBSs can smartly and locally coordinate their transmissions.Unlike previous studies [17]–[19], we investigate not onlylocation-based clustering methods, but we also consider theeffects of BS capabilities to dynamically handle traffic, andfurther compare the performance of centralized and decentral-ized clustering solutions. Moreover, we propose a novel inter-and intra-cluster implicit coordination mechanism for dynami-cally switching ON/OFF BSs while striking a balance betweenthroughput and energy consumption. In the proposed approach,each cluster of BSs aims at minimizing a cost function whichcaptures the individual energy consumption and the flowlevel performance in terms of the fractional time required totransfer their traffic load. Since clusters compete among eachother, we cast the problem as a noncooperative game amongclusters of BSs. To solve this game, we propose a distributedalgorithm using notions from regret learning, in which clustersimplicitly coordinate their transmissions without informationexchange [20], [21]. Furthermore, we prove that the proposedalgorithm converges to a stationary distribution which resultsin an ǫ-coarse correlated equilibrium in the noncooperativegame, and the optimality of the solution is analyzed. Finally,simulation results show that the proposed approach improvesthe energy efficiency and reduces the overall network trafficsignificantly as compared to conventional approaches.

The rest of the paper is organized as follows. The systemmodel is presented in Section II and the problem formulationand cluster-based coordination are provided in Section II-B.The proposed game theoretical approach for the decentralizedBS ON/OFF mechanism and the convergence and optimalityanalysis of the solution are discussed in Section IV. Section Vprovides simulation results and, finally, conclusions are drawnin Section VI.

Notation: The regular symbols represent the scalars whilethe boldface symbols are used for vectors.‖x‖ denotes theEuclidean norm of the vectorx. The sets are denoted by uppercase calligraphic symbols.|X | and∆(X ) represent the cardi-nality and the set of all probability distributions of the finitesetX , respectively. The function1X (x) denotes the indicatorfunction defined as1X (x) = 1 if x ∈ X and 1X (x) = 0if x /∈ X , andx+ = max(x, 0) defines the selection of thelargest non-negative component of(x, 0). Let Eπ

[f(x)

]be

the expected value of the functionf(·) of random vectorxover a probability distributionπ. Furthermore,R, Z+, 1 and0 represent the set of real numbers, the set of non-negativeintegers, vector of all ones, and vector of all zeros, respectively.

II. SYSTEM MODEL AND PROBLEM FORMULATION

A. System Model

Consider the downlink of a wireless small cell networkwhose BSsB = {1, . . . , B} which includes macro BSs(MBSs) and SBSs. We assume that BSs are uniformly dis-tributed over a two-dimensional network layout and we letx

be the vector of two-dimensional coordinates with respect tothe origin. LetLb be the coverage area of BSb such that anygiven user equipment (UE) at a given locationx is served byBS b if x ∈ Lb.

We assume that UEs connected to BSb are heterogeneousin nature such that each UE has a different QoS requirementbased on its individual packet arrival rate. In this respect, letηb(x) andRb(x) be the traffic influx rate and the data rate ofany UE at locationx ∈ Lb which follows an M/M/1 queue.The fraction of time BSb needs to serve the trafficηb(x) fromBS b to locationx is defined as,

b(x) =ηb(x)Rb(x)

. (1)

Consequently, thefractional transfer time of BS b is givenby [22]:

ρb =

∫

x∈Lb

b(x)dx. (2)

Here, the fractional transfer timeρb represents the fractionaltime required for a BS to deliver its requested traffic. More-over, the average number of flows at BSb is given by ρb

1−ρb

and it is proportional to the expected delay at BSb [23, pp.169], [22]. Thus, the parameterρb which indicates the flowlevel behavior in terms of fractional time can be referred toastime load or simply load, hereinafter [4], [11], [22]. For anyBS, a successful transmission implies that the BS delivers thetraffic to its respective UEs, i.e.ρb ∈ (0, 1) for any b ∈ B.Therefore, the effective transmission powerP

Work

b of BS b,when it uses a transmission power ofPb is P

Work

b = ρbPb.A switched-ON BSb consumesP

Base

b power to operate itsradio frequency components and baseband unit in addition tothe effective transmission powerP

Work

b [24]. From an energyefficiency perspective, some BSs might have an incentive toswitch OFF. This allows to reduce the power consumption ofthe baseband units by a fractionqb < 1 by turning OFF theradio frequency components [25], [26]. However, during theOFF state, BSs need to sense the UEs in their vicinity andthus, have non-zero energy consumption ensured byqb > 0.Let Ib be the transmission indicator of BSb such thatIb = 1indicates the ON state whileIb = 0 reflects the OFF state.Thus, the power consumption of BSb can be given by:

PTotal

b =

{

qbPBase

b if Ib = 0,(

1ϑbP

Work

b + PBase

b

)if Ib = 1,

(3)

whereϑb(< 1) captures the efficiency of the power amplifierunit and losses in the main supply and cooling units of BSb [24]. We note that, with this model, we are able to capturenot only the power consumption due to the actual transmission,but also the variations in the power consumption which arerequired to maintain the BS in either ON or OFF modes.

For the downlink transmission, the channels between BSsand UEs are modeled as additive white Gaussian noise(AWGN) channels with noise varianceN0 [27]. We assumethat all BSs transmit on the same frequency spectrum (i.e., co-channel deployment). Thus, the data rate of a UE at locationx ∈ Lb which is served by BSb ∈ B is given by:

Rb(x) = ω log2

(

1 +Pbhb(x)

∑

∀b′∈B\b PWork

b′ Ib′hb′(x) +N0

)

, (4)

3

where ω is the bandwidth andhb′(x) is the channel gainbetween BSb′ and UE atx. We assume that each BS usesorthogonal resource blocks in the time domain to serve itsUEs, and there is no central controller available in the systemto coordinate the transmission power and RB allocation amongBSs.

B. Problem Formulation

Our objective is to improve the energy saving of the networkwhile minimizing the time loads of all the BSs during theirdownlink transmission. The downlink transmission consists ofthree phases: 1) all the UEs determine the BSs to which theywish to connect, 2) the decentralized transmission power andON/OFF state selection of BSs in each cluster is performed,and 3) the data transmission and acquiring the knowledgeon the network based on limited information – cluster-baseddistributed learning – take place.

1) User association: Classical UE association mechanismsare often based on the maximum received signal strength(RSS) or signal to interference and noise ratio (SINR) [28].However, these criteria are oblivious to the time load (and thus,the expected number of flows and delays) which may lead tooverloading BSs yielding lower spectral efficiencies. Thus, asmarter mechanism in which BSs advertise their current loadto all UEs within their coverage area is needed [4], [22].

At time instantt, each BSb advertises its estimated loadρb(t) via a broadcast control message. Considering both thereceived signal strength and load at timet, UE at locationxselects BSb(x, t), wherex ∈ Lb(x,t), as follows:

b(x, t) = argmaxb∈B

{(

1− ρb(t))n

PRx

b (t)}

. (5)

Here,PRx

b (t) = Pb(t)Ib(t)hb(x, t) is the received signal powerat UE in locationx from BSb at timet. The impact of the loadis determined by the coefficientn ≥ 0. The classical referencesignal strength indicator (RSSI)-based UE association is aspecial case of (11) wheren = 0. For n > 0, the BS loadaffects the UE association alongside the received signal power.According to (5), from the UE perspective, each UE selectsa BS assuming it is the most suitable one. However, the BSmight have a preference of switching OFF by offloading itstraffic to one of its neighbor. Thus, the actual serving BS mightbe different from the BS chosen by the UE. Hereinafter, werefer to UE’s choice as theanchor BS.

Since the BSs need to estimate their load beforehand, theestimation must accurately reflect the actual load. Here, weuse the notion of moving time-average load estimationρb(t) =∫

x∈Lbρb(x, t)dx at time t based on history as follows:

ρb(x, t) = ν(t)ρb(x, t− 1) +(

1− ν(t))

ρb(x, t− 1), (6)

whereρb(x, t) is the moving time-average load estimation atlocation x, ν(t) is the learning rate of the load estimation1

and the initial condition as a predefined preferred load, i.e.

1 Choosing the learning rate such thatν(t) ∈ [0, 1] for all t ∈ N andlimt→∞ ν(t) = 0 ensures thatρb(t) yields the time average load of BSbas t → ∞.

ρb(x, 0) = ρPref

. Leveraging different time-scales, we assumethatν(t) is selected such that the load estimation procedure (6)is much slower than the UE association process given in (5).

2) Decentralized transmit power and ON/OFF state se-lection: Once the UE association is done, the BSs need tooptimize their transmissions. The configuration of the entirenetwork is defined by the transmit powers and the ON/OFFstates of all BSs. This configuration is captured by a trans-mission power vectorP = (P1, . . . , P|B|) and an ON/OFFstate indicator vectorI = (I1, . . . , I|B|). Therefore, we usethe tuple(P , I) to represent the network configuration.

For a given network configuration(P , I), BS b consumesP

Total

b amount of energy and handles a loadρb by servinga set of UEs in its coverage areaLb. Based on (2)-(4), itcan be seen that∂ρb

∂Pb< 0. Therefore, from the perspective

of uncoordinated BSs that are oblivious to the choices madeby the rest of the network, each individual BS will concludethat a load reduction is viable only with an increased energyconsumption. Hence, there is a tradeoff between load andenergy consumption reduction. Here, for each BSb ∈ B, wedefine a cost function that captures both energy consumptionand load, as follows:

Υb(P , I) = λP

Total

b(

1ϑbP

Max

b + PBase

b

)

︸ ︷︷ ︸

energy consumption

+ µ ρb

︸︷︷︸

load

, (7)

where the coefficientsλ and µ are weight parameters thatindicate the impact of energy and load on the cost, andP

Max

b

is the maximum transmit power of BSb. The objective is tominimize the total network-wide cost

∑

∀b∈B Υb(P , I).

Whenever a given BS switches OFF, it should maintainservice to its UEs via a re-association process. For a properre-association, BSs need to coordinate with the rest of thenetwork which requires a centralized controller. Such a cen-tralized approach involves large information exchange withinthe network incurring large delays. Endowing individual BSsor sets of well-chosen and locally-coupled BSs with decisionmaking capabilities is crucial. Therefore, clustering is lever-aged to group locally-coupled BSs in terms of mutual inter-ference and load together. Allowing coordination per clusterenables efficient UE load balancing with limited informationexchange. Furthermore, intra-cluster coordination is carriedout for joint load balancing and interference management.

In view of the above, we consider that all BSs are groupedinto sets of judiciously-chosen clustersC = {C1, . . . , C|C|}.Here, a given clusterCi ∈ C consists of a set of BSs whichcan coordinate with one another. The clustering mechanism isdiscussed in Section III in details. For each clusterCi ∈ C,the per-cluster costΥCi

is the aggregated cost of each clustermember,ΥCi

(P , I) =∑

∀b∈CiΥb(P , I). Thus, the minimiza-

tion of the network cost is given by the following optimization

4

problem:

minimizeP ,I,C

∑

∀Ci∈CΥCi

(P , I), (8a)

subject to |Ci| ≥ 1, ∀Ci ∈ C, (8b)

Ci ∩ Cj = ∅, ∀Ci, Cj ∈ C, Ci 6= Cj , (8c)

ρb∈ [0, 1], Pb≤PMax

b , Ib∈{0, 1}, ∀b ∈ B.(8d)

Here, constraints (8b) and (8c) ensure that any BS is partof only one cluster. Solving (8) involves BS clustering, UEassociation, and transmission power optimization.

3) Data transmission and inter-cluster distributed learn-ing: Due to the absence of a central controller, the clustersneed to determine the network configuration(P , I) based ontheir limited knowledge. Using the knowledge of the clustermembers, clusters need to implicitly coordinate and learntheir transmission powers and ON/OFF states, and find thebest network configuration which solves (8). As shown inSection IV, the cost minimization problem can be posed asa noncooperative game that allows to minimize the costs ofeach cluster.

III. C LUSTER FORMATION AND INTRA-CLUSTER

COORDINATION

Solving (8) requires an efficient BS clustering and intra-cluster coordination mechanism. For clustering, we map thesystem into a weighted graphG = (B, E). Here, the setBrepresents the BSs while the setE represents the links betweenlocally-coupled BSs.

A. Similarity Matrix-Based Clustering

The key step in clustering is to identify similarities betweenBSs to group BSs with similar characteristics. This will allowto perform coordination between BSs with little signalingoverhead. While many similarity features can be used, twoimportant ones include the locations of BSs and their loads.The BS locations define the capability of coordination amongnearby BSs while the BS loads reflect mutual interference andthe willingness of cooperation. Next, we examine a numberof techniques to calculate similarities between BSs based onlocation and load.

a) Neighborhood based on BS adjacency and staticGaussian similarity: Consider the graphG = (B, E). The setof edgesE indicates the adjacency between nodes. We use aparameterebb′ to characterize the presence of a link betweennodesb andb′ whenebb′ = 1. Let yb be the coordinates of thevertex (or BS)b ∈ B in the Euclidean space. The neighborhoodof nodeb is defined by the adjacency of the nodes in itsεd-neighborhood:

Nb = {b′| ebb′ = eb′b = 1, ‖yb − yb′‖ ≤ εd}. (9)

The links between the vertices are weighted based on theirsimilarities. Theradial basis function kernel known as theGaussian similarity, is a popular kernel function used forlocation based classifications in machine learning [29]–[31].

Thus, the Gaussian similarity is based on the distance betweenBSs b andb′ calculated as follows:

sdbb′ =

{

exp(−‖yb−yb′‖

2

2σ2d

)if ‖yb − yb′‖ ≤ εd,

0 otherwise,(10)

whereσd controls the impact of neighborhood size2. Further-more, the distance-based similarity matrixSd is formed usingsdbb′ as the(b, b′)-th entry. The rationale behind (10) is that theBSs located far from each others have low similarities, and asthey come closer, similarities increase in which BSs are morelikely to cooperate with each other.

b) Load-based dissimilarity: Unlike the static distance-based clustering in (10), load-based clustering provides amoredynamic manner of grouping neighboring BSs in terms oftime load. Consider BSsb and b′ with the loadsρb and ρb′ .The BSs with similar loads yield no benefit by offloadingtraffic to one another, and thus, there is no advantage forsuch BSs to cooperate whenρb ≈ ρb′ . On the other hand,if ρb ≫ ρb′ , BS b′ has the capability of handling additionaltraffic and thus, is capable to cooperate withb. Based on theseobservations, the weight of the link betweenb and b′ shouldbe (i) positive and(ii) increasing with the load difference.Therefore, a dissimilarity metric is used for the time loadinstead of similarity. Thus, a modified version of (10) is usedto calculate load-based similarity between BSsb and b′ asfollows:

slbb′ = exp

(‖ρb − ρb′‖22σ2

l

)

, (11)

whereσl controls the range of the similarity3. The value ofslbb′ is used as the(b, b′)-th entry of the load-based similaritymatrix Sl.

c) Combining the similarities: One approach for com-biningSd andSl is by forming a linear combination such as(θSd + (1 − θ)Sl

)with 0 ≤ θ ≤ 1. According to (9), any

two BSsb, b′ with ebb′ = 0 will not be able to perform anycoordination, and thus, the similarity between BSsb and b′

needs to be equal to zero. Sinceslbb′ > 0 as per (11), a linearcombination ofsdbb′ andslbb′ with θ ∈ [0, 1) always results ina positive similarity between BSsb and b′ which contradictsthe previous claim. Therefore, the classical linear combinationof Sd andSb cannot be used for clustering.

In order to preserve the characteristics of similarities, thejoint similarity S with sbb′ as the(b, b′)-th element is formu-lated as follows:

sbb′ = (sdbb′ )θ · (slbb′ )(1−θ), (12)

where 0 ≤ θ ≤ 1 controls the impact of the distance andthe load similarities on the joint similarity. Here, cooperationis possible only if a physical link between nodes exists4, i.e.∀θ ∈ [0, 1], ebb′ = 0 =⇒ sbb′ = 0.

2 For a givenεd, the range of the distance-based similarity for any twoconnected BSs is[e−εd/2σ

2d , 1]. The lower bound is determined by the

selection ofσd.3 The range of the load-based similarity is[1, e1/2σ

2l ]. The upper bound

of the dissimilarity depends on the choice ofσl.4 We assume that0θ = 0 is held forθ = 0 as well.

5

Theorem 1: The joint similarity S formulated in (12)simplifies to a Gaussian similarity, and thus, all the propertiesof the Gaussian similarity hold forS.

Proof: Let sbb′ = (sdbb′)θ · (slbb′ )(1−θ) is the (b, b′)-th

element ofS with θ ∈ [0, 1], sdbb′ = exp(−‖yb−yb′‖

2

2σ2d

),

slbb′ = exp(‖ρb−ρb′‖

2

2σ2l

), andyi = (y

(1)i , y

(2)i ) for i = {b, b′}.

Then,

sbb′ = (sdbb′)θ · (slbb′)(1−θ)

= exp

(−(y(1)b − y

(1)b′ )2 − (y

(2)b − y

(2)b′ )2

2(σ2d/θ)

+(ρa − ρb)

2

2(σ2l /1− θ)

)

.

(13)

Define a mappingζi = (ζ(1)i , ζ

(2)i , ζ

(3)i ) for i = {b, b′} with,

ζ(1)i =

y(1)i

√θ

σdσ, ζ

(2)i =

y(2)i

√θ

σdσ, ζ

(3)i =

ıρi√1− θ

σlσ,

(14)

where ı =√−1 and σ is an arbitrary constant. Using the

above mapping,

sbb′ = exp

(−‖ζb − ζb′‖22σ2

)

.

Thus,sbb′ follows the relation of Gaussian similarity and thus,S satisfies the properties of Gaussian similarity matrix.

Note that asS follows the form of a Gaussian similarity,the location and the load of any BS can be mapped intoa 3-dimensional (3D) coordinates. Therefore, any clusteringmethod which uses the joint similarity of distance and loadcan be interpreted as a distance-based clustering in a 3D space.

B. Clustering

Once the similaritiesS of the graphG = (B, E) areformed, BS clustering is performed. There are many clusteringmechanisms available in the literature [12]–[16]. Given theknowledge of the entire similarity matrix, any of these clus-tering methods can be applied. However, the main challengeis the need to obtain the full similarity matrix of the entirenetwork. In essence, clustering methods can be categorizedinto two subgroups: centralized and decentralized clustering.Centralized clustering requires the knowledge of the entiresimilarity matrix, based on which, a central controller performsthe clustering process. Decentralized clustering methodsde-pend on the information gathered from neighbors and decisionmaking is done per node.

With this in mind, we focus on three clustering methods:i)k-mean clustering,ii) spectral clustering, andiii) peer-to-peersearched-based clustering. The main intention is to identifythe performance gains due to the capability of exploitingnodes’ physical locations (spectral clustering overk-meanclustering), and the capability of the decentralization (peer-to-peer search-based clustering over spectral clusteringandk-mean clustering). We assume that each method yields a set ofclustersC = {C1, . . . , C|C|} where clusterCi ∈ C consists ofa set of BSs who cooperate with each other.

1) k-mean clustering [12]: The objective ofk-mean clus-tering is to partition the set of nodes into|C| = k clustersin which each node belongs to the cluster with the nearestmean distance. This is commonly used for nodes distributedin a physical space (an area or a volume). Formally speaking,k-mean clustering is given by,

argminC

∑

∀Ci∈C

∑

∀b∈Ci‖ζb − ζCi

‖, (15)

whereζb represents the coordinates of nodeb ∈ B andζCi=

1|Ci|

∑

∀b∈Ciζb is the center of the clusterCi. As per Theorem

1, using the mapping between the locationyb and the loadρbof BS b into a 3D vectorζb, solving (15) yields BS clustersbased on joint similarity. However, this clustering algorithmrequires the knowledge of all the locations and loads of BSsand thus, it is categorized as a centralized clustering method.

2) Spectral clustering [13]: Unlike k-mean clustering,spectral clustering method exploits the connectivity and thecompactness, thus, the geometry of the node distribution inagraph. The graph Laplacian matrix is formed asL = D − SwhereD is the diagonal matrix with theb-th diagonal elementgiven as

∑|B|b′=1 sbb′ . For a given cluster size|C|, the concate-

nation of the smallest|C| eigenvectors ofL can be used toevaluate the modifiedk-dimensional coordinatesψ1, . . . ,ψ|B|.Then,k-mean clustering is applied onψ1, . . . ,ψ|B| to producethe clustersC.

The number of clustersk is closely related to the eigenval-ues ofL and is given by [13];

k = argmaxi

(|ςi+1 − ςi|

), i = 1, . . . , |B|, (16)

where ςi is the i-th smallest eigenvalue ofL. When nodesare distributed ask groups, the firstk eigenvalues becomesmall while the (k + 1)-th eigenvalue becomes relativelylarge. As the node distribution follows more likely a uniformdistribution, the difference between consecutive eigenvaluesbecomes constant. In such cases, when the area of interest islarge, nodes will be isolated from each other and thus,k willbecome large. If the area is small, all the nodes are close toeach other, and thus, will be grouped into a single cluster,i.e. k = 1. Due to fact that the spectral clustering algorithmrequires full knowledge ofS, it is also a centralized clusteringmechanism.

3) Peer-to-peer search-based clustering: The locationbased decentralized clustering method for a multi-agent net-work is presented in [15]. Here, each agent in the systemsearches other similar agents in a peer-to-peer (P2P) fashion.The knowledge over the neighborhood is sufficient to form theclusters, and the size of any cluster is limited by a predefinedparameter (|C|Max

), i.e. |Ci| ≤ |C|Maxfor anyCi ∈ |C|. Modifying

the parameters of the algorithm in [15] including both locationand load information, we leverage the P2P-search based (P2P-SB) clustering technique for BSs.

C. Coordination

Clustering BSs serves three main purposes:(i) reducingthe signaling overhead required for coordination among BSs

6

compared to a system with a centralized controller,(ii) re-ducing intra-cluster interference among locally coupled BSs,and (iii) efficiently offloading UEs to ON-BSs from BSswhich need to switch OFF. As intra-cluster coordination fulfillsthe above requirements, the number of switched-OFF BSs isexpected to increase compared to the case without clusters andcoordination.

Once clusters are formed, the lightly loaded BS within thecluster is selected as the cluster head5. The function of a clusterhead is to coordinate the transmissions between the clustermembers. Consequently, the entire traffic load of the cluster isdistributed between its members in which orthogonal resourceallocation helps to mitigate intra-cluster interference.As theBSs within a cluster have the ability to coordinate, the entirecluster can be seen as asingle super BS serving all UEs withinits vicinity as illustrated in Fig. 1.

By enabling such re-association of UEs between BSs withina given cluster, BSs can efficiently be switched OFF whileensuring users’ QoS. The actual load of each BS is unknownbefore the downlink transmission takes place. Therefore, UEoffloading within the cluster is carried out with the objectiveof minimizing the estimated load in a given cluster. This caneventually reduce the number of UEs served with low rates.Let MCi

be the set of UEs whose anchor BSs belong to clusterCi. Here, we focus on a discrete coverage area by redefiningLb as the collection of circular regions with negligible radiiaround the UEs associated with BSb. Thus, the integrationforms of ρb and ρb can be modeled as a summation over theset of UEs. LetzCi

bm ∈ {0, 1} be an indicator which defines theconnectivity between UEm ∈ MCi

and BSb ∈ Ci, such thatzCi

bm = 1 if UE m is served by BSb and,zCi

bm = 0 otherwise6.Thus, the scheduling problem within clusterCi ∈ C can bewritten as follows:

minimizezCi

∑

∀b∈Ci

∑

∀m∈MCiρbmzCi

bm (17a)

subject to∑

∀b∈CizCi

bm = 1, ∀m ∈ MCi(17b)

zCi

bm ∈ {0, 1}, ∀m ∈ MCi, ∀b ∈ Ci,(17c)

wherezCi consists of allzCi

bm indicators for all UEsm ∈ MCi

andb ∈ Ci. The constraints ensure that any UE is served by asingle cluster member. This is amixed-integer linear program(MILP) and thus, an NP-hard problem. Nevertheless, relaxingthe integer constraint (17c) yields alinear program which canbe easily solved. Therefore, the relaxed problem is given by:

minimizezCi

∑

∀b∈Ci

∑

∀m∈MCiρbmzCi

bm (18a)

subject to∑

∀b∈CizCi

bm = 1, ∀m ∈ MCi(18b)

0 ≤ zCi

bm ≤ 1, ∀m ∈ MCi, ∀b ∈ Ci.(18c)

where zCi is the relaxed optimization variable ofzCi . Asuboptimal solution for (17),(zCi)⋆, is obtained as follows:

(zCi

bm)⋆ =

{

1 if (zCi

bm)⋆ = argmax∀b′∈Ci

(

(zCi

b′m)⋆)

,

0 otherwise.(19)

5 Partitioning the macrocell area and selecting the BSs located at the centerof each partition and assigning them as cluster heads is another option.

6 Note thatzCibm = 1 ⇔ xm ∈ Lb andzCi

bm = 0 ⇔ xm /∈ Lb. Therefore,the BS loadρb is a linear combination ofzCi

bm,∀m ∈ MCiandb as in (2).

D. Modeling the Overhead Costs Due to Intra-cluster Coor-dination

The signaling overhead due to intra-cluster coordination isa significant factor for a fair comparison between clusteredand non-clustered approaches. However, most of the existingworks [17], [18], [32], [33], and [34] simply ignore the intra-cluster signaling overhead while some studies such as [35] and[36] point out that signaling overhead depends on the clustersize in terms of both the number of members and the areawithin which the cluster is located. However, those works donot provide any concrete model of overhead. Motivated by theabove facts, we model the signaling overhead cost due to intra-cluster coordination as an increment of power consumptionδPBase

b and it is calculated by,

δPBase

b = χ(|Nb| − 1)εd, (20)

where the parameterχ defines the sensitivity of the powerincrement to neighborhood size and range. Clearly, the isolatedBSs (when eitherεd = 0 or |Nb| = 1 with εd > 0)do not suffer from signaling overhead due to intra-clustercoordination. For a clustered BS, the total overhead cost isthesummation of overheads due to the individual communicationswith each member of the cluster. Assuming all the BSs in acluster are treated equally yields an equal individual overheadand, thus, the total overhead will be varyinglinearly withthe size of the neighborhood. As the range of neighborhoodεd increases, there is a higher chance to increase the clustersize and, consequently, the signaling overhead. Sinceεd and|Nb| are coupled with one another, we simply assume that thesignaling overhead due to communication with another genericBS has a linear relation with the range of the neighborhood.Thus, a change of the rangeεd alone has a linear impacton δP

Base

b while the changes of|Nb| due to εd will furtherimpact the signaling overheadδP

Base

b . Moreover, the abovechoice allows us to explicitly include the cost of intra-clustercoordination in our original objective and thus, optimize thetransmission parameters accordingly.

IV. SELF-ORGANIZING INTER-CLUSTER SWITCHING

ON/OFF MECHANISM

Given the formation of clusters, our next goal is to developa self-organizing mechanism for solving (8) in which eachcluster of BSs adjusts its transmission parameters based onlocal information. To do so, we use a regret-based learningapproach [20], [37].

A. Game Formulation

In the proposed approach, clusters need to autonomouslyselect their transmission configurations to minimize theircostfunctions. However, the cell coverage and the achievablethroughput of BSs depend not only on the action of their owncluster, but also on the choices of neighboring clusters dueto interference. In this regard, we formulate a noncooperativegameG =

(C, {ACi

}Ci∈C, {uCi

}Ci∈C

)in which the set of

clustersC is the set of players. Each playerCi ∈ C has aset ACi

={aCi,1, . . . , aCi,|ACi

|

}of actions where an action

7

BS 1 BS 2 BS 3

UE 1 UE 2 UE 3 UE 4

BS 1 BS 2 BS 3

UE 1 UE 2 UE 3 UE 4

or

Uncoordinated base stations Clustered base stations

Data Transmission: option 1 Data Transmission: option 2 Interfering link

#1 #1 #1 #1#1#1

#2 #3 #2 #3 #2 #3#3#2

#4 #4 #4 #4 #4#4#4

BS 1

BS 3

BS 2

time

#1#1

#2 #2 #3 #3

#4

BS 1

BS 3

BS 2

#4

#2 #2#1#1

#3 #3 #4 #4

BS 1

BS 3

BS 2

time

Super BS

Fig. 1: Intra-cell interference mitigation through clustering based coordination. Clustered base stations do not interfere witheach other and users may offloaded.

of cluster Ci, aCi, is composed of the configurations of all

its cluster members, i.e.aCi, (Pb1 , Ib1 , . . . , Pb|Ci|

, Ib|Ci |)

with Ci = {b1, . . . , b|Ci|}. The utility function of clusterCiis uCi

(aCi,a−C) = −ΥCi

(P , I) with uCi: (ACi

,A−Ci) 7→ R

whereaCiis the action of clusterCi anda−Ci

∈ A−Ciare the

actions of the other clusters.

Let πCi(t) =

(πCi,1(t), . . . , πCi,|ACi

|(t))

be a probabilitydistribution using which clusterCi selects a given actionfrom ACi

at time instantt. Here, πCi,j(t) = Pr(aCi

(t) =aCi,j

)is the clusterCi’s mixed strategy where aCi

(t) isthe action of playerCi at time t. When clusterCi playsaction aCi

(t), it observes its utility feedback7 uCi

(t; aCi

(t))

based on which it minimizes its regret associated to ac-tion aCi

(t). Subsequently, playerCi estimates its utilityuCi

(t) =(uCi,1(t), . . . , uCi,|ACi

|(t))

and regretrCi(t) =

(rCi,1(t), . . . , rCi,|ACi

|(t))

for each action assuming it hasplayed the same action during all previous time slots1, . . . , t−1. At each timet, playerCi updates its mixed strategy probabil-ity distributionπCi

in which the actions with higher regrets areexploited while exploring the actions with low regrets. Suchbehavior is captured by the Boltzmann-Gibbs (BG) distributionGCi

= (GCi,1, . . . , GCi,|ACi|) with GCi

: r 7→ ∆(ACi) which

is calculated as follows:

GCi,j

(rCi

(t))=

exp(κr+Ci,j

(t))

∑

∀j′∈ACiexp

(κr+Ci,j′

(t)) , j ∈ ACi

, (21)

where κ > 0 is a temperature parameter which balancesbetween exploration and exploitation. At each timet, all theestimations for any playerCi ∈ C, uCi

(t), rCi(t) andπCi

(t),

7 Since the action played by the clusterCi at time t is already defined asaCi

(t), we useuCi(t) instead ofuCi

(

t; aCi(t)

)

for the sake of notationsimplicity. The system utilityu

(

t;a(t))

=∑

∀Ci∈CuCi

(

t; aCi(t)

)

issimply written asu(t) =

∑

∀Ci∈CuCi

(t).

are updated as follows;

uC,i(t) = uC,i(t− 1)

+τCi(t)1{aC,i=vCi

(t−1)}

(

uCi(t)− uC,i(t− 1)

)

,

rC,i(t) = rC,i(t− 1)

+ιCi(t)(

uC,i(t− 1)− uCi(t− 1)− rC,i(t− 1)

)

,

πC,i(t) = πC,i(t− 1)

+εCi(t)(

GC,i

(rCi

(t− 1))− πC,i(t− 1)

)

.

(22)with the learning rates satisfyinglimt→∞

τ(t)ι(t) =

limt→∞τ(t)ε(t) = 0, and limt→∞

∑tn=1 ξ(n) = +∞ and

limt→∞

∑tn=1 ξ

2(n) < +∞ for all ξ = {τ, ι, ε}. Our choiceof the learning rates follows the format of1/tφ with exponentφ ∈ (0.5, 1).

To solve the game, we propose a novel learning algorithmthat proceeds as follows. At the beginning of each time instantt, the clusters select their actions(aCi

) based on their mixed-strategy probabilities(πCi

), i.e. for all Ci ∈ C,

aCi(t) = f

(πCi

(t− 1)), (23)

wheref : πCi7→ aC,i ∈ ACi

is the mapping from probabilitydistribution to an action. Depending on the actions, all BSsadvertise their loads(ρb) and UEs select their anchor BSs.Based on the estimated loads and actions, UE schedulingwithin clusters takes place. All the clusters of BSs carry out thetransmission based on the actionsa(t) =

(a1(t), . . . , a|C|(t)

)

and calculate the utilitiesuCi(t) for all Ci ∈ C. Each

cluster individually updates its utility and regret estimations(uCi

(t), rCi(t))

along with the mixed strategy probabilities(πCi

(t))

while all the BSs update their load estimationsρ(t).With the updated load estimations, the clusters are updatedfor each time intervalN . The entire operation of the proposedmethod is illustrated in Fig. 2 and the proposed algorithm issummarized in Algorithm 1.

8

Gather distance and

load information

Calculate similarities

using Sd, Sl and (12)

Cluster formation

using a method in

Section III-B

Action selection (22)

Load advertising

and anchor BS

selection: (5)

Per cluster UE

scheduling: (16)-(18)

Calculate ρCi(t),ΥCi

(

ρ(t))

, uCi

(

ρ(t))

Update uCi(t+ 1),

rCi(t+ 1), πCi

(t+ 1)using (21)

Update load

estimations: (6)

ClusteringIntra-cluster

coordination

Inter-cluster

coordination

Initialization

uC(t), rC(t) and πC(t)for t = 0

Is t

Nan

integer?

t → t+ 1

yes

no

Fig. 2: Flow diagram representation of the proposed method.

Next, we invoke the Gibbs-Markov equivalence to show thatthe system following (21)-(23) is aGibbs Field with steady-state distribution.

Theorem 2: Let a(t) =(a1(t), . . . , a|C|(t)

)∈ A be the

collection of actions played by all the clusters based ontheir mixed strategy probabilities, whereA =

∏

Ci∈CACi

.Let π(t) =

(π1(t), . . . , π|A|(t)

)with πj(t) = Pr

(a(t) =

aj

), ∀aj ∈ A, be the action selection probability of the

system. Under Algorithm 1, ast → +∞, π(t) converges to astationary distribution Π = (Πa, ∀a ∈ A) with,

Πa =exp(κΓa)

∑

∀a′∈A exp(κΓa′), (24)

where Γa is the ensemble average of the actiona’s regretestimation ast → +∞.

Proof: Let UCiandU =

∑

∀Ci∈CUCi

be the ensembleaverages of the utilities of clusterCi and the system, respec-tively. As t → ∞, for the clusterCi ∈ C, the ensembleaverages of utility estimation, regret estimation and mixed-strategy probability becomeUCi

, ΓCi= UCi

− UCi1, and

ΠCi,j = GCi,j(ΓCi) for j ∈ ACi

as per (22), respectively.Since

∑

∀Ci∈CΓCi,aCi

=∑

∀Ci∈C(UCi,aCi

−UCi) = Ua−U =

Γa for any actiona ∈ A, Γ = (Γa, ∀a ∈ A) is the ensembleaverage of the regret estimations for the entire system. Duetothe fact that an actiona of the system is composed by the setof actions (aCi,ji , ∀Ci ∈ C) per each cluster, the probabilityof selectinga is

∏

∀Ci∈CΠCi,ji =

∏

∀Ci∈CGCi,ji(ΓCi

) whichcan be simplified toΠa in (24).

Following the notation in [38, Chapter 7], we define aGibbs field with a set of cliquesC, configuration space ofV =

∏

∀Ci∈CVCi

with a finite size, set of finite poten-tials {ΓCi

(vi), ∀vi ∈ VCi}∀Ci∈C

, and probability distribution

Π = (Πv, ∀v ∈ V) where Πv =exp(κΓ(v)

)

∑∀v′∈V exp

(κΓ(v′)

) and

Γ(v) =∑

∀Ci∈CΓCi

(vi) with v = (v1, . . . , v|C|). Suppose the

configurationv(t) at timet changes to the configurationv(t+1) at timet+1 according toΠ(t). Along with the propertieswe have introduced, the evolution of the configurations in theabove Gibbs field is equivalent to a Markov chain [38, Chapter7, Theorem 2.1]. Since the configuration spaceV is finiteat any given timet, the evolution of configurations followsa time-inhomogeneous Markov chain. Note thatΠ(t) ≻ 0

due to the fact that the potentials are finite. Henceforth, allconfigurations have self-loops with positive probability andthus, the Markov chain is aperiodic. Furthermore,Π(t) beinga positive vector implies that Pr

(v(t+1) = v′′|v(t) = v′

)> 0

for any v′, v′′ ∈ V . Thus, it verifies that the process canstart from configurationv′ and ends in configurationv′′ witha positive probability in which the irreducible and positiverecurrence properties are held. Since the time-inhomogeneousMarkov chain is aperiodic, irreducible and positive recurrent,ast → ∞ its transition probabilityΠ converges to a stationarydistribution [38, Chapter 3, Theorem 3.1].

It is shown that the transition probability of the above Gibbsfield is equivalent to the action selection probability providedby Algorithm 1. Since the transition probability convergestoa stationary distribution, we can claim thatΠ given in (24)becomes a stationary distribution as well.

Note that the convergence to a stationary distribution is af-fected by the dynamically changing clusters. Dynamic changesin clustering are based on load estimation (locations are fixed)and mixed strategy probabilities depend on the utility andregret estimations. Therefore, minimizing the fluctuations inestimations helps to achieve the stationary distribution.Thisis achieved byi) defining the cost as a separable function,ii) avoiding fully connected graph with properly selectedεd, σd and σl ensures physically-separated clusters and thus,prevents radical changes to the mixed strategy probabilities,and iii) introducing time-scale separation for clustering andpower optimization. The separable cost function allows thecost calculation to be unchanged through the entire network.

9

Algorithm 1 Dynamic Clustering and BS Switching ON/OFF

1: Input: uC(t), rC(t) andπC(t) for t = 0 and ∀C ∈ C,and ρ(t)

2: while true do3: t → t+ 1

Intra-cluster operations:4: Action selection:aCi

(t) = f(

πCi(t− 1)

)

, (23)

5: Load advertisingρ(t) and anchor BS selection: (5)6: Per cluster UE scheduling: (17)-(19)

Inter-cluster operations:7: Calculations:ρCi

(t), ΥCi

(ρ(t)

), uCi

(ρ(t)

)

8: Update utility and regret estimations, and probability:9: uCi

(t+ 1), rCi(t+ 1), πCi

(t+ 1), (22)10: Update load estimations: (6)11: if t

N ∈ Z+ then12: Update clustersC.13: end if14: end while

Further, we restrict clusters to remain unchanged for a timeinterval ofN(≫ 1) allowing (22) to yield stable estimations.The changes in clusters diminish as the load estimationsbecome stable. Henceforth, the system yields a stationarydistribution and form stable clusters.

B. Optimality of The Solution

Theorem 2 shows that for a fixedκ, the mixed strategyprobabilities converge to the stationary distributionΠ. In thissection we analyze the effect of BG temperature parameterκ on the optimality of the solution at the convergence ofAlgorithm 1. Thus, we useΠ(κ) in order to denote thedependency ofκ on Π. First, we show that asκ approachesinfinity, the solution of Algorithm 1 converges to the solutionof (8).

Theorem 3: Whenκ → ∞, Π(κ)a becomes

limκ→∞

Π(κ)a = Π(∞)

a =

{1

|A⋆| if a ∈ A⋆,

0 if a /∈ A⋆,(25)

whereA⋆ is the set of global optimal solutions to (8).Proof: Leta ∈ A⋆. Therefore,Ua > Ua′ =⇒ Γa > Γa′

for all a′ /∈ A⋆. Thus,

Π(∞)a = lim

κ→∞

exp(κΓa)∑

∀a′∈A exp(κΓa′)

= limκ→∞

1

|A⋆|+∑∀a′ /∈A⋆ exp(κ(Γa′ − Γa)

) =1

|A⋆| .

Theorem 3 states that for any finiteκ, sinceΠa′ ≥ 0 forall a′ /∈ A⋆, a non-optimal action can be selected with non-zero probability. Therefore, for finiteκ, we cannot ensure thatAlgorithm 1 leads to pick optimal actions under the stationarydistribution. However, the optimality of the solution increaseswith κ as given in the following theorem.

Theorem 4: For any optimal actiona ∈ A⋆, the probabil-ity of selecting the optimal solution withΠ(κ)

a , monotonicallyincreases withκ.

Proof: Let us study the first derivative of (24):

∂Π(κ)a

∂κ=

∂

∂κ

exp(κΓa)∑

∀a′∈A exp(κΓa′)= Πa

(

Γa − EΠ(κ)

[Γa

])

,

where Γ = (Γa, ∀a ∈ A) is the regret estimations for thesystem. SinceΓa ≥ Γa′ for anya ∈ A⋆ and for alla′ ∈ A,

Γa > EΠ(κ)

[Γa

]is true and thus,∂Π

(κ)a

∂κ > 0 is held, i.e.the probability of choosing the optimal action monotonicallyincreases withκ.

From Theorem 3 and Theorem 4, we can observe thatthe choice ofκ affects the optimality as the solution ofAlgorithm 1 converges to the stationary distribution. Finally,we prove that selecting largeκ not only increases probabilityof choosing the optimal action, but also increases the expectedutility of the system as given in the theorem below.

Theorem 5: Once the solution of Algorithm 1 convergesto Π

(κ), the expected value of the system utilityEΠ(κ)

[Ua

]

monotonically increases withκ.Proof: Consider the first derivative of the expected re-

grets:

∂EΠ(κ)

[Γa

]

∂κ=

∂

∂κ

(∑

∀a∈A Π(κ)a Γa

)

=∑

∀a∈A

Π(κ)a Γa

(Γa − E

Π(κ)

[Γa

])

= EΠ(κ)

[(Γa)

2]− E

2Π(κ)

[Γa

]

= EΠ(κ)

[(Γa − E

Π(κ)

[Γa

])2].

Here,EΠ(κ)

[(Γa − E

Π(κ)

[Γa

])2]is the variance ofΓ over

Π(κ) and thus, ∂

∂κ

(

EΠ(κ)

[Γa

])

> 0 is held. SinceΓ is

monotonically increasing withU due to linear dependency,∂∂κ

(

EΠ(κ)

[Ua

])

> 0 is held as well.Based on the above theorems, the choice of a largeκ ensures

the close global optimality. However, further increasingκalways improves the result and thus,κ needs to be boundedabove for practical implementations. Following Theorem 5,Corollary 1 states that we can determineκ which satisfies agiven threshold.

Corollary 1: For any given thresholdU < EΠ(∞)

[Ua

],

there exists a finiteκ such that U < EΠ(κ)

[Ua

]<

EΠ(∞)

[Ua

].

This result follows directly from Theorem 5.

C. Convergence to An Equilibrium

The results of the stationary distribution, the optimalityofthe solution, and its dependency on the BG temperatureκare used to study the equilibrium of the game. The stationarydistribution Π defines the sequence of actions which theplayers are going to select. If players have no intention todeviate from such a sequence based on their interest onimproving the utilities, the game is considered to be in anǫ-coarse correlated equilibrium (CCE) [20], [37].

Definition 1: (ǫ-coarse correlated equilibrium): The mixedstrategy probabilityΠ = (Πa, ∀a ∈ A) is anǫ-CCE if, ∀Ci ∈

10

LHS =∑∑

∀α∈AC,∀β∈A−C

UC,(α′,β)Π(∞)(α,β) −

∑∑

∀α∈AC,∀β∈A−C

UC,(α,β)Π(∞)(α,β)

≥ ∑∑

∀α∈AC,∀β∈A−C

UC,(α′,β)Π(κ)(α,β) −

∑∑

∀α∈AC,∀β∈A−C

UC,(α,β)Π(∞)(α,β)

≥ ∑∑

∀α∈AC,∀β∈A−C

UC,(α′,β)Π(κ)(α,β) −

∑∑

∀α∈AC,∀β∈A−C

UC,(α,β)Π(κ)(α,β) − ǫ. (26)

C and∀a′Ci∈ ACi

, where:∑

∀a−Ci∈A−Ci

UCi,(a′Ci

,a−Ci)Π−Ci,a−Ci

− ∑

∀a∈A

UCi,aΠa ≤ ǫ,

for some ǫ > 0 with Π−Ci,a−Ci=∑

∀aCi∈ACi

Π(aCi,a−Ci

)

being the marginal probability distribution with respect to theactionaCi

.Here, the players do not deviate from the mixed-strategy aslong as the increment of the payoff is belowǫ. Finally, theconvergence of the proposed algorithm to anǫ-CCE is givenby the following theorem:

Theorem 6: Algorithm 1 converges to a stationary mixedstrategy probability distributionΠ(κ) which constitutes anǫ-CCE for the gameG =

(C, {ACi

}Ci∈C, {uCi

}Ci∈C

).

Proof: For the notation simplicity any actiona ∈ A isdenoted asa , (α, β) with α ∈ ACi

andβ ∈ A−Cifor any

given Ci ∈ C. Using a simplified notation, the left hand side(LHS) of theǫ-CCE condition can be reformulated as follows:

LHS =∑

∀β∈A−Ci

UCi,(α′,β)

∑

∀α∈ACi

Π(α,β)−∑∑

∀α∈ACi,∀β∈A−Ci

UCi,(α,β)Π(α,β)

=∑∑

∀α∈ACi,∀β∈A−Ci

Π(α,β)

(UCi,(α′,β) − UCi,(α,β)

).

First, consider the scenario whereκ → ∞. Therefore, themixed strategy probability of action(α, β) ∈ A, Π(α,β) =

Π(∞)(α,β) is either0 or 1/|A⋆| according to Theorem 3 and thus,

LHS =∑∑

(α,β)∈A⋆

(UCi,(α′,β) − UCi,(α,β)

)/|A⋆|. Suppose

Π(∞) does not ensure anǫ-CCE withǫ = 0, i.e. LHS> 0. This

yields thatUCi,(α′,β) > UCi,(α′′,β) for some(α′′, β) ∈ A⋆ andα′ ∈ ACi

. Since the regretΓCi,(α′,β) monotonically increaseswith UCi,(α′,β), the result is a higher regret for action(α′, β)over the actions(α′′, β) ∈ A⋆, i.e. ΓCi,(α′,β) > ΓCi,(α′′,β).However, according to Theorem 3, it can occur only when(α′, β) ∈ A⋆ and (α′′, β) /∈ A⋆. This contradicts the formerdefinition of the optimal set of solutions. Therefore, theassumption LHS> 0 with Π

(∞) does not hold, andΠ(∞)

converges to anǫ-CCE with ǫ = 0.Theorem 5 states that the expected utility with any finite

κ is lower than with κ → ∞. Suppose the differencebetween expected utilities for a given finite and infiniteκ values is no greater than a positive valueǫ in which∑

∀(α,β)∈AUC,(α,β)Π(∞)(α,β) −

∑

∀(α,β)∈AUC,(α,β)Π(κ)(α,β) ≤ ǫ.

Furthermore, the maximum expected utility withΠ

(∞) ensures that∑

∀(α,β)∈AUC,(α′,β)Π(κ)(α,β) ≤

∑

∀(α,β)∈AUC,(α,β)Π(∞)(α,β) for any α′ ∈ AC with Π

(κ).Combining above results the LHS is remodeled as (26). Since

LHS≤ 0 is held as discussed before, the result is LHS+ ǫ ≤ ǫand thus,Π(κ) becomes anǫ-CCE as per Definition 1.

D. Practical aspects of the choice of κ

Based on the above discussion regarding the optimality andconvergence, it can be seen that the choice ofκ should belarge to ensure optimality. However, note that the optimalityis only relevant when the algorithm converges to the steadystate and the utilities and regrets are calculated in terms of theirensemble averages. Before achieving convergence, the learningalgorithm needs sufficient time to explore all the actions inorder to make accurate estimations on the ensemble averages.

Consider a largeκ and the initial stept = 1 with anarbitrary actionj0 ∈ ACi

that has been played by the playerCi. Since no other action has been played so far exceptj0, theestimations of the utility and the regret for actionj0 will bedominant, i.e.uCi,j0(1) > uCi,j(1) and rCi,j0(1) > rCi,j(1)for all j ∈ ACi

\ {j0}. According to (21), as a result ofusing a largeκ, exp

(κrCi,j0(1)

)≫ exp

(κrCi,j(1)

)results

in GCi,j0

(rCi

(1))

≈ 1 while GCi,j

(rCi

(1))

≈ 0 for allj ∈ ACi

\ {j0}. Since the learning rateεCi(1) = 1, the

mixed strategy probability fort = 2 becomes deterministicas πCi,j(2) = 1 if j = j0 and πCi,j(2) = 0 if j 6= j0.Thus, the choice of the playerCi’s action remains unchanged,hereinafter. Although this exhibits a fast convergence, thelearning algorithm does not get the opportunity to explorethe remaining actions and make an accurate estimation onthe ensemble averages thus yielding an inaccurate steady statedistribution.

In contrast, the choice of a smallκ will yieldexp

(κrCi,j(1)

)≈ exp

(κrCi,j′ (1)

)and GCi,j

(rCi

(1))

≈1

|ACi| for anyj, j′ ∈ ACi

. Thus, the mixed strategy probabilityremains as a (almost) uniform distribution based on theupdating procedure given in (22). Here, the learning algorithmexplores all the actions indefinitely without exploiting actionswith higher regrets which eventually converges to a stationarydistribution with a low performance. Therefore, from a prac-tical implementation point of view, the choice ofκ needs tobe sufficiently small for the algorithm to explore the actionspace and sufficiently large to improve the optimality of thesolution once it converges to a stationary distribution.

V. SIMULATION RESULTS

For our simulations, we consider a single macrocell un-derlaid with an arbitrary number of SBSs and UEs uniformlydistributed over the area. All the BSs share the entire spectrumand thus, suffer from co-channel interference. We conductmultiple simulations for various practical configurationsand

11

TABLE I: Simulation parameters.

Parameter ValueCarrier frequency, System bandwidth 2 GHz, 10 MHzThermal noise (N0) −174 dBm/HzMean traffic influx rate

(

η(x))

180 kbpsMaximum transmission powers: MBS, SBS 46, 30 dBmEfficiency of power units (ϑ): MBS, SBS [24] 23.55%, 5.42%

Base power consumption (PBase

b ): MBS, SBS 40, 33 dBmFraction of energy saved by switching OFF (qb) 0.5

Minimum distances and path loss models(d in km) [27]MBS – SBS, MBS – UE 75 m, 35mSBS – SBS, SBS – UE 40 m, 10 mMBS – UE path loss 128.1 + 37.6 log10(d)SBS – UE path loss 140.7 + 37.6 log10(d)

ClusteringRange of neighborhood (εd) 250 mImpact of neighborhood width (σd, σl) 300, 1Tradeoff between similarities (θ) 0.5Sensitivity of intra-cluster coordination cost (χ) 4.78 dBm/m

LearningImpact of load in UE association (n) 1Boltzmann temperature (κ) 10Energy and load impacts on cost (λ, µ) 0.5, 0.5learning rate exponents forτ, ι , ε andν 0.6, 0.7, 0.8, 0.9

the presented results are averaged a large number of indepen-dent runs. The parameters used for the simulations are sum-marized in Table I. The proposed cluster-based coordinationand learning based ON/OFF mechanisms are compared withthe conventional network operation referred to hereinafter as“classical approach” in which BSs always transmit. For furthercomparisons, we consider a random BS ON/OFF switchingwith equal probability and finally an uncoordinated learningbased ON/OFF mechanism without forming clusters. Theseare referred to as “random ON/OFF” and “learning withoutclusters”. As we consider three different clustering techniques,k-mean clustering, spectral clustering and P2P-BS clustering,they are referred to as “learning withk-mean clustering”,“learning with spectral clustering”, and “learning with P2P-BS clustering” hereinafter, respectively. For all these threeclustering methods, the clusters remain unchanged for aninterval ofN = 100.

A. Performance of Proposed Mechanisms Based on NetworkCost

Fig. 3 shows the changes in the average cost per BS as thenumber of UEs varies. As the number of UEs increases, the BSenergy consumption and load increase and thus, the averagecost increases. However, Fig. 3 shows that the proposedlearning approaches reduce the average cost by balancingbetween the energy consumption and the load. The reductionof average cost in learning without clusters, learning withk-mean clustering, learning with spectral clustering, and learningwith P2P-BS clustering compared to the classical scenario are60.2%, 60.5%, 59.8%, and60.6%, respectively. Although therandom ON/OFF approach manages to reduce the cost by31.3% compared to the classical scenario, the cost reductionis not as high as compared to the learning approaches. Therandom ON/OFF behavior leads to overloading the switched-ON BSs and thus, the energy saving of switched-OFF BSs isinsignificant due to the increased load due to the traffic and the

10 20 30 40 50 60 700

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

Number of users

Ave

rage

cos

t per

BS

Classical approachRandom ON/OFF approachLearning without clusters

Learning with k−mean clusteringLearning with Spectral clusteringLearning with P2P−SB clustering

Cooperationreduces the cost

Fig. 3: Average cost per BS as a function of the number of UEswith 10 SBSs. The joint similarity is used for the clustering.

energy consumption of overloaded BSs. As the number of UEsincreases, BSs have lesser opportunities to switch-OFF whilesatisfying the UEs’ QoS and thus, the behavior becomes closerto the classical approach.

In Fig. 3, we can see that for small network sizes, thecluster-based approaches consume more energy than the learn-ing approach without clustering. This is due to the fact that,for such networks, only a small number of BSs needs to beswitched-ON and, thus, the cluster-based approaches wouldrequire extra energy ofδP

Base

b for coordination. In contrast, forhighly-loaded networks, as seen in Fig. 3, clustering allowsto better offload traffic and, subsequently, improve the overallenergy efficiency. Moreover, for highly-loaded networks, thespectral clustering approach reduces the cost compared toall three clustering mechanisms. Although P2P-SB cluster-ing is a decentralized clustering method, the cost reductionof learning with P2P-SB is higher than the centralizedk-mean clustering for highly loaded networks, in which P2P-SB clustering exploits connectivity and compactness of nodesin the graphG = (B, E) similar to the centralized spectralclustering method. In this respect, Fig. 3 shows that spectralclustering yields, respectively, up to48%, 46%, 26%, 15%,and12% of cost reduction, relative to the classical approach,random ON/OFF, learning without clusters, learning withk-mean clustering, and learning with P2P-SB clustering, for anetwork with 65 UEs.

B. Reductions of Energy Consumption and Time Load

In Fig. 4a, we show the CDF of the BSs’ energy con-sumption for 10 SBSs and 50 UEs. Fig. 4a illustrates thatthe random ON/OFF approach achieves the highest energyconsumption. This is due to the fact that the switching OFFof some random BSs overloads the switched-ON BSs whichcauses a higher energy consumption. The increased energyconsumption is not compensated by the energy saving ofswitched-OFF BSs and thus, a higher average energy con-sumption per BS can be observed. Moreover, we can seethat for both classical and random ON/OFF approaches, the

12

0 5 10 15 20 250

0.2

0.4

0.6

0.8

1

Average energy consumption per BS [W]

CD

F

Classical approachRandom on/off approachLearning without clustersLearning with k−mean clusteringLearning with Spectral clusteringLearning with P2P−SB clustering

Increasedlow−energy

consumed BSs dueto cooperation

(a) Cumulative density function of BS energy consumption.

0 0.2 0.4 0.6 0.8 10

0.2

0.4

0.6

0.8

1

Average time load per BS

CD

F

Classical approachRandom on/off approachLearning without clustersLearning with k−mean clusteringLearning with Spectral clusteringLearning with P2P−SB clustering

Increased switchedOFF BSs due to

cooperation

(b) Cumulative density function of BS time load.

Fig. 4: Cumulative density functions of BS energy consump-tion and time load for 10 SBSs and 50 UEs. The jointsimilarity is used for the clustering.

fraction of BSs having a high energy consumption is muchhigher than in cases with learning. Indeed, the proposed learn-ing method allows lightly-loaded BSs to offload their trafficand switch OFF, thus yielding significant energy reductions.Coordination between clusters allows more BSs to switch OFFand, thus, as shown in Fig. 4a, the proposed learning ap-proaches with clustering yield larger number of BSs consum-ing less energy. The average energy consumption reductionsof learning without clusters approach is11% compared tothe classical approach. With cluster-based coordination,fork-mean clustering, spectral clustering, and P2P-SB clustering,the average energy consumptions is further reduced by18%,30%, and25%, respectively, compared to the learning withoutclusters.

In Fig. 4b, we show the CDF of the BSs’ load for 10 SBSsand 50 UEs. Here, the random ON/OFF method exhibits alarge number of BSs with low load. Similar to the CDF ofBSs’ energy consumption, Fig. 4b shows that the proposedlearning and the dynamic clustering yield a higher numberof switched-OFF BSs. However, we can see that the averageload of learning without clusters is increased by5% comparedto the classical approach. This increased load is due to the

100 125 150 175 200 225 2503

3.5

4

4.5

5

Range of neighborhood [m]

Ave

rage

cos

t per

BS

Distance−based clustering (θ = 1)

Load−based clustering (θ = 0)

Joint similarity based clustering (θ = 0.5)

(a) Impact of similarities on the average BS cost.

100 150 200 2500

1

2

3

4

5

6

7

8

Range of neighborhood [m]

Clu

ster

siz

e / n

umbe

r of

clu

ster

s

Distance−based clustering (θ = 1)Load−based clustering (θ = 0)Joint similarity based clustering (θ = 0.5)

Average cluster size

Average number of clusters

(b) Impact of similarities on the average number of clustersand the averagecluster size.

Fig. 5: Comparison of the effects of the similarities usedfor clustering with 10 SBSs and 50 UEs. The spectral clus-tering technique is used with the similarities calculated forθ = {0, 0.5, 1}.

selfish behavior of BSs in learning without clusters approach.This is due to the fact that these BSs offload their trafficwithout coordination with the neighboring BSs, and thus, in-efficient offloading causes an increased load on average. Withthe cluster-based coordination, the traffic offloading amongBSs become more efficient. Thus,k-mean clustering, spectralclustering and P2P-BS clustering methods yield, respectively,34%, 48%, and55% of average load reductions compared tothe classical approach. Furthermore, Fig. 4b illustrates thatthe proposed ON/OFF mechanism allows about45% of BSsin learning without clusters approach to switch OFF. Withthe cluster-based coordination, traffic offloading becomesmoreefficient and thus, the number of switched-OFF BSs are furtherincreased by12% − 20% compared to the scenario withoutclusters.

C. Impact of Similarity and Range of Neighborhood

Fig. 5a shows the effect on the network cost as a functionof the similarity. The range of neighborhood (εd) defines thecapability of having a physical link for the communication

13

between BSs. Whenθ = 1, the similarity only depends on thedistance between BSs. Thus, clustering becomes static anddoes not consider the BSs’ loads. Asεd increases, similaritiesbetween BSs located far from each other increases from zeroto a positive quantity, and thus, larger clusters are formedwith the distance-based similarity. Since these BSs may notbeable to share the traffic with one another, these large clustersbecome inefficient and result in increased network cost. Forθ = 1/2, as the knowledge of the load is combined withthe distance similarity, clusters are formed with more locally-coupled BSs. Therefore, the joint clustering results in betterperformance with increasing neighborhood width. However,communication within a larger neighborhood requires moreresources (cable costs and higher transmission losses for wiredbackhaul networks while higher transmission power and largerspectrum for over-the-air backhaul networks). Therefore,fur-ther expanding the neighborhood increases the cost of the BSswhich can be seen forεd ≤ 200 m. Clustering with only load-based similarity,θ = 0, displays similar behavior as the jointsimilarity. The main reason is that the load-based similarityuses (9) to determine the existence of edges. Therefore,the limitation based on the distance implicitly appears inthe load-based similarity as well. Thus, the use ofθ = 0produces clusters with BSs which have sufficient capacity tosupport each others resulting in reduced cost compared to theclustering based on distance similarity only.

Fig. 5b shows the average number of clusters and theaverage cluster sizes for the graphs with distance-based, load-based and joint similarities. As the range of neighborhood(εd) increases, the non-zero weighted edges inG = (B, E)increases. Therefore, clustering based on distance similarityallows to group more BSs together thus, yielding a smallernumber of clusters with large cluster size. As the cluster sizeincreases, the traffic in a cluster increases resulting increasedloads in cluster members which directly impact the networkcost. As shown in Fig. 5b, forθ < 1, up to εd < 175 m, thenetwork cost is reduced by forming large clusters. However,for εd > 175 m, increased traffic in a cluster affects the load-based similarity and thus, forming large clusters is avoided.

D. Impact of Sensitivity χ on Clustering Cost

Fig. 6 shows the average cost per BS for learning withoutclusters and learning with spectral clustering for differentchoices of the sensitivity parameterχ in (20). The sensitivityparameterχ models how sensitive the clusters’ power con-sumption is with respect to the overhead of their intra-clustercoordination. The choices of these parameters are such thati) χ = 0 will lead to an overhead free (unrealistic) scenario,ii) χ = 3 × 10−3 implies δP

Base

b ≈ 12P

Base

b , iii) χ = 6 × 10−3

implies δPBase

b ≈ PBase

b , for εd = 250 m and the average clustersize at the above neighborhood range. Since learning withoutclusters has no effect on this sensitivity due to the absenceof clusters and intra-cluster coordination, the average costremains unchanged for different choices ofχ. For learningwith spectral clustering, asχ increases, clusters need higherpower consumption for their intra-cluster coordination andthus, the average cost per BS is increased. According to (20),

0 1.5 3 4.5 6

x 10−3

0.2

0.25

0.3

0.35

0.4

0.45

0.5

Sensitivity χ [Wm−1]

Ave

rage

cos

t per

BS

Learning without clusters, 30 UEsLearning with Spectral clustering, 30 UEsLearning without clusters, 60 UEsLearning with Spectral clustering, 60 UEs

Fig. 6: Comparison of the average cost per BS for learningwith and without clusters for different choices of the sensitivity(χ) parameter. The setup is for 10 SBSs andεd = 250 m.

intra-cluster coordination cost is independent of the numberof UEs, and Fig. 6 shows that the changes inχ have thesame effects on both systems with 30 UEs and 60 UEs. It canbe noted that the benefit of clustering to reduce the averagecost increases with the number of UEs compared to learningwithout clustering method, as explained under Fig. 3.

VI. CONCLUSIONS

In this paper, we have proposed a dynamic cluster-basedON/OFF mechanism for small cell base stations. Clusteringallows clustered base stations to coordinate their transmissionwhile the clusters compete with one another to reduce a per-cluster cost based on their energy consumption and time loaddue to their traffic. In this regard we have formulated theper-cluster cost minimization problem as a noncooperativegame among clusters. To solve the game, we have proposed adistributed algorithm and an intra-cluster coordination methodusing which base stations choose their transmission modeswith minimum overhead. Our proposed clustering method usesinformation on both the locations of BSs and their capabilityof handling the traffic and dynamically form the clustersin order to improve the overall performance. For clustering,both centralized and decentralized clustering techniquesareintroduced and the performances are compared. Simulationresults have shown significant gains in energy expenditureand load reduction compared to the conventional transmissiontechniques. Moreover, the results provide an insight of whenand how to reap the benefits from the cluster-based coordina-tion in small cell networks.

REFERENCES

[1] ComScore, “Mobile future in focus 2013,”Tech. Rep., Feb. 2013. [Online]. Available:http://www.comscore.com/Insights/Presentationsand Whitepapers/2013/2013Mobile

[2] M. W. Arshad, A. Vastberg, and T. Edler, “Energy efficiency improve-ment through pico base stations for a green field operator,” in Proc.IEEE Wireless Communications and Networking Conf. (WCNC), Paris,France, Apr. 2012, pp. 2197–2202.

14

[3] P. Gonzalez-Brevis, J. Gondzio, Y. Fan, H. V. Poor, J. Thompson,I. Krikidis, and P.-J. Chung, “Base station location optimization forminimal energy consumption in wireless networks,” inProc. IEEEVehicular Technology Conf. (VTC), Budapest, Hungary, May 2011, pp.1–5.

[4] K. Son, H. Kim, Y. Yi, and B. Krishnamachari, “Base station operationand user association mechanisms for energy-delay tradeoffs in greencellular networks,”IEEE J. Sel. Areas Commun., vol. 29, no. 8, pp.1525–1536, Sep. 2011.

[5] Y. Q. Bian and D. Rao, “Small Cells Big Opportuni-ties,” Huawei, Tech. Rep., Feb. 2014. [Online]. Available:www.huawei.com/ilink/en/download/HW330984

[6] B. Rengarajan and G. de Veciana, “Practical adaptive user associationpolicies for wireless systems with dynamic interference,”IEEE/ACMTrans. Netw., vol. 19, no. 6, pp. 1690–1703, Dec. 2011.

[7] S. Zhou, A. J. Goldsmith, and Z. Niu, “On optimal relay placementand sleep control to improve energy efficiency in cellular networks,” inProc. IEEE Intl. Conf. on Communications (ICC), Kyoto, Japan, 2011,pp. 1–6.

[8] S. Bhaumik, G. J. Narlikar, S. Chattopadhyay, and S. Kanugovi, “Breatheto stay cool: adjusting cell sizes to reduce energy consumption,” in Proc.ACM Special Interest Group on Data Communications on Green Netw.(SIGCOMM), New Delhi, India, 2010, pp. 41 – 46.

[9] H. Celebi, N. Maxemchuk, Y. Li, and I. Guvenc, “Energyreduction insmall cell networks by a random on/off strategy,” inProc. IEEE GlobalCommunications Conf. (GLOBECOM) Workshop, Atlanta, GA, USA,Dec. 2013.

[10] Y. S. Soh, T. Q. Quek, and M. Kountouris, “Dynamic sleep modestrategies in energy efficient cellular networks,” inProc. IEEE Intl.Conf. on Communications (ICC), Budapest, Hungary, Jun. 2013, pp.3131–3136.

[11] H. Kim, H. Y. Kim, Y. Cho, and S.-H. Lee, “Spectrum breathing andcell load balancing for self organizing wireless networks,” in Proc. IEEEIntl. Conf. on Communications (ICC) Workshop, Budapest, Hungary,Jun. 2013, pp. 1139–1144.

[12] L. Rokach and O. Maimon,Data Mining and Knowledge DiscoveryHandbook. Springer US, 2005, ch. Clustering Methods, pp. 321–352.

[13] U. von Luxburg, “A tutorial on spectral clustering,”CoRR, vol.abs/0711.0189, 2007. [Online]. Available: http://arxiv.org/abs/0711.0189

[14] R. Jacob, S. Ritscher, C. Scheideler, and S. Schmid,Algorithms andComputation. Springer Berlin Heidelberg, 2009, ch. A Self-stabilizingand Local Delaunay Graph Construction, pp. 771–780.

[15] E. Ogston, B. Overeinder, M. van Steen, and F. Brazier, “A methodfor decentralized clustering in large multi-agent systems,” in Proc. Intl.Joint Conf. on Autonomous Agents and Multiagent Systems (AAMAS),Melbourne, Australia, 2003, pp. 789–796.

[16] L. Backstrom and J. Leskovec, “Supervised random walks: Predictingand recommending links in social networks,” inProc. Intl. Conf. onWeb Search and Data Mining (WSDM), Hong Kong, China, 2011, pp.635–644.

[17] A. Abdelnasser, E. Hossain, and D. I. Kim, “Clustering and resourceallocation for dense femtocells in a two-tier cellular OFDMA network,”IEEE Trans. Wireless Commun., vol. 13, no. 3, pp. 1628–1641, Mar.2014.

[18] K. Hosseini, H. Dahrouj, and R. Adev, “Distributed clustering andinterference management in two-tier networks,” inProc. IEEE GlobalCommunications Conf. (GLOBECOM), California, USA, Dec. 2012, pp.4267–4272.

[19] G. Lee, H. Kim, Y.-T. Kim, and B.-H. Kim, “Delaunay triangulationbased green base station operation for self organizing network,” in Proc.IEEE Intl. Conf. on Green Computing and Communications and Internetof Things and Cyber, Physical and Social Computing (GreenCom-iThings-CPSCom), Beihang University, China, Aug. 2013, pp. 1–6.

[20] M. Bennis, S. M. Perlaza, and M. Debbah, “Learning coarse correlatedequilibria in two-tier wireless networks,” inProc. IEEE Intl. Conf. onCommunications (ICC), Ottawa, Canada, Jun. 2012, pp. 1 – 5.

[21] L. P. Qian, Z. Y.J.A., and M. Chiang, “Distributed nonconvex powercontrol using Gibbs sampling,”IEEE Trans. Commun., vol. 60, no. 12,pp. 3886–3898, 2012.

[22] H. Kim, G. de Veciana, X. Yang, and M. Venkatachalam, “Distributedα-optimal user association and cell load balancing in wireless networks,”IEEE/ACM Trans. Netw., vol. 20, no. 1, pp. 177–190, Feb. 2012.

[23] D. P. Bertsekas and R. G. Gallager,Data Networks, 2nd ed. PrenticeHall, 1992, ch. Delay Models in Data Networks.

[24] G. P. Koudouridis and H. Li, “Distributed power on-off optimisationfor heterogeneous networks – a comparison of autonomous andcoop-erative optimisation,” inProc. IEEE Intl. Workshop on Computer Aided

Modeling and Design of Communication Links and Networks (CAMAD),Barcelona, Spain, Sep. 2012, pp. 312–317.

[25] P. Frenger, P. Moberg, J. Malmodin, Y. Jading, and I. Godor, “Reducingenergy consumption in LTE with cell DTX,” inProc. IEEE VehicularTechnology Conf. (VTC), Budapest, Hungary, May 2011, pp. 1–5.

[26] K. Hiltunen, “Utilizing enodeb sleep mode to improve the energy-efficiency of dense LTE networks,” inProc. IEEE Personal Indoor andMobile Radio Communications (PIMRC), London, UK, Sep. 2013, pp.3249–3253.

[27] 3GPP, “Evolved universal terrestrial radio access (E-UTRA); Furtheradvancements for E-UTRA physical layer aspects,” 3rd Generation Part-nership Project (3GPP), TR 36.814-900, Mar. 2010. [Online]. Available:http://www.3gpp.org/ftp/Specs/archive/36series/36.814/36814-900.zip

[28] F. Richter and G. Fettweis, “Base station placement based on forcefields,” in Proc. IEEE Vehicular Technology Conf. (VTC), May 2012,pp. 1–5.

[29] A. L. Yuille and N. M. Grzywacz, “The motion coherence theory,” inProc. Intl. Conf. on Computer Vision (ICCV), Florida, USA, Dec. 1998.

[30] A. Shashua, “Introduction to machine learning: Class notes 67577,”The Hebrew University of Jerusalem, Israel, 2009. [Online]. Available:http://arxiv.org/abs/0904.3664

[31] K. Sung, S. K. Lee, and H. Kim, “Improving location estimation withtwo-tier particle filtering in mobile wireless environment,” in Proc. IEEEIntl. Conf. on Computer Communications and Networks (ICCN), Maui,Hawaii, Jul. 2011, pp. 1–6.

[32] S. Bandyopadhyay and E. Coyle, “An energy efficient hierarchicalclustering algorithm for wireless sensor networks,” inAnnual JointConference of the IEEE Computer and Communications (INFOCOM),California, USA, 2003, pp. 1–6.

[33] P. Baracca, F. Boccardi, and N. Benvenuto, “A dynamic clusteringalgorithm for downlink CoMP systems with multiple antenna UEs,”EURASIP Journal on Wireless Communications and Networking, vol.2014, no. 1, 2014.

[34] E. Katranaras, M. A. Imran, M. Dianati, and R. Tafazolli, “Greeninter-cluster interference management in uplink of multi-cell processingsystems,”IEEE Trans. Wireless Commun., vol. 13, no. 12, pp. 6580–6592, Dec. 2014.

[35] M. Hammoudeh, R. Newman, and S. Mount, “Modelling clustering ofsensor networks with synchronised hyperedge replacement,” in GraphTransformations, ser. Lecture Notes in Computer Science. SpringerBerlin Heidelberg, 2008, vol. 5214, pp. 490–492.

[36] S. Bandyopadhyay and E. J. Coyle, “An energy efficient hierarchicalclustering algorithm for wireless sensor,” inAnnual Joint Conferenceof the IEEE Computer and Communications (INFOCOM), Mar. 2003,vol. 3, pp. 1713–1723.

[37] S. Perlaza and S. Lasaulce, “Game-theoretic solution concepts andlearning algorithms,” inMechanisms and Games for Dynamic SpectrumAllocation, T. Alpcan, H. Boche, M. L. Honig, and H. V. Poor,Eds. Cambridge University Press, pp. 185–226. [Online]. Available:http://dx.doi.org/10.1017/CBO9781139524421.009

[38] P. Bremaud,Markov Chains: Gibbs Fields, Monte Carlo Simulation,and Queues, ser. Texts in Applied Mathematics (Book 31). Springer,1998.

Sumudu Samarakoon received his B. Sc. (Hons.)degree in Electronic and Telecommunication Engi-neering from the University of Moratuwa, Sri Lankaand the M. Eng. degree from the Asian Instituteof Technology, Thailand in 2009 and 2011, respec-tively. He is currently persuading Dr. Tech degree inCommunications Engineering in University of Oulu,Finland. Sumudu is also a member of the researchstaff of the Centre for Wireless Communications(CWC), Oulu, Finland. His main research interestsare in heterogeneous networks, radio resource man-

agement, machine learning, and game theory.

15

Mehdi Bennis received his M.Sc. degree in Electri-cal Engineering jointly from the EPFL, Switzerlandand the Eurecom Institute, France in 2002. From2002 to 2004, he worked as a research engineer atIMRA-EUROPE investigating adaptive equalizationalgorithms for mobile digital TV. In 2004, he joinedthe Centre for Wireless Communications (CWC)at the University of Oulu, Finland as a researchscientist. In 2008, he was a visiting researcher at theAlcatel-Lucent chair on flexible radio, SUPELEC.He obtained his Ph.D. in December 2009 on spec-

trum sharing for future mobile cellular systems.His main research interests are in radio resource management, heteroge-

neous networks, game theory and machine learning in 5G networks andbeyond. He has co-authored one book and published more than 100 researchpapers in international conferences, journals and book chapters. He wasthe recipient of the prestigious 2015 Fred W. Ellersick Prize from theIEEE Communications Society. Dr. Bennis serves as an editorfor the IEEETransactions on Wireless Communications.

Walid Saad (S’07, M’10, SM15) Walid Saad re-ceived his Ph.D degree from the University of Osloin 2010. Currently, he is an Assistant Professor atthe Bradley Department of Electrical and ComputerEngineering at Virginia Tech, where he leads theNetwork Science, Wireless, and Security (NetSci-WiS) laboratory, within the Wireless@VT researchgroup. His research interests include wireless and so-cial networks, game theory, cybersecurity, and cyber-physical systems. Dr. Saad is the recipient of theNSF CAREER award in 2013, the AFOSR summer

faculty fellowship in 2014, and the Young Investigator Award from the Officeof Naval Research (ONR) in 2015. He was the author/co-authorof fourconference best paper awards at WiOpt in 2009, ICIMP in 2010,IEEE WCNCin 2012, and IEEE PIMRC in 2015. He is the recipient of the 2015FredW. Ellersick Prize from the IEEE Communications Society. Dr. Saad servesas an editor for the IEEE Transactions on Wireless Communications, IEEETransactions on Communications, and IEEE Transactions on InformationForensics and Security.

Matti Latva-aho received the M.Sc., Lic.Tech. andDr. Tech (Hons.) degrees in Electrical Engineeringfrom the University of Oulu, Finland in 1992, 1996and 1998, respectively. From 1992 to 1993, hewas a Research Engineer at Nokia Mobile Phones,Oulu, Finland after which he joined Centre forWireless Communications (CWC) at the Universityof Oulu. Prof. Latva-aho was Director of CWCduring the years 1998-2006 and Head of Departmentfor Communication Engineering until August 2014.Currently he is Professor of Digital Transmission

Techniques at the University of Oulu. His research interests are related tomobile broadband communication systems and currently his group focuseson 5G systems research. Prof. Latva-aho has published 300+ conference orjournal papers in the field of wireless communications.