Two-Stage Friend Recommendation Based on …...friend recommendation method with two stages. In the...

1520-9210 (c) 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TMM.2017.2652074, IEEETransactions on Multimedia

1

Two-Stage Friend Recommendation Based onNetwork Alignment and Series-Expansion of

Probabilistic Topic ModelShangrong Huang∗, Jian Zhang∗, Dan Schonfeld†, Lei Wang‡, and Xian-Sheng Hua§

∗School of Computing and Communications, University of Technology Sydney, Australia†Multimedia Communications Laboratory, University of Illinois at Chicago, Chicago, USA‡School of Computing and Information Technology, University of Wollongong, Australia

§Alibaba Group, Hangzhou, China

Abstract—Precise friend recommendation is an importantproblem in social media. Although most social websites providesome kinds of auto friend searching functions, their accuraciesare not satisfactory. In this paper, we propose a more precise autofriend recommendation method with two stages. In the first stage,by utilizing the information of the relationship between textsand users, as well as the friendship information between users,we align different social networks and choose some ”possiblefriends”. In the second stage, with the relationship between imagefeatures and users we build a topic model to further refine therecommendation results. Because some traditional methods suchas variational inference and Gibbs sampling have their limitationsin dealing with our problem, we develop a novel method to findout the solution of the topic model based on series expansion.We conduct experiments on the Flickr dataset to show thatthe proposed algorithm recommends friends more precisely andfaster than traditional methods.

Index Terms—Friend Recommendation, Topic Model, Seriesexpansion

I. INTRODUCTION

FRIEND recommendation is a primary function in socialnetwork services and aims to recommend new social

links for each user. Today when we lodge on the main socialwebsite such as Facebook, Twitter, and LinkedIn etc., wereceive many recommendations of online friends. Seeing andhearing what the friends look at and listen to, or sharingour experience with our friends is an unparalleled experience.However, the decision of making friends is a complex humanbehaviour and can be affected by many different factors suchas age, gender, location, interest [1], etc. As a consequence,similar to real life, finding a good on-line friend is not easywithout the help of good recommendations. Traditional friendrecommendations widely applied by Facebook and Twitter areoften based on common friends and similar profiles such ashaving the same hobbies or studying in the same fields. Thesemethods usually provide a long ranked possible friend list, butthe recommendation precision is usually not satisfactory dueto its complexity.

Nowadays people are commonly retained in a multi-resource environment, and usually do not seek friends based on

Manuscript received XXXX X, XXXX; revised XXXX XX, XXXX.Corresponding author: S. Huang ([email protected])

only one kind of information anymore. Recently cross domainfriend recommendation technologies have been extensivelyexplored [2][3][4]. [2] applies a matrix factorisation methodto combine the image and text information, [3] considers theproximity and homophily information for synthesised recom-mendation, and [4] specifies individuals’ requirements fromdifferent domains. Most of these papers utilise informationfrom different resources simultaneously for recommendation.In this paper, we approach this recommendation problem ina different way by utilizing the multi-domain information indifferent stages for a more precise recommendation.

The reason why we apply the multi-stage friend recom-mendation scenario lies in the complexity of multi-sourceinformation and the decision making behaviour of people. Forexample, an individual might make an on-line friend becausethey discuss a hard mathematical problem, or it is possiblethat he/she makes a friend because they both enjoy a film.The reason for friend making might be very diverse. It wouldbe relatively difficult if we consider different factors togetherat the same time for recommendation. In our opinion, it ismore convenient and clearer to analyse these factors step bystep, rather than to deal with such cross-domain informationas a whole. By untwisting the different factors in the recom-mendation procedure and analysing each factor in depth, amore precise recommendation performance is expected. As aconsequence, we apply a two-stage framework to synthesiseheterogeneous information from different domains.

In this paper, we concentrate on the widely-used imageand image-related experience sharing website Flickr, whereindividuals can upload photos and tags for sharing as wellas make online friends (Flickr Contact) and join communities(Flikr Group). Tag (text) information is quite useful for friendrecommendation since it is simple and direct. For example,two individuals that both have interest in tags “travel” and“historical people” have higher probability to be friends witheach other. Text-based friend recommendation have been suc-cessfully developed in [5][6]. We apply the text informationin our first stage.

Image information is also helpful for friend recommenda-tion. On the other hand, image information is vaguer and morecomplex for this task. For example, it is hard to claim that twoindividuals who both enjoy some vivid and colourful photos,



2

or some photos of beautiful women have the higher probabilityto be friends. As a consequence, in our algorithm, we utilisethe image information as a supplementary source in the secondstage of the algorithm, to refine the result of the first stage.

In the first stage, similar to [5], based on the correlationof different networks, we align the tag-similarity network tofriend network to obtain a possible friend list. Specifically, weconsider each user as one node in a graph, and we crawl theuploaded tags from each user and calculate the tag similaritybetween any two users as the edges to form a tag-similaritynetwork. On the other hand, we also obtain the friendshipinformation in Flickr, and if two users have friendship witheach other, we add an edge between the two to form a contactnetwork. In this way we build a tag-similarity network anda contact network that have the same nodes but differenttopologies. Because the tag-similarity network and contactnetwork on Flickr are related to each other, we dig theircorrelation by choosing important tag features, to make thetag-similarity network more similar to the contact network. Inthis way, the chosen tag features provide a guideline for friendrecommendation. This stage makes a mass election of possiblefriends.

In the second stage, to overcome the problem that the masselection considering only the tag information might not beprecise, we build a topic model to illustrate the relationshipbetween user’s friend making behaviour and the image featuresthey have uploaded. This stage refines the list obtained inthe first stage. The main reason for applying a topic modelin our second stage lies in the fact that the topic model hasthe ability to tell on what probability a user would prefer aphoto/item/friends.

The probabilistic topic model discovers the abstract “topics”that occur in a collection of documents/datasets, and it hasbeen widely used in recommendation systems [7][8][9]. Byintroducing some latent variables and applying the Bayesianrule, it is conceptually easy to combine information fromdifferent domains and make specific recommendations [7][9].Generally it assumes that people’s various behaviours such asshopping, posting and friend making are controlled by somelatent topics. Certain people have particular bias on differentlatent topics. For an individual that acts differently in differentdomains, his/her latent interest topic might be similar. Forexample, a user who posts many different photos about foodon Flickr might have higher probability to be interested inthe topic of cooking, and thus it is reasonable to recommendsome kitchenware to him/her on Amazon. Furthermore, thetopic model provides a relatively precise probability to showto what extent an individual is interested in a topic, and thusmakes it easy for further recommendation.

In this paper, we propose a topic model to correlate thedata about the Flickr image information and the contactinformation. Compared with some previously cross-domaintopic models, our model is more compact with less parameters,which leads to some computational convenience. Briefly, weassume that the attractiveness of photos is controlled by alatent variable, and individuals’ photo uploading behavioursand their friend making behaviours are controlled by someother latent variables. By determining the values of these latent

variables we can predict individuals’ friends.However, it is often not easy to find the solution of a

topic model when different domains are concerned, for itinvolves the integrals of several coupled random variables,which is a complicated mathematical problem in general [10].Two methods are widely used to deal with this problem:Gibbs sampling [11] and variational inference [10], or thecombination of the two [12]. Although applied successfullyin many cases, both of them have some disadvantages: forGibbs sampling, it is inefficient for large count values sinceit requires averaging over many samples to reduce variance;for variational inference, though it is efficient to deal withlarge scale data, the variational step makes it hard to controlthe precision when approximating the integrals when makingthe Bayesian inference. In this paper, with the help of Mellinand inverse Mellin transform, we propose a novel way basedon series expansion to calculate the coupled integrals that arerequired in the Bayesian inference.

Matrix factorization (MF) method can be also appliedto deal with the cross domain recommendation problems[13][14]. It decomposes different social networks into latentvectors to find the important factors that influence individuals’social behaviours, and make recommendations based on theselatent factors. However, it lacks a mechanism to draw thecomplete distributions of the whole social network, and thusmight lead to some local optimum. Our proposed methodprovides a way to describe the whole distribution of the socialnetwork, to perform a better recommendation.

To sum up, we build a two-stage friend recommendationsystem based on text and image data: in the first stage, weapply tag-user information to get a possible friend list, and inthe second stage we refine the list by utilizing the image-userinformation. Our main contributions are as follows: Firstly, webuild a compact topic model to analyse the relationship of thedata from different domains. Secondly, we propose a novelmethod based on the study of the distribution of algebra ofrandom variables to find a solution of the model. The solutionis given in a series expansion form, and can lead to moreprecise solutions of the model. As far as we know,this is thefirst time to solve a topic model from the aspect of integralseries expansion. We also make comprehensive experimentsto show the effectiveness of our method.

The rest of the paper is organised as follows: SectionII outlines related work. Section III introduces our systemframework. Section IV gives the detailed explanation of ourseries expansion method. Section V evaluates the performanceof our method and some analysis is made according to theresults. Lastly, Section VI concludes our work.

II. RELATED WORK

Our work in this paper is mainly related to the followingresearch fields: friend recommendation, topic model, and al-gebra of random variables.

A. Friend Recommendation

Friend recommendation is a relative challenging issue com-pared with item or group recommendations, for there might be



3

various reasons for two persons to become friends, and onlineand offline friendships are quite different. Recently, [15] evenprovides some method to distinguish the online and offlinefriends.

[6] makes a survey of individuals’ daily life, and then sum-marises the reports as their “life styles” using latent Dirichletallocation algorithm (LDA). [8] collects individuals’ posts inMicro-blogs and arranges them in a chronological order. Bybuilding a temporal-topic model it can recommend differentfriends to each user at different time, as the user’s interestchanges from time to time. [16] utilises the information fromdifferent platforms (Flickr, Twitter, Google+, etc) to alleviatethe sparsity problem of social networks, the idea is thatGoogle+ can provide a information bridge between thesedifferent social platforms. In this paper, We dig the friendrecommendation problem deep by considering multimediainformation one platform, and applying a two-step scenarioto refine the result.

1) Cross-Domain Recommendation: As mentioned in sec-tion I, individuals’ decision of making friends are often multi-dimensional. As a result, recently many researchers considerfriend recommendation based on cross-domain information.[17] considers the friend recommendation problem at workingplaces and conferences, by utilising both users’ temporallocation as well as their common friend information. [2]combines three aspects of each user’s information: the itemsone likes, the friends one has, and the groups one belongsto. Such information of different aspects is synthesised andintegrated into one cost function. By optimizing the costfunction, the heterogeneous data are fused for item, groupand friend recommendations. In [16], individuals that haveboth accounts in Flickr, Twitter and Google+ are collected tobuild the relationship of the two social websites. The commonbehaviours of each user in Flickr and Twitter are analysed andthe friend recommendation of the two domain is made basedon these common behaviours.

[4] divides the different data in Flickr into two classes:interaction data(comments, making favorite photos) and sim-ilarity data(common friends, groups, tags, geo, visual), andapplies these two classes of data comprehensively to estimatethe strength of the ties between users. [18] utilises Flickr socialrelations for further multimedia recommendation. It buildsa topic model to combine the image, text, and friendshipinformation to discovery individuals’ preferences. The topicmodel is solved via Gibbs sampling.

For the works listed above, the data from different domainsare processed simultaneously or fused together to get the finalrecommendation result. On the one hand, the above methodstake the advantage that data from different domains mightbe related to each other; On the other hand, these methodscombine the cross-domain information in one step ([16]) orsynthesise it in one cost function ([2]), thus usually can notgive a good explanation of how the data from a specific domaincontribute to the final recommendation result, and the twisteddata from different domains often makes the problem morecomplex. To have a better understanding of the effectivenessof the data from each domain, in this paper, we design a two-step recommendation that in each step we utilise the data from

one domain.2) Multi-Stage Recommendation: Existing multi-stage rec-

ommendations are usually applied to find some patterns ofusers or items. For example, in [19], a two-stage mobilerecommendation is proposed to help users find the correctevents. The first stage clusters people according to theirprofile similarity and the second stage discovers the event-participating pattern. [20] designs the first stage to find somerelated resources that one user requires, and the second stageis used to find some patterns that the user might prefer fromthe previous stage for further recommendation. Both [19] and[20] can handle the cold-start problem well but do not considermuch about the cross-domain problem.

In this paper we apply a different strategy: in the first stage,some relatively good results are chosen by observing the textdata; then we refine the results in our second stage, with thehelp of image data. In our previous paper [21], we provide atwo-stage recommendation and each stage utilises data fromdifferent domains by alignment and co-clustering. However,co-clustering method lacks the ability to tell the intimacydistance between two individuals exactly but only to grouppeople roughly with similar properties, and thus can not makeprecise recommendation. To overcome this problem, in thispaper we propose a probabilistic topic model in the secondstage for a better recommendation. We also provide a noveland more precise method to solve the topic model problem.

B. Probabilistic Topic ModelIn the second stage of our model, a topic model is applied

to get a more precise recommendation.1) Topic Model in Recommendation: The probabilistic

topic model is a successful approach solving the problemfor information retrieval[10] and recommendation[7][8][9].For example, [8] recommends temporary friends to users bybuilding models that contain latent variables that illustrateusers’ interests change with time.

By assuming some latent factors it is conceptually easy tobuild the relationships among different domains. [7] designsa model that connects the Flickr and Foursquare data forimage, topic and item recommendation. It assumes that bothdomains have some common latent factors and each domainalso has its own latent factors, and the users’ activities on thesetwo platforms are the synergism of all these factors. Gibbssampling is applied to find the value of the latent factors.[9] considers the friendships and the votings on the largeFilm rating website. To predict individual’s flavour about filmshis/her social relationships and scores of films are combinedwith some latent factors. Variational methods are applied tosolve the model.

To make the model to illustrate the situation of the realworld more accurately and reasonably, both [7] and [9] makemany assumptions of the latent topic and thus contain manyunknown parameters to infer: [9] contains more than 10unknown parameters and [7] has more than 30. The presenceof so many unknown variables not only greatly increases thecomplexity of the algorithm, but also leads to other problemssuch as over-fitting or redundancies. In this paper we build themodel in a more compact manner.



4

2) Gibbs Sampling, Variational Inference and Matrix Fac-torization: Due to the coupling of latent variables, the directinference is usually impossible for a specific topic model.Generally there are two methods to find a solution for topicmodel: Gibbs sampling [22] and variational inference [10].For some complex multivariate probability distributions, todetermine the parameters of the distribution, direct samplingis difficult. Gibbs sampling samples the marginal distributionof one variable each time, and iteratively samples all themarginal distributions. The variational method, on the otherhand, approaches the solution by approximating the originalcomplex distribution with a factorised one, which is easier tohandle.

As stated in section I, both of the two methods have someweaknesses: Gibbs sampling has difficulties in handling bigdata problems, and the variational method can not determineif the approximation is close to the original one. Someresearchers consider combining the two in one problem: In[12], small counts of data are sampled and the variationalmethod is applied to update large counts, which improves theperformance on the large dataset. However, how accurate theapproximation of variational method is not yet discussed in[12]. In this paper, we propose a new solution to a topic modelby directly calculating the distribution of the latent variables.

Compared with the above two methods, MF-based methodalso assumes some latent variables but instead of determiningthe marginal distribution of the observed data, it factorizesthe observed data into different latent factors, which leads tosome computational convenience and efficiency. Both of [13]and [14] utilize user friendship network and user-item networkand obtain some latent factors that show the preference of in-dividuals. The recommendations based on these latent factorsare relatively effective. On the other hand, they do not try tofind the probabilistic distribution of the network and all ofthese methods apply some gradient descent methods, that arerelatively easy to be trapped into a local optimum. Our methodavoids this drawback by deducing the distribution of the wholeprobabilistic model.

C. Algebra of Random variables

The essential problem of our approach in this paper isto get the exact mathematical expression of the coupling ofdifferent random variables, mainly the sum and product ofrandom variables. These problems were extensively discussedin the 1950s to 1970s year, last century, during which timethe random process was a hot research topic but the computersimulation technology was not well developed. In [23] [24][25], the products of typical distributions such as Beta, Gammaand Rayleigh are discussed. Most of these works utilise theMellin transform [26] as the essential tool for deducing. [27]gives a good summary of these works and also discusses thedistribution of the sum of random variables. The algebra ofrandom variables has also been studied recently in certainfields such as wireless communication in [28] and [29]. Theseworks show that the product and quotient of random variableswith certain distributions can be expressed analytically. Wewill mainly apply some of the results in [25][27] later in our

work. As Gaussian distribution has some good properties(itsdomain of definition is all the real values, and has a centralpoint, etc.) we assume that our latent variables to be Gaussiandistributed.

III. SYSTEM MODEL

The main framework of our model is shown in Figure 1,which contains two stages: In the first stage, network align-ment is applied to generate a possible friend list, by correlatingthe tag and contact data in Flickr; In the second stage, the user-uploaded image features generate some topics by utilising aprobabilistic topic model, and a new method is developed tosolve the model for precise friend recommendation.

Fig. 1: Two-Stage System Illustration

A. First Stage: Network Alignment

The detailed alignment method has been discussed in [5].The following is an introduction of its basic idea. An individ-ual may join different social networks for different purposes.For example, one may at the same time join a football fannetwork for physical practice and a restaurant informationsharing network to look for the best food. He/she playsdifferent social roles in these different networks, and mightmake different friends. However, these different social rolesfor one individual are not independent, but related to eachother.(The man might look for some food that helps quicklyrecuperate after hard physical practice). The motivation forsocial network alignment lies on the fact that these differentnetworks, though having different edges (relationships), areusually related to each other. Taking Flickr as an example,according to the uploaded-tag-similarity of each user and theircontact list, a tag similarity network and a contact network areformed. Although the topologies of the two networks are notthe same, they are related to each other, for users uploadingsimilar tags on Flickr have higher probability to make friendswith each other. By digging the correlation of the topologies ofdifferent networks we may make inference for the knowledgefrom one domain to another.



5

Specifically, we align the Flickr tag-similarity network withthe contact network, so that after the alignment, one tight edgebetween two users in the tag similarity network would implythat these two users have higher probability to have contactwith each other. We align the tag-similarity network withthe contact network by selecting important tag features. Thereason we apply feature selection here lies in the phenomenonthat when we look for online friends, it is common thatwe do not take care of all the factors of a person butconcentrate on certain points that would interest ourselves.As an example, a traveller might post his photos with thefollowing tags: “Sydney”, “Blue Mountain”, “great view”, and“street”. Among these tags some people might contact him/herfor some more details about the experiences in ‘’Sydney”and ‘’Blue Mountain”, but seldom would have interests about“great view” or “street” because they are too common. We cantreat these two tags as redundancy for friend making. Basedon this observation, we believe that some Flickr tags can bemore indicative in the task of friend recommendation, becausethey are more important to reflect the connections on thecontact network. We can treat these tags as important featuresfor friend recommendation. Inspired by this phenomenon, wedesign a method to choose some important features that aremore helpful for friend making decision.

Mathematically, assume that the feature selection matrix tobe W, the known tag-user matrix to be X, the tag distancematrix to be L, and the first d eigenvector-matrix of the contactnetwork to be V, the important feature can be obtained bysolving the following problem:

minW‖XW −V‖+ µtr(WTXTLXW) + λ‖W‖2,1 (1)

The first term of Eq. (1) aligns the tag-similarity networkto the contact network so that they become more similar toeach other, and the second term preserves the local structureof the original tag-similarity network. The third term is forregularization. In this way the tag feature selection matrix Wmakes the topology of the tag-similarity network more similarto the contact network, while preserving the topology of thetag-similarity network as much as possible. In other words,we align the tag-similarity network to the contact network. Bycomparing the similarity of two users on the those importanttags we can generate a possible friend list for each user. Thesolution of W in Eq. (1) is discussed more thoroughly in [5].

B. Second Stage: Topic Model

In the previous stage we get a possible friend list by con-sidering the correlation between the tag and contact networkson Flickr. However, as the real world friend relationship isaffected by many factors[1], one stage is usually not enoughfor a precise friend recommendation. In the following stage,we introduce the image data as auxiliary information to refinethe recommendation list.

We apply the topic model to combine the image data andthe friendships in Flickr. It is common sense that a personuploads a photo on Flickr because he/she likes the photo.Why does he/she like the photo? We assume that in one’smind, some latent interest factors control his/her taste of

image. For example, some people like colourful, vivid photos,while others prefer spectacular or imposing ones; childrenenjoy comic-style pictures while adults have more interestsin realistic-style paintings; young women pay much attentionto photos of beautiful clothes while young men to electricaldevices. These latent factors are determined by various aspectssuch as age, gender, living experiences, etc. and can notbe observed or simply summarised. We assume individuals’interest latent factor to be v. Each image contains the factorsthat attract people, such as colour, line, or history, which weassume to be a. The correlation of v and a determines whethera user would upload an image.

Similarly, we assume that each user exhibits some attractivefactors during his/her activities in Flickr such as uploadingphotos, writing descriptions of photos and making comments,etc. We also summarise these attractive factors with thethird latent variable b. Notice that the same user’s interestlatent factor v and attractive factor b are not the same. Thecombination of b and v determines whether two users shouldmake friends with each other. For simplicity we view themas independent from each other. The topic model is shown inFigure 2.

Fig. 2: The Probabilistic Topic Model Combining Image-UserNetwork and Contact Network

In Figure 2, C and I stand for the 0 − 1 contact networkand image-user network, respectively. C is an n × n matrixwhere n is the number of users. I is an n× f matrix where fstands for the number of total features. For C, if user k anduser j are friends with other, then Ckj equals one, and zerootherwise. For I, if the uploaded photos of user i contain animage feature j, then Iij equals one, and zero otherwise. astands for image factor, and b stands for individuals’ socialinterest factor, respectively. v stands for individuals’ commoninterest factor that has effect on both his choice of images andfriends. NI and NC stand for zero-mean additive noises. Therelationship can be mathematically expressed as follows:

Iij = ai × vj + NIij ,Ckj = bk × vj + NCkj (2)

We assume that all the latent random variables ai, bk and vjare Gaussian distributed with the parameters of means andvariances of µa, σa, µb, σb, µv , and σv , respectively. Thereason we choose Gaussian distribution is as follows: Althoughsome other distributions that are in the form of an H-function(such as Beta, Gamma or Rayleigh distributions) would leadto some calculation convenience [27], we assume Gaussiandistribution here because it is defined on the whole real domainand contains negative values and has a central point, while



6

other distributions such as Beta are only defined on the positivereal domain.

The coupling between random variables a, b and v makesthe integral of Eq. (2) often intractable. Traditional methodsdealing with Eq. (2) contain Gibbs sampling [11] and varia-tional inference [10]. Gibbs sampling meets with difficultieswhen the data scale is large, and the variational method appliessome approximation that the precision is hard to control. Inthe following we develop a new approach to solve Eq. (2) thatis based on Mellin transform and series expansion.

IV. SERIES EXPANSION

A. Product of Gaussian Random Variables

When dealing with the distribution of product of randomvariables, the Mellin transform is an essential tool[27]. Wetake the first equation in Eq.(2) to explain its basic idea. Forsimplicity we first neglect the noise term Nij (its effectivenessis to be discussed later) and we have Iij = aivj for tworandom variables ai and vj with different probability distri-bution functions. One useful property for Mellin transofrmis: the Mellin transform of the product of two probabilitydensity functions (PDF) is equal to the product of the Mellintransforms of their PDFs.

Mathematically, we recall the following rule [27]: If aiand vj are two non-negative random variables with the PDFsfa(ai) and fv(vj), their product Iij = aivj has a distributionh(Iij), and then the Mellin transform of h(Iij) is precisely theproduct of Mellin transform of fa(ai) and fv(vj), respectively.The expression is given as:

M(h(Iij)) =M(fa(ai))M(fv(vj)) (3)

where the Mellin transform and its inverse of an analyticalfunction f(x) are defined as follows:

M(s) =

+∞∫0

xs−1f(x)dx (4)

M−1(x) =1

2πi

c+i∞∫c−i∞

x−sM(s)ds (5)

where c in Eq. (5) stands for an arbitrary real number. Withthe help of Eq. (3)-(5) and the known distribution of ai and vj ,we can give an exact mathematical expression for distributionof the coupling of the two random variables ai and vi.

In this way we can first deduce the Mellin transform ofeach of the distributions, then make product of the two, andfinally inverse the Mellin transform to get the final productdistribution. In this way, we first calculate the distribution ofI in Eq. (2).

From the previous assumption we know that ai, bk and vjfollow the Gaussian distribution with mean µai, µbk, µvj andthe variance σai, σbk, σvj . We further take the symbol of fai,fbk and fvj as their PDFs. We first do the Mellin transform onai and vj separately to get M(fa(ai)) and M(fv(vj)), andthen we product them and do the inverse Mellin transform tofinally get the distribution of product of two random variables,

which is the distribution of the variables in image-user matrixI. The details are given in [27] and [25], which provide twoequivalent expressions for the distribution of two Gaussianrandom variables. We apply the expression from [25] and thedetails are briefly outlined in the following.

To calculate the distribution of Iij = aivj with Gaussianrandom variables ai and vj , we take the Mellin tranformof fa(ai) and fv(vj). Notice that according to Eq. (4), thepositive and negative parts of the distribution of ai and vjshould be considered separately. We apply the property thatthe Mellin transform of the standard Gaussian distributionis Gamma function[30]: M{e−x2/2} = 2s/2−1Γ(s/2), anda non-central Gaussian distribution can be expressed as astandard Gaussian distribution multiplied by a series in theform: e−

12(x−µ)2 = e−µ

2/2∞∑j=0

1j!µjxje−x

2/2. If we define the

following:ai1 = max(ai, 0), vi1 = max(vj , 0),

ai2 = min(ai, 0), vi2 = min(vj , 0),

Iij−1 = ai1vi1, Iij−2 = ai1vv2,

Iij−3 = ai2vv1, Iij−4 = ai2vj2

And we also define the probability distribution function ofIij−1, Iij−2, Iij−3, and Iij−4 to be h1(Iij), h2(Iij), h3(Iij)and h4(Iij), respectively. Following the methods of [25], andtaking Iij−1 as an example, we have:

MIij−1(s) =

∞∑o=0

µ2oai

(2o)!

µ2ovj

(2o)!Γ2(s) (6)

To get the distribution of Iij−1, we do the inverse Mellintransform of Eq. (6) as:

h1(Iij) =∞∑o=0

(1

2πi)

c+i∞∫c−i∞

(y2)−sµ2oai

2o!

µ2ovj

2o!Γ2(s+ o)ds (7)

Eq. (7) is an integral on half of the complex plane. Accordingto Residue Theorem [31], the solution is expressed with theinfinite residues that are related to the poles on the real plane.By calculating the residues we get:

h1(Iij) = C1

[ ∞∑o=0

C2∞∑s=o

[(Iij)

2s

s−o−1∏t=0

(−s+ o+ t)2(2ψ(1) (8)

−2s−o−1∑w=0

1

−s+ o+ w)− (Iij)

2s ln((Iij)2)

s−o−1∏w=0

(−s+ o+ w)2]

]

where C1 = 1πe− 1

2(µ2aiσai

+µ2vjσvj

), C2 =

(( 1(2o)!

)2(2µ2aiσai

µ2vj

σvj)o), and

ψ(1) is the Euler-Mascheroni constant.Similarly we should also consider the case of h2(Iij) for

a > 0 ∩ v < 0, h3(Iij) for a < 0 ∩ v > 0, and h4(Iij) fora < 0 ∩ v < 0. To sum up, we have:

h(Iij) = h1(Iij) + h2(Iij) (y > 0) (9)= h3(Iij) + h4(Iij) (y < 0)

In a similar manner we can give the expression for h(Ckj)



7

Here we give a short discussion about this series. In the firstplace, this is basically an alternating and power series [32]with infinite terms, with some of the terms multiplied witha logarithm factor. This is a series that when the sequencenumber of the term increases, the absolute value of theterm increases. Some of the terms are positive and someare negative, and the sum of the terms eventually becomesconvergent, as discussed in [33]. However, similar to some ofthe convergent Taylor series, when the absolute value of theseries terms is large, these series converge only when the termnumber of the series is also large. In order to make the seriesto converge rapidly with relatively a small number of terms,in practice, we may normalise the value of Iij to be relativelysmall (In the experiments, the ground truth of Iij and Cij are0 or 1, which is small enough).

B. Additive Noise

From Fig. 2 we see that after the products of a, v and b,v, the results should also add a bias value or noise to get thevalue of Iij and Ckj . In practice it can be interpreted as allthe outer environmental influences other than the users and theitems. For example, the change of seasons for the favour ofclothing, or the change of temperature for the preference offood, etc. Mathematically the PDF of two independent randomvariables are the convolution of their PDFs of the two [27]. Inour case, we can simply consider the environmental influencesNI and NC to be independent from the image factor a,social attractive factor b and individual’s latent factor v. Forsimplicity we assume the additive noise of NI and NC to beGaussian distributed with zero mean and variance of σNi andσNc, respectively. Taking Iij for example, from Eq. (8) wesee that the most important calculation is the convolution ofthe Gaussian function from additive noise e−I

2ij/σ

2Ni and the

term Iij2s log(Iij

2) from Eq. (8), which is formally written asfollows:

d2(Iij) = e−I2ij/σNi ∗ I2sij ln(I2ij) (10)

By calculating the convolution we see Eq. (10) can be ex-pressed as follows:

d2(Iij) = Iij2s+2(

ln Iij2

2s+ 2− 1

(2s+ 2)2)e

(−Iij2)

σNi (11)

In this way we can get a series expression of Eq. (9).So the expression for the distribution of Iij considering the

additive noise is given in Eq.(12).In a similar way we can also obtain the distribution of Cjk.

C. EM for Parameter Estimation

Applying the above, we obtain the exact infinite expansionexpression of the PDF. of I in a series form given in Eq.(12).The expression of Ckj can be obtained in a similar way. FromEq. (12) we can see that the exact value of µai and σai doesnot matter much, but the value of µ2

ai

σaimatters. So we can

assume that ai has standard derivation of 1, and we only needto calculate the average value of ai. Similarly, we also do notneed to calculate σbk and σvj but only assume that vj and bkhave standard derivation.

All the parameters P are summarised in Table I. As men-tioned in Section IV-A, in the experiments, when we choosethe starting point of the parameters not too large, we can makethe series converge in a relatively small number of terms. Thenwe can apply the standard EM method to refine the parametersiteratively. Experimental result shows that the number of seriesterms can be no longer than 10 and after several EM iterations,the precision becomes stable.

TABLE I: Summary of Parameters

µai mean of image factor aiµbk mean of individuals’ social attractive factor bkµvj mean of individual’s interest factor vjσNI variance of image noise NIσNC variance of social noise NC

The EM training process is introduced as follows. For Estep, Consider Eq.(12), which is the Equation we want tomaximize by knowing the value of Iij , with respect to theparameters P as follow:

maxP

h(I | P)) (13)

In the M step, we find the derivative of each parameter inP by fixing other parameters. Then we set the derivative to bezero to get the value for each parameter. The whole processgoes until convergence.

One problem to solve Eq. (12) is that Eq.(12) containsnot only polynomial terms but also exponential terms for theparameters. For simplicity we can make an assumption that theparameters are relatively small, and then we can use the firstseveral terms, or following [34] to get a polynomial expressionof the parameters, to make Eq (12) solvable.

Another problem is that for some parameters such as µai,it contains infinity high order terms that makes the solutionintractable. Again we can make the assumption that theseparameters to be smaller than one, and discard the high orderterms. In practise we keep the terms whose orders are equalor lower than 4, and follow the method discussed in [35] tocalculate the values of the parameters.

From Eq. (12) we can obtain the parameters that relatedto the image-user matrix I, such as µai, µvj , and σNI . Ina similar manner we can also get the parameters related tothe contact matrix C, such as σNC , µbk, and also µvj . Byiteratively updating these parameters relating to the two matrixwe can finally determine the value of all the parameters.

After the EM iterations we fix all the parameters in Table Iand according these parameters we can make the final friendrecommendation.

D. Recommendation Method

When a new user i comes into the network, he/she mayupload some favourite photos as well as some tags. Therecommendation procedure is divided in two stages. In the firststage, a list of possible friends is generated according to thesimilarity of the selected important tags. In the experiments,we put the top 200 users into the list.

In the second stage, according to the features of the imagesuploaded by use i, we get the individuals’ interest factor vi



8

(12)

h(Iij) =1πe− 1

2(µ2aiσai

+µ2vjσvj

)

[[ ∞∑t=0

(1

(2t)!1

(2t)!(2µ2aiσai

)t(2µ2vj

σvj)t) ∞∑s=t

[I2sij e

(−Iij2)

σNi

s−t−1∏m=0

(−s+t+m)2(2ψ(1)− 2

s−j−1∑m=0

1−s+t+m −

I2sij ln(I2ij)

s−t−1∏i=0

(−s+t+m)2]

+∞∑r=0

∞∑t=0

(1

(2t)!(2µ2aiσai

)t 1(2r)!

(2µ2vj

σvj)r + 1

(2t)!(2µ2vj

σvj)t 1

(2r)!(2µ2aiσai

)r) r−i∑s=t

[I2sij

r−s−1∏m=1

m

s−j−1∏q=0

−q−1

ln(I2ij)]+∞∑t=0

∞∑r=t+1

(1

(2t)!(2µ2aiσai

)t

1(2r)!

(2µ2vj

σvj)r + 1

(2t)!(2µ2vj

σvj)j 1

(2r)!(2µ2aiσai

)2r) ∞∑s=r

[ I2sij (2ψ(1)−r−t−1∑m=0

1−s+t+m−

s−t−1∑q=r−t

2−s+t+q )

r−t−1∏m=0

(−s+t+m)s−t−1∏q=r−t

s−t−1∏q=r−t

(−s+t+q)2−

I2sij ln(I2ij)e

(−Iij2)

σNi

r−j−1∏i=0

(−s+j+i)s−j−1∏k=r−1

(−s+j+k)2

]]

±[ ∞∑k=0

(1

(2k+1)!1

(2k+1)!

) ∞∑s=k

[ (I2ij)s+1/2

s−q−1∏m=0

(−s+q+m)2(2ψ(1)− s

s−q−1∑m=0

1−s+q+m )−

(I2ij)s+1/2 ln(I2ij)e

(−Iij2)

σNi

s−q−1∏i=0

(−s+q+i)2

]+∞∑p=1

p−1∑q=0

(1

(2q+1)!(2µ2aiσai

)(p+0.5)

(2µ2vj

σvj)p+0.5 + 1

(2q+1)!(2µ2vj

σvj)q+0.5 1

(2p+1)(2µ2aiσai

)p+0.5) p−1∑s=q

[ (I2ij)(s+1/2)p−s−m∏m=1

(m)

s−q−1∏n=0

(−n−1)

ln(I2ij)]+ 1

(2q+1)!(2µ2vj

σvj)q+0.5 1

(2p+1)!(2µ2aiσai

)p+0.5)

∞∑s=p

[ (I2ij)s+1/2(2ψ(1)−p−q−1∑m=0

1−s+q+1

−s−q−1∑l=p−q

2−s+q+l )

p−q−1∏i=0

(−s+q+m)s−q−1∏l=p−q

(−s+q+l)2−

(I2ij)s+1/2 ln(I2ij)e

(−Iij2)

σNi

p−q−1∏i=0

(−s+q+m)s−q−1∏l=p−q

(−s+q+l)2

]]]

of this user. For a user k in the possible friend list obtainedfrom the first stage, we can also calculate his/her attractiveand interest factors bk. The similarity score of user i and k isobtained by Sik = vibk. The higher the similarity score, themore likely that they are to be friends. So we can rank the 200users in the list according to the similarity score with user i,and recommend the top ones as user i’s friends

The whole procedure is given in Algorithm 1.

Algorithm 1 Two Stage Friend Recommendation

Input:tag feature matrix T, contact matrix C, image-user matrixI, tag and image feature of the new user t and i, thenumbers of possible friends in Stage 1 and final friendsk1 and k, respectively

Output:Friend recommendation list of the new friend

Training:Stage I

1: Determine λ and µ in Eq. (1) via cross validation.2: Solve Eq. (1) with the method in [5]

Stage II3: Generate the expression of distribution of h(Eq. (12)) in

the form of series.4: Apply EM method determining the parameters in Table I

Testing:5: Stage I: Use W calculated in Step 2 to obtain k1 possible

friend list.6: Stage II: Use the parameters in Step 4 to refine the final

recommendation friend list, recommend top k users

E. Complexity Analysis

The complexity analysis of our algorithm is also divided bythe two stages as follows:

Considering the first stage, the complexity of the networkalignment is mainly decided by two steps: the eigenvaluecalculation and the inverse of the similarity matrix, which isgiven by max(min{n, e}3, dn2) as discussed in [5]. e standsfor the number of total tags. As previous defined, n stands forthe number of users, and d stands for the first d eigenvectors.

To solve the topic model of the second stage, Assumetogether we need to make L time iterations. in each iterationof the EM step, assume that we calculate the first g termsof the series of Eq. (12)(In practise we make g = 4). Andit takes e steps to solve a 4th order polynomial equation, asmentioned in Section IV-C. Then the complexity would be ofO(L∗ e∗ g ∗ (n∗ f +n∗n)), where f is the number of imagefeatures, as previously defined.

V. EXPERIMENTS

In this section, we make experiments to show the advantageof our proposed method. First, we introduce our social mediadataset, and then we discuss the results of our algorithm bycomparing it with reference methods. We utilise a cluster con-taining 16 cores and 128G memories to run our experiments.

A. Dataset and Feature Extraction

We crawled a social network from the big image sharingsite Flickr. As the data set is quite large, a relatively unbiaseddataset was obtained. In total we crawled the data of 30000users, and for each user, we crawled all their photos, and tagsof each photo. In this paper we tried the SIFT feature and thedeep network extracted features through an CNN autocoderrealized by Caffe[36]. For the CNN features we follow the



9

steps of the widely used AlexConvNet [37] and use the 4096dimensional features vectors from the last full-connected layer.In most cases the CNN features performs better than the SIFTfeatures, so we chose the CNN extracted features for the restof our experiments. In the future we can also refine featureextraction method for better performance. We then crawledthe user contact information to form the contact network. Thecontact information in Flickr was acquired by checking if auser added another user to his/her friend list, or vice versa. Wecrawled all the contacts between any two users in our dataset.A short summary of our dataset is given in Table II :

TABLE II: Dataset Statistics

Users 30000Photos 1,356,293 photos from 30000 usersCNN features 4096Contact 628,153 friend links among usersTags 42,739 words after filtering

B. Settings and Metrics

Our task is to make precise contact information prediction.When a new user enters into the social network, we recom-mend new friends according to key words and photos thatrepresent the user’s interests.

In friend recommendation, assume we recommend T friendsto each user. We use the existing contact information as theground truth for training and testing. In the first stage, theparameter µ of Eq. (1) is determined on the training set by afour-fold cross validation to find the best. The range for theparameter is: µ ∈ 10[−2:1:3].

We use the method summarised in Algorithm 1 to rec-ommend friends to new users. We use the recommendationprecision metrics to show the effectiveness of the proposedalgorithm. In our experiment, precision is defined as thenumber of correctly recommended friends divided by all therecommended users. We also introduce the precision-recallcurve to further show the advantage of our algorithm, whererecall is defined as the number of the correctly recommendedfriends divided by the number of all friends.

During our experiments we divide the whole usere setrandomly into two groups: 4/5 of all the users are in thetraining set and the rest are in the test set. The importantfeatures in the first stage are selected on the training set, wherethe parameters in the second stage are also trained. When anew user in the test set comes into the system with someuploaded tags and photos, T friends will be recommendedto him/her from the training set. Assume that together wehave recommended RecAll real friends to the test users(totally 6000 users), then the overall precision is calculatedby RecAll/(6000× T ). We adopt a five-fold cross validationto ensure that all the users are utilised as training and testingdata once

C. Reference Methods

The performance analysis of our first stage: network align-ment methods can be seen in some previous related paperssuch as [21][5]. For the performance analysis of the second

stage in which the topic model method is applied, we chooseseveral widely-used methods for comparison.

The first is the variational method, which has been widelyapplied in this decade for solving the Bayesian networkproblem[10]. Basically we apply the methods in [9] with someslight modifications to our problem.

The second is the widely-used Gibbs sampling method,which is also very popular in dealing with topic model.Compared with the variational method, the idea of Gibbssampling is simpler but usually it has difficulty in dealingwith large scale problems. We apply the method based on [7]for comparison.

The third method is a co-clustering based method [21]. It isnot a topic model-based method, but has a relatively simplerconcept: In the second stage, we do co-clustering of imagefeatures, users and tags to get a . We apply a simple rankingmethod, similar to [21] for the final friend recommendation.

To further check the advantage of our method, we alsocompare our whole two-stage recommendation algorithm withseveral state-of-the-art recommendation systems. The first oneis based on matrix factorization(MF). MF method decomposesthe item-user or user-user matrix to infer the latent factorsthat catch individuals’ interests and has been widely discussedfor different kinds of recommendation prolems[13][14]. Inthis paper we apply a recent method proposed in [14] forcomparison, for it jointly considers the information from twodifferent domains.

Another recent method is based on Bayesian collaborativefiltering that takes the social connections into account, calledSBPR[38]. As a widely-used recommendation method, collab-orative filtering assumes that two users that choose the sameitems behave similar on other items. Traditional collaborativefiltering methods do not consider much about the socialconnections between users. SBPR removes this drawback bytaking the social connections into account by assigning a socialcoefficient to each user. 1.

At last we consider a multi-network based algorithm forcomparison. When considering social multiple network prob-lems, transition probability propagation is a method that is fre-quently used [39][40]. We choose [39] as a reference methodfor the following reasons: 1) It considers the relationships ofdifferent networks, which is similar to our idea; 2) It usesthe information of other networks for recommendation, whichagain has some similarities with ours. [39] enhances the linksin one network and between different networks using a randomwalk propagation method. After a sufficient number of walks,it obtains the modified link weights between each user pair.We use the weights for friend recommendation.

D. Experimental Results

Here we report the results of our method for friend recom-mendation as follows.

1) Performance of Series Expansion: In this experimentwe compare the proposed series expansion method with thevariational, Gibbs sampling, and co-clustering methods in the

1The realization of [14] and [38] is based on the existing open-source Javapackage LibRec at http://www.librec.net/



10

second stage. We treat the performance of the first stage asthe baseline.

P@5 P@10 P@15 P@20 P@2512

14

16

18

20

22

24

Number of Recommendated Users

Pre

cisi

on(%

)

Stage 2 Recommendation Precision Comparison

Stage2: Proposed

Stage1: Baseline

Stage2:Gibbs Sampling[6]

Stage2:variational[8]

Stage2:Co−clustering[17]

Fig. 3: Stage 2 Recommendation Precision Comparison

From Fig. 3 we can see that our method has the best per-formance for accurate recommendation. P@X stands for thateach time we recommend the top X friends to users. Generally,the second stage improves the recommendation precision fromonly applying the first stage, illustrating the effectiveness andnecessity of applying the two staged methods. Our proposedmethod improves about 5-7% compared with the performanceof the first stage, and also makes about 2-3% improvementcompared with the Gibss sampling method and the variationalmethod. The reason for the improvement mainly lies in thatwe apply an exact expression to approach the PDF of thedata, rather than an approximation or sampling method. Theco-clustering method lacks the ranking ability and thus theperformance is not good.

5 6 7 8 9 10 11 12 13 14 1512

14

16

18

20

22

24

26

Recall(%)

Pre

cisi

on(%

)

Stage 2 Comparison of P−R Curve for P@5−25

Stage2:Proposed

Stage1:Baseline

Stage2:Gibbs Sampling[6]

Stage2:variational[8]

Stage2:Co−clustering[17]

Fig. 4: Recommendation Precision and Recall for Stage 2

Fig. 4 illustrates the precision-recall curve of the pro-posed and reference methods. Based on the result of thefirst stage, the series expansion method achieves the highest

performance(The upper right line on the figure). We can seefrom Figure. 4 that when precision or recall is fixed, we canachieve a 3-4% improvement over the best reference methods.This means that the proposed method can achieve both thehighest precision and recall. This experimental results showsthat the series expansion method can best approximate thereal distribution of the data, and thus makes the most preciserecommendation.

On the other hand, the proposed method have also imposedGaussian distribution assumption to the latent variables a, b,and v. This may also cause some negative effect although itcan give an analytic expression. It is worthy to make a depthobservation of the distribution of the latent variables in ourfuture studies.

2) Performance of The Proposed Two-stage Method:Now we compare our two-stage method with some recently-proposed recommendation systems as mentioned in V-C. Themain results for precision and precision-recall curve are shownin Fig. 5 and 6.

P@5 P@10 P@15 P@20 P@2512

14

16

18

20

22

24

Number of Recommended Users

Pre

cisi

on

System Recommendation Precision Comparison

Proposed Two−stage Method

Matrix Factoriazation[35]

Collaborative Filtering[36]

Random Walk[37]

Fig. 5: Two-stage Recommendation Precision Compared withState-of-The-Art Systems

From Fig. 5 and Fig. 6 we can see that our system achievesthe best performance, compared with other state-of-the-artrecommendation systems. In average, our system improves therecommendation accuracy by about 3-4%, compared with thesecond best one. MF based method [14] has the best perfor-mance among all the reference methods, for it decomposes theitem-user and user-user matrix into different social factors ina proper way. The reason that the proposed method performsbetter than MF might lies in that the MF method does notconsider the whole distribution of the network and is trappedinto some local optimum. Collaborative filtering based method[38] has slight lower performance than [14], the reason mightbe that its assumptions about the users’ positive and negativefeedback are not very proper for the Flickr dataset. Finally, therandom-walk based method [39] has the lowest performance,since the random walk algorithm is not accurate enough forprecise friend recommendation.



11

4 6 8 10 12 14 1612

14

16

18

20

22

24

26

Recall(%)

Pre

cisi

on(%

)System Comparison of P−R Curve for P@5−25

Proposed

Matrix Factoriazation[35]

Collaborative Filtering[36]

Random Walk[37]

Fig. 6: Recommendation Precision and Recall Compared withState-of-The-Art Systems

E. The Influence of Several Settings

a) The Influence of Additional Noise: The introduction ofthe additive noise, as shown in Section IV-B, makes the modelmore precise. However, it also leads to complicated inferencesand calculations. In the following experiment we study theinfluence of the additive noise. In Table III, we comparethe recommendation accuracy of the model that contains theadditive noise and the model that does not contain the noise.

TABLE III: The Influence of Additive Noise

Precision(%) P@5 P@10 P@15 P@20 P@25Model With Noise 24.6 21.0 19.8 18.1 17.5Model Without Noise 22.7 19.3 18.2 16.8 15.9

From Table III we see that by considering the additive noisewe get a precision gain of about 1 − 2%, which is useful inthe case where a more precise result is required.

b) The Influence of the Value of Ckj and Iij : As shortlydiscussed in Section IV, the convergence speed of the seriesis largely determined by the level of values of C and I. If it istoo large, then the convergence speed will decrease, leading toeither the inaccuracy of the model, or larger number of terms.On the other hand, if the level is too small, the logarithmicterms in Eq. (12) will drop quickly and make the systemunstable. In our experiments, contact network C stands for theintimacy of two individuals and in the image-user network I,it stands for to what extent an individual favours an imagefeature. The values of each entry of C and I can be setaccording to our requirements. For example, we can set Cjk

to be 1 if two individuals are friends with each other and 0otherwise; for image-user network we can also set Iij = 1 ifan individual has a certain image feature in his/photos, and0 otherwise. On the other hand, we can also raise the levelof the elements in C and I to be 5 or 10, or reduce it to besmaller than 1. The relationship between any two nodes wouldnot change in the networks by varying the element value of Cand I, but the value does have an influence on the accuracy in

our algorithm. We set the value of C and I on four levels tobe 0.3, 1, 5 and 10 to check its influence on the performance.

In the following we compare the recommendation precisionof these four levels.

TABLE IV: The Influence of Values of C and I

y 0.3 1 5 10Precision(%) 19.6 24.6 13.7 11.0

From Table IV we see that the recommendation precisiondecreases rapidly as we increase the value of C and I. Onthe other hand, if it is too small, the performance also goesdown as the system becomes unstable around the poles of thelogarithmic terms in Eq. (12). This indicates that we shouldchoose the value of I and C around 1 for precise calculation.

VI. CONCLUSION AND FUTURE WORKS

In this paper, we develop a two-stage friend recommen-dation scenario utilizing multimedia information. In the firststage, tag information is utilised to build a tag-similaritynetwork and is aligned to a contact network by a numberof important features to generate a “possible friend list”.In the second stage, a topic model is proposed and a newmethod based on series expansion is developed to combineimage features and contact information to make more preciserecommendations.

The experimental results show that the proposed methodoutperforms other methods in friend recommendation in thatour method achieves the highest precision and recall in friendprediction. The network alignment of Stage One is effective.The topic model in Stage Two refines the result of stage oneand the new series expansion method has better performancethan the traditional variational and Gibbs sampling methods.

We will further develop our algorithm. For the series ex-pansion method, it is a novel and effective method but notperfect. It is still to some extent mathematically complicatedand has difficulties to apply on different models. We plan torefine the idea to make it more manoeuvrable and can beapplied on general topic models. There are two directions todig further. Firstly, for more complicated topic models, it mightbe viewed as a combination of some simpler models and thusare solvable based on our method. Secondly, our method isspecially developed for Gaussian distributed random variables.For some other simple distributions, their algebra has beendiscussed in [23][24][27], etc. It is our future work to developsome general frameworks to combine all these distributionstogether.

For our staged recommendation framework, we will extendour ideas to further applications such as product recommenda-tion, media retrieval, etc. One problem of the current methodis that in the first stage, some real friend might be omitted. Wewill further study how to increase the recalls in the first stage.We will develop other algorithms in each of our two stages,and to utilise the information form different domains. We willalso make some studies about the ranks of the informationfrom different domains. That is, which data should be appliedin the first stage to achieve better performance. In the last, wecan also introduce the concept of deep learning in our scenariofor more efficient feature learning.



12

REFERENCES

[1] W. Alex, “Why is it hard to make friends over 30?,” The New YorkTimes, June 2012.

[2] W. Zeng and L. Chen, “Heterogeneous data fusion via matrix factor-ization for augmenting item, group and friend recommendations,” inProceedings of the 28th Annual ACM Symposium on Applied Computing,SAC ’13, (Coimbra, Portugal), pp. 237–244, ACM, 2013.

[3] A. Chin, B. Xu, and H. Wang, “Who should i add as a“friend”?: Astudy of friend recommendations using proximity and homophily,” inProceedings of the 4th International Workshop on Modeling SocialMedia, MSM ’13, (Paris, France), pp. 7:1–7:7, ACM, 2013.

[4] C. Guo, X. Tian, and T. Mei, “User specific friend recommendation insocial media community,” in Multimedia and Expo (ICME), 2014 IEEEInternational Conference on, (Chengdu, China), pp. 1–6, July 2014.

[5] S. Huang, J. Zhang, L. Wang, and X. S. Hua, “Social friend recom-mendation based on multiple network correlation,” IEEE Transactionson Multimedia, vol. 18, pp. 287–299, Feb 2016.

[6] Z. Wang, J. Liao, Q. Cao, H. Qi, and Z. Wang, “Friendbook: Asemantic-based friend recommendation system for social networks,”Mobile Computing, IEEE Transactions on, vol. 14, pp. 538–551, March2015.

[7] W. Min, B.-K. Bao, C. Xu, and M. Hossain, “Cross-platform multi-modal topic modeling for personalized inter-platform recommendation,”Multimedia, IEEE Transactions on, vol. 17, pp. 1787–1801, Oct 2015.

[8] N. Zheng, S. Song, and H. Bao, “A temporal-topic model for friendrecommendations in chinese microblogging systems,” Systems, Man, andCybernetics: Systems, IEEE Transactions on, vol. 45, pp. 1245–1253,Sept 2015.

[9] Y. Song, J. Zhang, and L. Cao, “A joint interest-social latent factor modelfor social recommendation,” Technical report, 2013.

[10] D. M. Blei, A. Y. Ng, and M. I. Jordan, “Latent dirichlet allocation,” J.Mach. Learn. Res., vol. 3, pp. 993–1022, Mar. 2003.

[11] T. L. Griffiths and M. Steyvers, “A probabilistic approach to semanticrepresentation,” Proceedings of the 24th Annual Conference of theCognitive Science Society, pp. 381–386, 2002.

[12] M. Welling and Y. W. Teh, “Hybrid variational/gibbs collapsed inferencein topic models,” in In Proceedings of the Conference on Uncertaintyin Artifical Intelligence (UAI), pp. 587–594, 2008.

[13] H. Ma, H. Yang, M. R. Lyu, and I. King, “Sorec: Social recommendationusing probabilistic matrix factorization,” in Proceedings of the 17th ACMConference on Information and Knowledge Management, CIKM ’08,(New York, NY, USA), pp. 931–940, ACM, 2008.

[14] G. Guo, J. Zhang, and N. Yorke-Smith, “Trustsvd: Collaborative filteringwith both the explicit and implicit influence of user trust and of itemratings,” 2015.

[15] D. Lu, J. Sang, Z. Chen, M. Xu, and T. Mei., “Who are your realfriend: Analyzing and distinguishing between offline and online friend-ships from social multimedia data,” accepted by IEEE Transactions onMultimedia, 2017.

[16] M. Yan, J. Sang, T. Mei, and C. Xu, “Friend transfer: Cold-startfriend recommendation with cross-platform transfer learning of socialknowledge,” in Multimedia and Expo (ICME), 2013 IEEE InternationalConference on, pp. 1–6, July 2013.

[17] A. Chin, B. Xu, and H. Wang, “Who should i add as a ”friend”?:A study of friend recommendations using proximity and homophily,”in Proceedings of the 4th International Workshop on Modeling SocialMedia, MSM ’13, (Paris, France), pp. 7:1–7:7, ACM, 2013.

[18] J. Sang and C. Xu, “Right buddy makes the difference: An earlyexploration of social relation analysis in multimedia applications,” inProceedings of the 20th ACM International Conference on Multimedia,MM ’12, (New York, NY, USA), pp. 19–28, ACM, 2012.

[19] L.-H. Li, F.-M. Lee, Y.-C. Chen, and C.-Y. Cheng, “A multi-stage collab-orative filtering approach for mobile recommendation,” in Proceedingsof the 3rd International Conference on Ubiquitous Information Manage-ment and Communication, ICUIMC ’09, (Suwon, Korea), pp. 88–97,ACM, 2009.

[20] X. Zhao and J. Wang, “A theoretical analysis of two-stage recommen-dation for cold-start collaborative filtering,” in Proceedings of the 2015International Conference on The Theory of Information Retrieval, ICTIR’15, (Northampton, Massachusetts, USA), pp. 71–80, ACM, 2015.

[21] S. Huang, J. Zhang, S. Lu, and X.-S. Hua, “Social friend recom-mendation based on network correlation and feature co-clustering,” inProceedings of the 5th ACM on International Conference on MultimediaRetrieval, ICMR ’15, (Shanghai, China), pp. 315–322, ACM, 2015.

[22] S. Geman and D. Geman, “Stochastic relaxation, gibbs distributions, andthe bayesian restoration of images,” IEEE Trans. Pattern Anal. Mach.Intell., vol. 6, pp. 721–741, Nov. 1984.

[23] B. D. Carter and M. D. Springer, “The distribution of products, quotientsand powers of independent h-fucntion variates,” SIAM Journal onApplied Mathematics, vol. 33, pp. 542–558, Dec. 1977.

[24] M. D. Springer and W. Thompson, “The distribution of products ofbeta, gamma and gaussian random variables,” SIAM Journal on AppliedMathematics, vol. 18, pp. 721–737, June 1977.

[25] J. M. Pruett, “The distribution of products of some independent, non-standardized random variables,” in Ph. D. Dissertation, (University ofArkansas), 1972.

[26] H. Michiel, Mellin Transform. Encyclopedia of Mathematics, Springer,2001.

[27] M. D. Springer, The Algebra of Random Variables. John Wilye & Sons,Inc, 1979.

[28] S. Ahmed, L.-L. Yang, and L. Hanzo, “Probability distributions ofproducts of rayleigh and nakagami-m variables using mellin transform,”in Communications (ICC), 2011 IEEE International Conference on,(Kyoto, Japan), pp. 1–5, June 2011.

[29] R. Mallik and N. Sagias, “Distribution of inner product of complexgaussian random vectors and its applications,” Communications, IEEETransactions on, vol. 59, pp. 3353–3362, December 2011.

[30] H. Bateman, Tables of Integral Transforms, vol. 1. New York: McGraw-Hill, 1954.

[31] L. Ahlfors, Complex Analysis. McGraw Hill, 1979.[32] E.L.Lady, “Convergence of infinite series in general and taylor series in

particular,” October 1998.[33] W. Balser, From Divergent Power Series to Analytic Functions. Springer-

Verlag, 1994.[34] W. A. Richards and M. R. Acharya, “An efficient plynomial approxi-

mation to the normal distribution function and its inverse function,” inJournal of Mathematics Research, 2010.

[35] “Ferrari’s solution of a quartic equation,” in Maths notes.[36] Y. Jia, E. Shelhamer, J. Donahue, S. Karayev, J. Long, R. Girshick,

S. Guadarrama, and T. Darrell, “Caffe: Convolutional architecture forfast feature embedding,” arXiv preprint arXiv:1408.5093, 2014.

[37] A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classificationwith deep convolutional neural networks,” in Advances in Neural Infor-mation Processing Systems 25 (F. Pereira, C. J. C. Burges, L. Bottou,and K. Q. Weinberger, eds.), pp. 1097–1105, Curran Associates, Inc.,2012.

[38] T. Zhao, J. McAuley, and I. King, “Leveraging social connections toimprove personalized ranking for collaborative filtering,” in Proceedingsof the 23rd ACM International Conference on Conference on Informationand Knowledge Management, CIKM ’14, (New York, NY, USA),pp. 261–270, ACM, 2014.

[39] M. Jiang, P. Cui, F. Wang, Q. Yang, W. Zhu, and S. Yang, “Socialrecommendation across multiple relational domains,” in ACM CIKM’12,(Maui, HI, USA), October 2012.

[40] D. Liu, G. Ye, C.-T. Chen, S. Yan, and S.-F. Chang, “Hybrid socialmedia network,” in ACM MM’12, (Nara,Japan), pp. 659–668, October2012.

Shangrong Huang received the B.Sc. degreein telecommunication from Zhejiang University,Hangzhou, China, in 2005, the M. Sc. degree ininformation and communication technology fromDarmstadt University of Technology, Darmstadt,Germany, in 2012, and is currently working towardthe Ph.D. degree at the University of TechnologySydney, Sydney, Austraila.

His research interests include recommendationsystem, social medias and big data analysis.



13

Jian Zhang (SM04) received the B.sc. degree fromEast China Normal University, Shanghai, China, in1982; the M.sc. degree in computer science fromFlinders University, Adelaide, Australia, in 1994;and the Ph.D. degree in electrical engineering fromthe University of New South Wales (UNSW), Syd-ney, Australia, in 1999.

From 1997 to 2003, he was with the VisualInformation Processing Laboratory, Motorola Labs,Sydney, as a Senior Research Engineer, and laterbecame a Principal Research Engineer and a Foun-

dation Manager with the Visual Communications Research Team. From 2004to July 2011, he was a Principal Researcher and a Project Leader withNational ICT Australia, Sydney, and a Conjoint Associate Professor withthe School of Computer Science and Engineering, UNSW. He is currently anAssociate Professor with the Global Big Data Technologies Centre and Schoolof Computing and Communication, Faculty of Engineering and InformationTechnology, University of Technology Sydney, Sydney. He is the author orco-author of more than 100 paper publications, book chapters, and six issuedpatents filed in the U.S. and China. His current research interests includemultimedia processing and communications, image and video processing,machine learning, pattern recognition, media and social media visual informa-tion retrieval and mining, human-computer interaction and intelligent videosurveillance systems.

Dr. Zhang was the General Co-Chair of the International Conference onMultimedia and Expo in 2012 and Technical Program Co-Chair of IEEEVisual Communications and Image Processing 2014. He was AssociatedEditors for the IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMSFOR VIDEO TECHNOLOGY (2006 2015) and the EURASIP Journal onImage and Video Processing (2016 now).

Dan Schonfeld received the B.S. degree in Elec-trical Engineering and Computer Science from theUniversity of California at Berkeley, and the M.S.and Ph.D. degrees in Electrical and Computer Engi-neering from The Johns Hopkins University, in 1986,1988, and 1990, respectively. In 1990, he joined theUniversity of Illinois at Chicago, where he is cur-rently a Professor in the Departments of Electricaland Computer Engineering, Computer Science, andBioengineering.

Dr. Schonfeld has been elevated to the rank ofFellow of the IEEE for contributions to image and video analysis. He wasalso elevated to the rank of Fellow of the SPIE for specific achievementsin morphological image processing and video analysis. Dr. Schonfeld hasbeen elected University Scholar of the University of Illinois and receivedthe Graduate Mentoring Award of the University of Illinois at Chicago. Hehas authored over 200 technical papers in various journals and conferences.He was co-author of a paper that won the Best Paper Award at the ACMMultimedia Workshop on Advanced Video Streaming Techniques for Peer-to-Peer Networks and Social Networking 2010. He was also co-author ofpapers that won the Best Student Paper Awards in Visual Communicationand Image Processing 2006 and IEEE International Conference on ImageProcessing 2006 and 2007.

He is currently serving as Editor-in-Chief of the IEEE Transactions onCircuits and Systems for Video Technology. He has served as DeputyEditor-in-Chief of the IEEE Transactions on Circuits and Systems for VideoTechnology and Area Editor for Special Issues of the IEEE Signal ProcessingMagazine. His current research interests are in signal processing, imageand video analysis, video retrieval and communications, multimedia systems,computer vision, medical imaging, and genomic signal processing.

Lei Wang obtained his Ph.D. from Nanyang Techno-logical University in 2004. He is now with School ofComputing and Information Technology, Universityof Wollongong as Associate Professor.

His research interest lies in machine learning, pat-tern recognition and computer vision. For machinelearning and pattern recognition, he is interestedin feature selection, model selection, and kernel-based learning methods. For computer vision, he isinterested in image categorisation and content-basedimage retrieval. He is IEEE Senior Member.

Xian-Sheng Hua became a Researcher and SeniorDirector of Alibaba Group in April of 2015, lead-ing the multimedia technology team in the SearchDivision. Before that, he was a senior researcher ofMicrosoft Research Redmond since 2013, workedon Web-scale image and video understanding andsearch, as well as related applications. He was aPrincipal Research and Development Lead in Multi-media Search for the Microsoft search engine, Bing,since 2011, where he led a team that designed anddelivered leading-edge media understanding, index-

ing and searching features. He joined Microsoft Research Asia in 2001 asa researcher. Since then, his research interests have been in the areas ofmultimedia search, advertising, understanding, and mining, as well as patternrecognition and machine learning. He has authored or co-authored more than250 research papers in these areas and has filed more than 90 patents.

Dr Hua received his BS in 1996 and PhD in applied mathematics in2001 from Peking University, Beijing. He served or is now serving as anassociate editor of IEEE Transactions on Multimedia and an associate editorof ACM Transactions on Intelligent Systems and Technology. He servedas a program co-chair for IEEE ICME 2013, ACM Multimedia 2012, andIEEE ICME 2012, as well as on the Technical Directions Board of IEEESignal Processing Society. He was honored as one of the recipients of theprestigious 2008 MIT Technology Review TR35 Young Innovator Award forhis outstanding contributions to video search. He won the Best Paper and BestDemonstration Awards at ACM Multimedia 2007, the Best Poster Award atIEEE International Workshop on Multimedia Signal Processing 2008, the BestStudent Paper Award at ACM Conference on Information and KnowledgeManagement 2009, the Best Paper Award at International Conference onMultiMedia Modeling 2010, the best demonstration award at ICME 2014and best paper award of IEEE Trans. On CSVT in 2014.

Two-Stage Friend Recommendation Based on …...friend recommendation method with two stages. In the...

Documents

Transcript of Two-Stage Friend Recommendation Based on …...friend recommendation method with two stages. In the...