Factored Proximity Models for Top-NRecommendationsAthanasios N. Nikolakopoulos1, Vassilis Kalantzis2, EfstratiosGallopoulos3 and John D. Garofalakis3

1Digital Technology Center, University of Minnesota, USA.2Dept. of Computer Science & Engineering, University of Minnesota, USA.3Dept. of Computer Engineering & Informatics, University of Patras, Greece.August 08, 2017

IEEE International Conference on Big Knowledge, IEEE ICBK 2017

Introduction & Motivation i

Recommender Systems• Widely Applicable

Technology• Value for Customers• Value for Companies

Collaborative Filtering• Model: Ratings!• Recommendation task

• Rating Prediction• Top-N Lists

1

Introduction & Motivation ii

Sparsity Problem• Limits the quality recommendations;

especially for Long-Tail Items.• Intrinsic RS Characteristic

• Cold-Start Problem

Promising Approaches• Graph-based Models

(+) Good Performance(-) Scalability Issues

• Latent-Factor Models.Our Focus: Efficient and High-QualityTop-N Recommendations Even underExtreme Sparsity

Figure 1: Graph-Based Idea

Figure 2: Latent-Factor Idea2

EigenRec Framework

EigenRec Framework

EigenRec• Build a symmetric m×m Inter-Item Proximity Matrix A, each

element of which is defined to be a product of a Scaling and aSimilarity component.

[A]ij ≜ ξ(i, j) · κ(i, j)

• Build a Lower Dimensional Model using the principal eigenvectorsof A

• Project the Users’s feedback vectors onto the Latent Subspace:

Π ≜ RVfV⊺f

Simple Baseline Choices

Scaling function Similarity functions

ξ(i, j) ≜ f(i, j; d) = (∥ri∥∥rj∥)d κ(i, j) ≜

cos(vi, vj)

pc(vi, vj)

jaccard(vi, vj) 3

PureSVD within EigenRec

PureSVD

ΠPureSVD ≜ UfΣfQ⊺f ≡ · · · ≡ RQfQ

⊺f

Where Qf , the matrix containing the f principal eigenvectors of:

R⊺R =

users

items – r⊺vi–

×

items|

users rvj

|

=

items

items ·

∥rvi∥∥rvj

∥︸ ︷︷ ︸scaling

·cos θij︸ ︷︷ ︸similarity

,

• PureSVD ≡ EigenRec with Cosine similarity and f(i, j; 1)

4

Computing EigenRec

EigenRec:Input: Inter-Item proximity matrix A ∈ Rm×m. RatingMatrix R ∈ Rn×m. Latent Factors f .Output: Matrix Π ∈ Rn×m whose rows are the recom-mendation vectors for every user.1: qj = 0, set r← q as a random vector2: β0 ← ∥r∥23: for j ← 1, 2, ..., do4: qj ← r/βj−1

5: r← Aqj

6: r← r− qj−1βj−1

7: αj ← q⊺j r

8: r← r− qjαj

9: r← (I−QjQ⊺j )r,

10: βj ← ∥r∥211: Solve the tridiag problem (Q⊺

j AQj)Ξj = ΘjΞj

12: Form the j approximate eigenvectors QjΞj of A13: If the f top eigenvectors have converged, stop.14: end for15: Compute latent factors V = QfΞ

16: return Π← RVV⊺

Computational Aspects:

• The MV product in j Lanczossteps is O(j · nnz)

• Making the j-th vectororthogonal to the previous onescosts O(jm)

Parallel Implementation:

MovieLens20M

f=50 100 150 200 300

8 cores 11.0 18.0 24.2 28.7 36.816 cores 6.9 11.6 15.6 18.9 24.332 cores 4.8 8.1 11.1 13.5 17.664 cores 3.5 6.0 8.2 9.9 12.5

EigenRec

The Code is available here:https://github.com/nikolakopoulos/EigenRec

5

Qualitative Evaluation

Effects of Prior Popularity

Methodology• Randomly sample 1.4% of the ratings of

the dataset ⇒ probe set P• Use each item vj , rated with 5 stars by

user ui in P ⇒ test set T• Randomly select another 1000 unrated

items of the same user for each item in T• Form ranked lists by ordering all the 1001

items

Metrics• Recall• Precision• R-Score• NDCG@k• MRR

−2 −1 0 1 2

0.1

0.2

0.3

Yaho

o

Cosine

PureSVD

−2 −1 0 1 2

0.1

0.2

0.3

Pearson

−2 −1 0 1 2

0.1

0.2

0.3

Jaccard

−2 −1 0 1 20

0.10.20.3

d

ML1M

PureSVD

−2 −1 0 1 20

0.10.20.3

d

−2 −1 0 1 20

0.10.20.3

d

6

Standard Top-N Recommendations

Methodology• Randomly sample 1.4% of the ratings of

the dataset ⇒ probe set P• Use each item vj , rated with 5 stars by

user ui in P ⇒ test set T• Randomly select another 1000 unrated

items of the same user for each item in T• Form ranked lists by ordering all the 1001

items

Metrics• Recall• Precision• R-Score• NDCG@k• MRR

5 10 15 200

0.2

0.4

0.6

Yaho

o

Recall

0.2 0.4 0.6 0.8 10

0.1

0.2

Precision vs Recall

5 10 15 200

0.2

0.4

0.6

NDCG@N

5 10 15 200

0.2

0.4

0.6

RScore(α)

5 10 15 200

0.2

0.4

0.6

ML1M

0.2 0.4 0.6 0.8 10

0.1

0.2

5 10 15 200

0.2

0.4

5 10 15 200

0.2

0.4

0.6

EigenRec RCT MD RED L† MFA7

Long-Tail Recommendations

Methodology• We order the items according to their

popularity (measured in terms of numberof ratings)

• We further partition the test set T intotwo subsets, Thead and Ttail

• We discard the popular items and weevaluate EigenRec and the otheralgorithms on the Ttail test set, using theprocedure explained previously.

Metrics• Recall• Precision• R-Score• NDCG@k• MRR

5 10 15 200

0.2

0.4

0.6

Yaho

o

Recall

0.2 0.4 0.6 0.8 10

0.1

0.2

Precision vs Recall

5 10 15 200

0.2

0.4

NDCG@N

5 10 15 200

0.2

0.4

0.6

RScore(α)

5 10 15 200

0.2

0.4

ML1M

0.2 0.4 0.6 0.8 10

.05

0.1

5 10 15 200

0.1

0.2

0.3

5 10 15 200

0.2

0.4

EigenRec RCT MD RED L† MFA8

Cold-Start Recommendations I

Cold-Start Problem

• Difficulty of making reliable recommendations due to an initiallack of ratings

• In beginning stages, when there is not sufficient number of ratingsfor the collaborative filtering algorithms to uncover similarities ⇒New Community Problem

• Introduction of new users to an existing system where they have notrated many items ⇒ New Users Problem

9

Cold-Start Recommendations II

New Community:• Methodology: Randomly select to include 33%, 66%, and 100% of the overall ratings on

two new artificially sparsified versions of the dataset.

33% 66% 100% 33% 66% 100%

0.1

0.2

0.3

MRR

EigenRec RCT MD RED L† MFA

Recommending All Items Recommending Long Tail Items

New Users:• Methodology: Randomly select 50 users having rated at least 100 items and randomly

delete 95% of each users’ ratings.

5 10 15 200

0.1

0.2

0.3

NewU

sers

Full Recall

0.2 0.4 0.6 0.8 10

.02

.04

.06

.08

Precision vs Recall

5 10 15 200

0.1

0.2

NDCG@N

5 10 15 200

0.1

0.2

0.3

RScore(α)

EigenRec RCT MD RED L† MFA10

Conclusions and Future Work

Conclusions and Future Work

EigenRec• Computationally Efficient framework for Top-N Recommendations• Allows for flexible modeling and control of the effects of prior

popularity• Natural generalization of PureSVD

• (+) Optimize its Top-N recommendation performance• (+) Alleviate its inherent popularity bias• (+) Compute it more efficiently

• Good Top-N Recommendation Performance

Future Directions• Explore more elaborate Similarity and Scaling functions• Explore the Hierarchical structure of the Itemspace

11

References

A. N. Nikolakopoulos, V. Kalantzis, E. Gallopoulos, and J. Garofalakis.EigenRec: Generalizing PureSVD for Effective and Efficient Top-NRecommendations.Knowledge and Information Systems, 2018.A. N. Nikolakopoulos, V. Kalantzis, E. Gallopoulos, and J. Garofalakis.Factored Proximity Models for Top-N Recommendations.In 2017 IEEE International Conference on Big Knowledge (ICBK), pp. 80-87,Hefei, 2017.P. Cremonesi, Y. Koren, and R. Turrin.Performance of recommender algorithms on top-n recommendation tasks.In Proceedings of the fourth ACM conference on Recommender systems, RecSys’10, pages 39–46. ACM, 2010.F. Fouss, K. Francoisse, L. Yen, A. Pirotte, and M. Saerens.An experimental investigation of kernels on graphs for collaborativerecommendation and semisupervised classification.Neural Netw., 31:53–72, July 2012.

12

Thank you for your Attention!

12

Questions?

12