The Effect of Dimensionality Reduction in Recommendation Systems Juntae Kim Department of Computer...
-
Upload
alexia-burns -
Category
Documents
-
view
214 -
download
0
Transcript of The Effect of Dimensionality Reduction in Recommendation Systems Juntae Kim Department of Computer...
![Page 1: The Effect of Dimensionality Reduction in Recommendation Systems Juntae Kim Department of Computer Engineering Dongguk University.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649f1f5503460f94c375de/html5/thumbnails/1.jpg)
The Effect of Dimensionality Reduction in Recommendation Systems
Juntae Kim
Department of Computer EngineeringDongguk University
![Page 2: The Effect of Dimensionality Reduction in Recommendation Systems Juntae Kim Department of Computer Engineering Dongguk University.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649f1f5503460f94c375de/html5/thumbnails/2.jpg)
Contents Introduction
Collaborative Recommendation
Data Sparseness Problem
Dimensionality Reduction by using SVD
An Example
Experiments
Conclusion
![Page 3: The Effect of Dimensionality Reduction in Recommendation Systems Juntae Kim Department of Computer Engineering Dongguk University.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649f1f5503460f94c375de/html5/thumbnails/3.jpg)
Introduction e-CRM
Provides personalized service Enhance sales by
Product recommendation, target advertisement, etc.
Recommendation System
Demographic features
Item features
Sales history
Purchase historyCustomer
Recommend items
![Page 4: The Effect of Dimensionality Reduction in Recommendation Systems Juntae Kim Department of Computer Engineering Dongguk University.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649f1f5503460f94c375de/html5/thumbnails/4.jpg)
Introduction Use item-to-item similarity – content-based
Use item-to-item similarity – association
A
C
B
like
similarcontents
Recommend
A
C
B
like
highcorrelation
Recommend
![Page 5: The Effect of Dimensionality Reduction in Recommendation Systems Juntae Kim Department of Computer Engineering Dongguk University.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649f1f5503460f94c375de/html5/thumbnails/5.jpg)
Introduction Use people-to-people similarity – demographic
Use people-to-people similarity – collaborative
A
C
Bsimilarfeature
like
Recommend
A
C
B
A B
highcorrelation
like
like
Recommend
![Page 6: The Effect of Dimensionality Reduction in Recommendation Systems Juntae Kim Department of Computer Engineering Dongguk University.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649f1f5503460f94c375de/html5/thumbnails/6.jpg)
Collaborative Method Advantages
No needs of contents analysis Items that are difficult to analyze contents can be
recommended Ex> Movie, music, …
No needs of user information High precision
Method1. Find out similar users
2. Predict preferences based on similar users preferences
![Page 7: The Effect of Dimensionality Reduction in Recommendation Systems Juntae Kim Department of Computer Engineering Dongguk University.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649f1f5503460f94c375de/html5/thumbnails/7.jpg)
Collaborative Method Computing similarity
Pearson correlation coefficient ( [-1, 1] )
: Rating of user a to item i
Example User a: (1, 8, 9) (-5, +2, +3) User b: (2, 9, 7) (-4, +3, +1) User a is similar to b User c: (9, 3, 3) (+4, -2, -2)
![Page 8: The Effect of Dimensionality Reduction in Recommendation Systems Juntae Kim Department of Computer Engineering Dongguk University.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649f1f5503460f94c375de/html5/thumbnails/8.jpg)
Collaborative Method Prediction of preferences
Weighted sum of similar users’ preferences
: 사용자 a 와 u 의 유사도
Example Average rating of user a: 5 Preferences of user a User b: (2, 8, 8), wa,b = 0.5 = (5, 5, 5) + (-4, 2, 2)*0.5
User c: (4, 4, 7), wa,c = 0.1 + (-1, -1, 2)*0.1
= (2.9, 5.9, 6.2)
![Page 9: The Effect of Dimensionality Reduction in Recommendation Systems Juntae Kim Department of Computer Engineering Dongguk University.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649f1f5503460f94c375de/html5/thumbnails/9.jpg)
Data Sparseness Problem Example data
10000
01000
11000
00001
00110
01101
6
5
4
3
2
1.
user
user
user
user
user
userScreamHolloweenPocahontasKingLionCoMonster
A
![Page 10: The Effect of Dimensionality Reduction in Recommendation Systems Juntae Kim Department of Computer Engineering Dongguk University.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649f1f5503460f94c375de/html5/thumbnails/10.jpg)
Data Sparseness Problem Explicit ratings are not usually available
Available data purchase, click, etc.
0 or 1 Computing correlation is not appropriate
(no negative preference information)
use cosine similarity
ua
uaua rr
rrw
,
![Page 11: The Effect of Dimensionality Reduction in Recommendation Systems Juntae Kim Department of Computer Engineering Dongguk University.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649f1f5503460f94c375de/html5/thumbnails/11.jpg)
Data Sparseness Problem Available data are usually very sparse
Buy 2~3 items among thousands of items Cosine similarity can not be computed
Reduce dimension
10000
01000
11000
00001
00110
01101
A
?
A
![Page 12: The Effect of Dimensionality Reduction in Recommendation Systems Juntae Kim Department of Computer Engineering Dongguk University.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649f1f5503460f94c375de/html5/thumbnails/12.jpg)
Dimensionality Reduction Using category information
Represent user preference vector with item categories Monster Co., Lion King, Pocahontas animation Holloween, Scream horror
10
10
10
01
01
11
A
10000
01000
11000
00001
00110
01101
A
![Page 13: The Effect of Dimensionality Reduction in Recommendation Systems Juntae Kim Department of Computer Engineering Dongguk University.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649f1f5503460f94c375de/html5/thumbnails/13.jpg)
Dimensionality Reduction Singular Value Decomposition (SVD)
Decompose the user-item matrix Amn
Amn = Umm Smn (Vnn)T
S : Diagonal matrix that contains the singular values of A in descending order
U, V : Orthogonal matrices
Rotating the axes of the n-dimensional space 1st axis runs along the direction of largest variation
![Page 14: The Effect of Dimensionality Reduction in Recommendation Systems Juntae Kim Department of Computer Engineering Dongguk University.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649f1f5503460f94c375de/html5/thumbnails/14.jpg)
Dimensionality Reduction SVD example
22.058.033.041.012.0
41.058.012.022.033.0
19.000.020.063.045.0
63.058.045.019.020.0
29.000.075.053.028.0
53.000.028.029.075.0
U
39.000.000.000.000.0
00.000.100.000.000.0
00.000.028.100.000.0
00.000.000.059.100.0
00.000.000.000.016.2
S
09.058.041.065.026.0
16.058.015.035.070.0
61.000.037.051.048.0
73.000.059.033.013.0
25.058.057.030.044.0
TV
![Page 15: The Effect of Dimensionality Reduction in Recommendation Systems Juntae Kim Department of Computer Engineering Dongguk University.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649f1f5503460f94c375de/html5/thumbnails/15.jpg)
Dimensionality Reduction Approximation of A
Select largest k singular values
A’mn = Umk Skk (Vnk)T
Computing user similarity AAT = USVT(USVT)T
= USVTVSTUT
= (US)(US)T
Projection of A into k dimensionA’mn Vnk = Umk Skk
![Page 16: The Effect of Dimensionality Reduction in Recommendation Systems Juntae Kim Department of Computer Engineering Dongguk University.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649f1f5503460f94c375de/html5/thumbnails/16.jpg)
An Example User-item matrix
10000
01000
11000
00001
00110
01101
6
5
4
3
2
1.
user
user
user
user
user
userScreamHolloweenPocahontasKingLionCoMonster
A
![Page 17: The Effect of Dimensionality Reduction in Recommendation Systems Juntae Kim Department of Computer Engineering Dongguk University.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649f1f5503460f94c375de/html5/thumbnails/17.jpg)
An Example Reduction, k = 2
65.026.0
35.071.0
00.197.0
30.004.0
84.060.0
46.062.0
2USVA
10000
01000
11000
00001
00110
01101
A
![Page 18: The Effect of Dimensionality Reduction in Recommendation Systems Juntae Kim Department of Computer Engineering Dongguk University.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649f1f5503460f94c375de/html5/thumbnails/18.jpg)
An Example User-user similarity
00.1
74.000.1
93.094.000.1
87.032.062.000.1
54.016.018.088.000.1
10.074.047.040.078.000.1
))(( TUSUS
![Page 19: The Effect of Dimensionality Reduction in Recommendation Systems Juntae Kim Department of Computer Engineering Dongguk University.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649f1f5503460f94c375de/html5/thumbnails/19.jpg)
An Example User vectors in 2-D space
u6
u4
u5
u3
u2
u1
![Page 20: The Effect of Dimensionality Reduction in Recommendation Systems Juntae Kim Department of Computer Engineering Dongguk University.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649f1f5503460f94c375de/html5/thumbnails/20.jpg)
Experiments Dataset – MovieLens
943 users, 1628 movies, 1~5 rating, 6.4% rated Change ratings to 0/1 3.6% rated
Experiments Compare performance of plain collaborative(CF) and reduce
d dimension(SVD) recommendation CF: 60 neighbor SVD: rank 20
Change sparseness to 2.0%, 1.0%, 0.5%
![Page 21: The Effect of Dimensionality Reduction in Recommendation Systems Juntae Kim Department of Computer Engineering Dongguk University.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649f1f5503460f94c375de/html5/thumbnails/21.jpg)
Experiments Metric
Hit ratio Remove 1 rating from each user test data Recommend 10 items for each user If the test data is in the recommended item hit
Total # of hit
Total # of test data
Result Sparseness 3.6% SVD improves hit ratio by x % Sparseness 0.5% SVD improves hit ratio by x %
Hit ratio =
![Page 22: The Effect of Dimensionality Reduction in Recommendation Systems Juntae Kim Department of Computer Engineering Dongguk University.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649f1f5503460f94c375de/html5/thumbnails/22.jpg)
Experiments Results
0
0.05
0.1
0.15
0.2
0.25
3.6% 2.0% 1.0% 0.5% 0.1%
Sparseness
recall Avg
CF 60NN
SVD Rank20
![Page 23: The Effect of Dimensionality Reduction in Recommendation Systems Juntae Kim Department of Computer Engineering Dongguk University.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649f1f5503460f94c375de/html5/thumbnails/23.jpg)
Conclusion Solve data sparseness problem
Reduce dimension – heuristics Reduce dimension – SVD
Experimental results SVD shows more performance improvement in sparser data
Future research Statistical analysis Combined methods
![Page 24: The Effect of Dimensionality Reduction in Recommendation Systems Juntae Kim Department of Computer Engineering Dongguk University.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649f1f5503460f94c375de/html5/thumbnails/24.jpg)
References Basu, C, Hirsh, H., Cohen, W., “Recommeder Systems. Recommedation As Classification: Using Social And C
onent-Based Information,” Proceedings of the Workshop on Recommendation system. AAAI Press, Menlo Park California, 1998.
Billsus, D., Pazzani, M. j., “Learning Collaborative Information Filters,” Proceedings of workshop on recommender system, 1998.
Berry, M. W., Dumais, S. T., and O’Brain, G. W. “Using Linear Algebra for Intelligent Information Retrieval,” SIAM Review, 37(4), pp. 573-595, 1995.
Breese, J. S., Heckerman, D., and Kadie, C., “Empirical Analysis of Predictive Algorithm for Collaborative Filtering,”Proceeding of the Fourteenth Conference UAI, July 1998.
Goldberg, k., Roeder, T., Gupta, D., and Perkins, C., “Eigentaste: A Constant Time Collaborative Filtering Algorithm,” Technical Report M00/41. Electronics Research Laborotary, University of California, Berkeley, 2000.
Herlocker, J., Konstan, J., Borchers, A., Riedl, J., “An Algorithmic Framework for Performing Collaborative Filtering,”Proceedings of the 1999 Conference on Research and Development in Information Retrieval, Aug. 1999.
Sarwar, B. M. “Sparsity, Scalability, and Distribution in Recommender Systems,” Ph.D. Thesis, Computer Science Dept., University of Minnesota, 2001.
Sarwar, B. M., Karypis, G., Konstan, J. A., and Riedl, J., “Application of Dimensionality Reduction in Recommender System-A Case Study,”WebKDD 00-Web-mining for E-Commerce Workshop, 2000.
Schafer, J. B., Konstan, J., and Riedl, J., “Recommender Systems in E-Commerce ,” Proceedings of the ACM Conference on Electronic Commerce, November 1999.
Shardanand, U., "Social information filtering for music recommendation," Technical Report MA95, MIT Media Laboratory, 1995.