Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM...
Transcript of Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM...
![Page 1: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/1.jpg)
Recommender Systems: The Power of Personalization
Presenter Moderator Dr. Joseph A. Konstan Dr. Gary M. Olson University of Minnesota University California, Irvine [email protected] [email protected]
![Page 2: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/2.jpg)
ACM Learning Center (http://learning.acm.org)
• 1,300+ trusted technical books and videos by leading publishers including O’Reilly, Morgan Kaufmann, others
• Online courses with assessments and certification-track mentoring, member discounts at partner institutions
• Learning Webinars on big topics (Cloud Computing/Mobile Development, Cybersecurity, Big Data)
• ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by subject experts
• Learning Paths (accessible entry points into popular languages)
• Popular video tutorials/keynotes from ACM Digital Library, Podcasts with industry leaders/award winners
![Page 3: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/3.jpg)
![Page 4: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/4.jpg)
A Bit of History
• Ants, Cavemen, and Early Recommender Systems – The emergence of critics
• Information Retrieval and Filtering • Manual Collaborative Filtering • Automated Collaborative Filtering • The Commercial Era
![Page 5: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/5.jpg)
A Bit of History
• Ants, Cavemen, and Early Recommender Systems – The emergence of critics
• Information Retrieval and Filtering • Manual Collaborative Filtering • Automated Collaborative Filtering • The Commercial Era
![Page 6: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/6.jpg)
Information Retrieval
• Static content base – Invest time in indexing content
• Dynamic information need – Queries presented in “real time”
• Common approach: TFIDF
term frequency inverse document frequency – Rank documents by term overlap – Rank terms by frequency
![Page 7: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/7.jpg)
![Page 8: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/8.jpg)
![Page 9: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/9.jpg)
Information Filtering
• Reverse assumptions from IR – Static information need – Dynamic content base
• Invest effort in modeling user need – Hand-created “profile” – Machine learned profile – Feedback/updates
• Pass new content through filters
![Page 10: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/10.jpg)
![Page 11: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/11.jpg)
A Bit of History
• Ants, Cavemen, and Early Recommender Systems – The emergence of critics
• Information Retrieval and Filtering • Manual Collaborative Filtering • Automated Collaborative Filtering • The Commercial Era
![Page 12: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/12.jpg)
Collaborative Filtering
• Premise – Information needs more complex than keywords or topics:
quality and taste • Small Community: Manual
– Tapestry – database of content & comments – Active CF – easy mechanisms for forwarding content to
relevant readers
![Page 13: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/13.jpg)
A Bit of History
• Ants, Cavemen, and Early Recommender Systems – The emergence of critics
• Information Retrieval and Filtering • Manual Collaborative Filtering • Automated Collaborative Filtering • The Commercial Era
![Page 14: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/14.jpg)
Automated CF
• The GroupLens Project (CSCW ’94) – ACF for Usenet News
• users rate items • users are correlated with other users • personal predictions for unrated items
– Nearest-Neighbor Approach • find people with history of agreement • assume stable tastes
![Page 15: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/15.jpg)
Usenet Interface
![Page 16: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/16.jpg)
Does it Work?
• Yes: The numbers don’t lie! – Usenet trial: rating/prediction correlation
• rec.humor: 0.62 (personalized) vs. 0.49 (avg.) • comp.os.linux.system: 0.55 (pers.) vs. 0.41 (avg.) • rec.food.recipes: 0.33 (pers.) vs. 0.05 (avg.)
– Significantly more accurate than predicting average or modal rating.
– Higher accuracy when partitioned by newsgroup
![Page 17: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/17.jpg)
It Works Meaningfully Well!
• Relationship with User Behavior – Twice as likely to read 4/5 than 1/2/3
• Users Like GroupLens
– Some users stayed 12 months after the trial!
![Page 18: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/18.jpg)
A Bit of History
• Ants, Cavemen, and Early Recommender Systems – The emergence of critics
• Information Retrieval and Filtering • Manual Collaborative Filtering • Automated Collaborative Filtering • The Commercial Era
![Page 19: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/19.jpg)
![Page 20: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/20.jpg)
![Page 21: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/21.jpg)
Amazon.com
![Page 22: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/22.jpg)
![Page 23: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/23.jpg)
Recommenders
• Tools to help identify worthwhile stuff – Filtering interfaces
• E-mail filters, clipping services – Recommendation interfaces
• Suggestion lists, “top-n,” offers and promotions – Prediction interfaces
• Evaluate candidates, predicted ratings
![Page 24: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/24.jpg)
Historical Challenges
• Collecting Opinion and Experience Data
• Finding the Relevant Data for a Purpose
• Presenting the Data in a Useful Way
![Page 25: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/25.jpg)
Recommender Application Space
![Page 26: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/26.jpg)
Scope of Recommenders
• Purely Editorial Recommenders
• Content Filtering Recommenders
• Collaborative Filtering Recommenders
• Hybrid Recommenders
![Page 27: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/27.jpg)
Recommender Application Space
• Dimensions of Analysis – Domain – Purpose – Whose Opinion – Personalization Level – Privacy and Trustworthiness – Interfaces – <Algorithms Inside>
![Page 28: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/28.jpg)
Domains of Recommendation
• Content to Commerce – News, information, “text” – Products, vendors, bundles
![Page 29: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/29.jpg)
Google: Content Example
![Page 30: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/30.jpg)
C H
![Page 31: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/31.jpg)
Purposes of Recommendation
• The recommendations themselves – Sales – Information
• Education of user/customer
• Build a community of users/customers around products or
content
![Page 32: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/32.jpg)
Buy.com customers also bought
![Page 33: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/33.jpg)
Epinions Sienna overview
![Page 34: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/34.jpg)
OWL Tips
![Page 35: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/35.jpg)
ReferralWeb
![Page 36: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/36.jpg)
Whose Opinion?
• “Experts”
• Ordinary “phoaks”
• People like you
![Page 37: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/37.jpg)
Wine.com Expert recommendations
![Page 38: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/38.jpg)
PHOAKS
![Page 39: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/39.jpg)
Personalization Level
• Generic – Everyone receives same recommendations
• Demographic – Matches a target group
• Ephemeral – Matches current activity
• Persistent – Matches long-term interests
![Page 40: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/40.jpg)
Lands’ End
![Page 41: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/41.jpg)
Brooks Brothers
![Page 42: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/42.jpg)
Amazon.com
![Page 43: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/43.jpg)
Cdnow album advisor
![Page 44: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/44.jpg)
CDNow Album advisor recommendations
![Page 45: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/45.jpg)
![Page 46: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/46.jpg)
Privacy and Trustworthiness
• Who knows what about me? – Personal information revealed – Identity – Deniability of preferences
• Is the recommendation honest? – Biases built-in by operator
• “business rules” – Vulnerability to external manipulation
![Page 47: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/47.jpg)
Interfaces
• Types of Output – Predictions – Recommendations – Filtering – Organic vs. explicit presentation
• Agent/Discussion Interface Example
• Types of Input – Explicit – Implicit
![Page 48: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/48.jpg)
Wide Range of Algorithms
• Simple Keyword Vector Matches
• Pure Nearest-Neighbor Collaborative Filtering
• Machine Learning on Content or Ratings
![Page 49: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/49.jpg)
Collaborative Filtering:
Techniques and Issues
![Page 50: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/50.jpg)
Collaborative Filtering Algorithms
• Non-Personalized Summary Statistics • K-Nearest Neighbor • Dimensionality Reduction • Content + Collaborative Filtering • Graph Techniques • Clustering • Classifier Learning
![Page 51: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/51.jpg)
Teaming Up to Find Cheap Travel
• Expedia.com – “data it gathers anyway”
– (Mostly) no cost to helper – Valuable information that is otherwise hard to acquire – Little processing, lots of collaboration
![Page 52: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/52.jpg)
Expedia Fare Compare #1
![Page 53: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/53.jpg)
Expedia Fare Compare #2
![Page 54: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/54.jpg)
Zagat Guide Amsterdam Overview
![Page 55: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/55.jpg)
Zagat Guide Detail
![Page 56: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/56.jpg)
Zagat: Is Non-Personalized Good Enough?
• What happened to my favorite guide? – They let you rate the restaurants!
• What should be done?
– Personalized guides, from the people who “know good restaurants!”
![Page 57: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/57.jpg)
Collaborative Filtering Algorithms
• Non-Personalized Summary Statistics • K-Nearest Neighbor
– user-user – item-item
• Dimensionality Reduction • Content + Collaborative Filtering • Graph Techniques • Clustering • Classifier Learning
![Page 58: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/58.jpg)
CF Classic: K-Nearest Neighbor User-User
C.F. Engine
Ratings Correlations
![Page 59: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/59.jpg)
CF Classic: Submit Ratings
C.F. Engine
Ratings Correlations
ratings
![Page 60: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/60.jpg)
CF Classic: Store Ratings
C.F. Engine
Ratings Correlations
ratings
![Page 61: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/61.jpg)
CF Classic: Compute Correlations
C.F. Engine
Ratings Correlations
pairwise corr.
![Page 62: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/62.jpg)
CF Classic: Request Recommendations
C.F. Engine
Ratings Correlations
request
![Page 63: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/63.jpg)
CF Classic: Identify Neighbors
C.F. Engine
Ratings Correlations
find good …
Neighborhood
![Page 64: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/64.jpg)
CF Classic: Select Items; Predict Ratings
C.F. Engine
Ratings Correlations Neighborhood
predictions recommendations
![Page 65: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/65.jpg)
Understanding the Computation
Hoop Dreams
Star Wars
Pretty Woman
Titanic Blimp Rocky XV
Joe D A B D ? ? John A F D F Susan A A A A A A Pat D A C Jean A C A C A Ben F A F Nathan D A A
![Page 66: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/66.jpg)
Hoop Dreams
Star Wars
Pretty Woman
Titanic Blimp Rocky XV
Joe D A B D ? ? John A F D F Susan A A A A A A Pat D A C Jean A C A C A Ben F A F Nathan D A A
Understanding the Computation
![Page 67: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/67.jpg)
Hoop Dreams
Star Wars
Pretty Woman
Titanic Blimp Rocky XV
Joe D A B D ? ? John A F D F Susan A A A A A A Pat D A C Jean A C A C A Ben F A F Nathan D A A
Understanding the Computation
![Page 68: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/68.jpg)
Hoop Dreams
Star Wars
Pretty Woman
Titanic Blimp Rocky XV
Joe D A B D ? ? John A F D F Susan A A A A A A Pat D A C Jean A C A C A Ben F A F Nathan D A A
Understanding the Computation
![Page 69: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/69.jpg)
Hoop Dreams
Star Wars
Pretty Woman
Titanic Blimp Rocky XV
Joe D A B D ? ? John A F D F Susan A A A A A A Pat D A C Jean A C A C A Ben F A F Nathan D A A
Understanding the Computation
![Page 70: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/70.jpg)
Hoop Dreams
Star Wars
Pretty Woman
Titanic Blimp Rocky XV
Joe D A B D ? ? John A F D F Susan A A A A A A Pat D A C Jean A C A C A Ben F A F Nathan D A A
Understanding the Computation
![Page 71: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/71.jpg)
Hoop Dreams
Star Wars
Pretty Woman
Titanic Blimp Rocky XV
Joe D A B D ? ? John A F D F Susan A A A A A A Pat D A C Jean A C A C A Ben F A F Nathan D A A
Understanding the Computation
![Page 72: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/72.jpg)
ML-home
![Page 73: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/73.jpg)
ML-scifi-search
![Page 74: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/74.jpg)
ML-clist
![Page 75: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/75.jpg)
ML-rate
![Page 76: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/76.jpg)
ML-search
![Page 77: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/77.jpg)
ML-buddies
![Page 78: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/78.jpg)
User-User Collaborative Filtering
?
Target Customer
Weighted Sum
3
![Page 79: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/79.jpg)
A Challenge: Sparsity
• Many E-commerce and content applications have many more customers than products
• Many customers have no relationship • Most products have some relationship
![Page 80: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/80.jpg)
Another challenge: Synonymy
– Similar products treated differently • Have skim milk? Want whole milk too?
– Increases apparent sparsity – Results in poor quality
![Page 81: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/81.jpg)
Item-Item Collaborative Filtering
I
I I
I I I I I
I
I
I
I
I
I
I
I
I
![Page 82: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/82.jpg)
Item-Item Collaborative Filtering
I
I I
I I I I I
I
I
I
I
I
I
I
I
I
![Page 83: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/83.jpg)
Item-Item Collaborative Filtering
I
I I
I I I I I
I
I
I
I
I
I
I
I
I
![Page 84: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/84.jpg)
Item Similarities
1 2 3 i n-1 n 12
u
mm-1
j
R-
R -
R R
R R
R R
si,j=?
Used for similarity computation
![Page 85: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/85.jpg)
1 2
u
m
2nd 1st 3rd 5th 4th
5 closest neighbors
R R R R -
u R R R Ri1 2 3 i-1 m-1 m
si,1
si,3
si,i-1
si,m
-
pred
ictio
n
weighted sum regression-based
Raw scores for prediction generation
Approximation based on linear regression
Target item
Item-Item Matrix Formulation
![Page 86: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/86.jpg)
Item-Item Discussion
• Good quality, in sparse situations • Promising for incremental model building
– Small quality degradation • Nature of recommendations changes
– Big performance gain
![Page 87: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/87.jpg)
Collaborative Filtering Algorithms
• Non-Personalized Summary Statistics • K-Nearest Neighbor • Dimensionality Reduction
– Singular Value Decomposition – Factor Analysis
• Content + Collaborative Filtering • Graph Techniques • Clustering • Classifier Learning
![Page 88: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/88.jpg)
Dimensionality Reduction
• Latent Semantic Indexing – Used by the IR community – Worked well with the vector space model – Used Singular Value Decomposition (SVD)
• Main Idea – Term-document matching in feature space – Captures latent association – Reduced space is less noisy
![Page 89: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/89.jpg)
SVD: Mathematical Background
=
R
m X n
U
m X r
S
r X r
V’
r X n
Sk
k X k
Uk
m X k
Vk’
k X n
The reconstructed matrix Rk = Uk.Sk.Vk’ is the closest rank-k matrix to the original matrix R.
Rk
![Page 90: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/90.jpg)
SVD for Collaborative Filtering
. 2. Direct Prediction
m x n
1. Low dimensional representation O(m+n) storage requirement
m x k
k x n
![Page 91: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/91.jpg)
Singular Value Decomposition
Reduce dimensionality of problem – Results in small, fast model – Richer Neighbor Network
Incremental Update – Folding in – Model Update
Trend – Towards use of probabilistic LSI
![Page 92: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/92.jpg)
Collaborative Filtering Algorithms
• Non-Personalized Summary Statistics • K-Nearest Neighbor • Dimensionality Reduction • Content + Collaborative Filtering • Graph Techniques
– Horting: Navigate Similarity Graph • Clustering • Classifier Learning
– Rule-Induction Learning – Bayesian Belief Networks
![Page 93: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/93.jpg)
Resources
• Survey Articles – Recommender Systems: From Algorithms to User Experience
(2012): http://www.grouplens.org/node/480 – Collaborative Filtering Recommender Systems (2011):
http://www.grouplens.org/node/475 • Books
– Recommender Systems: An Introduction (2010) buy Jannach et al.
– Recommender Systems Handbook (2010) by Ricci et al. • Software Tools
– LensKit – http://lenskit.grouplens.org – MyMedia – http://www.mymediaproject.org – Mahout – http://mahout.apache.org
![Page 94: Recommender Systems: The Power of Personalization · Development, Cybersecurity, Big Data) • ACM Tech Packs on big current computing topics: Annotated Bibliographies compiled by](https://reader033.fdocuments.in/reader033/viewer/2022053008/5f0bfd0c7e708231d433350f/html5/thumbnails/94.jpg)
ACM: The Learning Continues
• Questions about this webinar? [email protected]
• ACM Learning Center: http://learning.acm.org
• ACM SIGCHI: http://www.sigchi.org
• ACM Conference on Recommender Systems http://recsys.acm.org