Context Recommendation

Using Multi-label Classification

Yong Zheng, Bamshad Mobasher, Robin Burke

Center for Web Intelligence, DePaul University, Chicago

IEEE/WIC/ACM Conference on Web Intelligence

Aug 14, Warsaw, Poland

Intro – Recommender Systems

• Information Overload Problem IR and RS

• Recommender systems (RS) are the systems being able to provide recommendations to the end users.

Currently, RS are popular everywhere:

• E-Commerce: Amazon, Ebay, Newegg, etc• Social networks: Twitter, Facebook, etc• Movie/Stream: Netflix, Movie Pilot, Youtube, etc• Music: Pandora, Last.FM, etc

Intro – Type of Recommendations• [Item Recommendations]

• [User Recommendations]

Intro – Context-aware Recommender• Context-aware Recommender Systems (CARS)

CARS is a new type of RS which provide recommendations by adapting to users’ contextual situations.

• Traditional RS: Users × Items Ratings

• Contextual RS: Users × Items × Contexts Ratings

• Assumptions behind:

1). User may have different preferences in different contexts;

2). Contexts are important in decision-makings.

However, what CARS recommended are still items or users.

Companion

New Application: Context Recommender

• In this paper, we propose a new application: context recommender (CR), which is able to recommend or suggest appropriate contexts for users to select or consume items.

• Sample of contexts: time, location, companion, etc

Context Rec

User RecItem Rec

Possible Examples (Amazon.com)

Note: They are not REAL applications/examples right now.

Possible Examples (IMDB.com)

Related Work

L. Baltrunas, et al. "Best usage context predictions for music tracks", The 2nd Workshop on Context-aware Recommender Systems, ACM RecSys, 2010

This is the only work related to context recommendation, where the authors tried to provide suggestions of appropriate contexts to listen to music tracks.

Pros: they proposed three KNN-based classifiers to suggest appropriate contexts;

Cons: they proposed a specific application, but not a general delineation of the problem of context recommendations.

Contributions: Context Recommender

• We formally provide the definition of CR

• We propose the formal framework of CR applications

• We discuss the algorithmic paradigms for CR

• We examine the algorithms using multi-label classifications

Context Recommender

1. Definitions

Context recommenders are the systems being able to recommend or suggest appropriate contexts for users to select or consume items.

Examples:When is the best season to travel to Poland for user Tom?Who is the suggested companion to see “Titanic” with Tom?This book is better to be gifted to Mom or Kids?

2. Research Problems

How to infer the appropriate contexts?And those contexts should be personalized or not?

Example:The best season to Poland is always the same for all the users, or it could be personalized for different specific users? E.g. Tom likes winter much more than summer, where most users prefer summers in Poland.

Context Recommender

3. Context Recommendations App

• There could be many other types, for example, it could be a group of users or items, instead of a single user or item. E.g. what is the best season to Poland by this group of travelers (e.g. Tour Group, etc), where the suggested contexts should meet the requirement of the group of users, instead of a single user.

• In this paper, we focus on the general form:{User, Item} Contexts; a pair of <user, item> as input

Input Output App

{User, Item} Contexts The best season to Poland for Tom

{User} Contexts The best travel season for Tom

{Item} Contexts The best travel season to Poland

Context Recommender

3. Algorithmic ParadigmsWhat are the possible algorithms to recommend contexts?

After analysis, we propose two series of frameworks:

1). Direct Context Prediction

We infer suggested contexts by <user, item, preferences>;In other words, contexts are predicted based on users’ previous preference histories associated with <item, contexts>

2). Indirect Context Recommendation

We reuse the context-aware recommendation algorithms:In CARS,Therefore, we vary the choices of contexts, and finally recommend the contexts which can contribute to the best ratings user will give to the item.

Context Recommender

3. Algorithmic ParadigmsWhich algorithms can be applied in each category?

1). Direct Context Prediction

Classification algorithms are the popular ones which are applied to this category, where they have been adopted in context predictions in the pervasive computing area.

2). Indirect Context Recommendation

We reuse the context-aware recommendation algorithms;Therefore, all available CARS algorithms can be applied to. However, the drawback of this category is the high computational cost if there are too many contextual conditions in the data. And it also relies on how the CARS algorithms perform.

In this paper, we focus on Direct Context Prediction using multi-label classification algorithms.

Context Recommender

4. Why Multi-label Classifications (MLC)?

1).Binary Classification (is this an apple?)2).Multi-class Classification (is it an apple|orange|pear?)3).Multi-label Classification (<round, apple, fruit, Mac>)

In other words, MLC allows the system to select more than 1 labels from the set.

Classification is used for context predictions;and MLC just fits the requirement of context recommendation task.

Context Recommender

4. Why Multi-label Classifications (MLC)?

Two series of MLC algorithms:

1). Transformation AlgorithmsThey can use traditional classification algorithms (e.g. decision trees, SVM, etc), and they transform the MLC task to multiple binary or multi-class classification tasks. So they do not need to develop new algorithms.

E.g., binary relevance (BR), label powerset (LP), classifier chains (CC), k-labelsets (RAkEL)

2). Adaptation AlgorithmsDevelop new classification algorithms to adapt to the MLC task. E.g., Binary relevance KNN (BRKNN), Multi-label KNN (MLKNN)

Context Recommender

5. Experiments and Evaluations (Algorithms)

Toolkit: Mulan (MLC toolkit) and Weka Java-based library

MLC algorithms: BR, LP, CC, RAkEL, MLKNN, BRKNN;

Classification methods used in BR, LP, CC and RAkEL:KNNclassifier (KNN), decision trees (J48), naive bayes(NB),Bayesian nets (BN) and support vector machine (SMO)

Baseline algorithms: 1).the three KNN classifiers: RatingBased (RB), BestContextVectorBased (BCVB) and BestContextBased(BCB) proposed by L. Baltrunas, et al2).Non-personalized methods: such as most popular algs

Data # of users # of items # of ratings # of labels Rating scale

AdomMovie 69 176 1,010 8 1-13

LDOS 113 1186 2,094 17 1-5

TripAdvisor 2731 2269 14,175 5 1-5

Context Recommender

5. Experiments and Evaluations (Metrics)

Inputs: User, Item, Binary Preference (Good or Bad)Outputs: A list of predicted contexts

Time =

Weekend

Time =

Weekday

Companion

= Kids

Companion

= Parents

Companion

= Girlfriend

Real 0 1 0 0 1

Prediction 1 0 0 0 1

Y is the set of TRUE labels in the ground truth, and Z is the set of predicted TRUE labels. m = # of examples.

Another metric is hamming loss which measures the average percentage of incorrectly predicted labels.

Context Recommender

6. Experimental ResultsDue to limited time, we only present results on LDOS data.

Context Recommender

6. Experimental Results (Optional)We also have further experiments and discussions about other insights of those algorithms:

1). We compare running performance between RAkEL and the LP approach. In traditional MLC task, RAkEL is considered as an improvement over LP, where RAkEL is actually an ensemble approach which allows the algorithm to be applied to large-scale data, especially when it comes to the data has several number of labels. RAkEL was demonstrated as more efficient than LP in traditional MLC task.

However, in context prediction, our experiments reveal that LP is more efficient than RAkEL, because contextual labels are limited and can be selected in advance, where RAkEL turns out to be cost-expensive on this task.

2). Special evaluation challenge: More than one conditions in the same context dimension could be recommended; or, none of conditions in a dimension was predicted. This situation is allowed; if not, LP and Classifier chains can be applied to avoid this problem.

Context Recommender6. Experimental Results (findings)1).Personalization is required, because personalized algorithms work much better than the non-personalized ones. i.e., simply recommending most popular ones (e.g. most popular season people visiting Poland) is not enough;

2).LP algorithm using SMO as classifier is the best MLC algorithms among all data sets and all methods examined. They beat all the other algorithms. KNN-based approaches worked bad, because context-aware data are usually sparse.

3).SMO is the best classifier used for MLC algorithms, but it increases computation costs if data is large and there are many contextual labels. The alternative choice is Bayesian Nets which worked good and not time-consuming

4). About running performance: LP using SMO is the best choice, but both LP and SMO increase computation costs if data is large and there are many contextual labels.

Solutions:a). Reduce the number of labels by pre-selections;b). Choose LP using Bayesian Nets

Conclusions and Future Work

• We formally introduce and discuss the application and research problem of context recommendations (CR). We believe that context recommenders will provide many more novel applications and new recommendation opportunities for both practical use and the research community.

• We propose the formal framework of CR applications and discuss the algorithmic paradigms for CR.

• We examine the algorithms using multi-label classifications, and infer some significant findings and patterns as introduced previously.

• Future work: Examine more other algorithms, and develop new evaluation metrics for this domain.

References

• Context Recommendations[1].Baltrunas, Linas, Marius Kaminskas, Francesco Ricci, LiorRokach, Bracha Shapira, and Karl-Heinz Luke. "Best usage context prediction for music tracks." In Proceedings of the 2nd Workshop on Context Aware Recommender Systems. 2010.[2].Yong Zheng, Bamshad Mobasher, Robin Burke. "Context Recommendation Using Multi-label Classification". In Proceedings of the 13th IEEE/WIC/ACM International Conference on Web Intelligence, 2014

• Multi-label Classifications[1].Tsoumakas, Grigorios, and Ioannis Katakis. "Multi-label classification: An overview." International Journal of Data Warehousing and Mining (IJDWM) 3, no. 3 (2007): 1-13.[2].Tsoumakas, Grigorios, Ioannis Katakis, and Ioannis Vlahavas. "Mining multi-label data." In Data mining and knowledge discovery handbook, pp. 667-685. Springer US, 2010.

Thanks!Yong Zheng, Bamshad Mobasher, Robin Burke

Center for Web Intelligence, DePaul University, Chicago

IEEE/WIC/ACM Conference on Web Intelligence

Aug 14, Warsaw, Poland