a a t t i i c c í í l l p m p m I I n n ó ó i i c c a a t...

RReettrrooaalliimmeennttaacciióónn IImmppllíícciittaaIIC 3633 - Sistemas Recomendadores - PUC Chile

Denis ParraProfesor Asistente, DCC, PUC CHile

Retroalimentación Implícita file:///Volumes/GoogleDrive/My Drive/PUC/IIC3633-2018-2/Website_R_2018/clase7_implicit-feedback.html#1

1 of 37 8/21/18, 09:09

Retroalimentación Implícita

ref: Hu, Y., Koren, Y., & Volinsky, C. (2008). Collaborative filtering for implicit feedback datasets.In ICDM'08. Eighth IEEE Internatioonal Conference on Data Mining (pp. 263-272).

Hasta hace pocos años, la gran mayoría de los modelos avanzados de recomendación,basados en factorización matricial, dependían de preferencias explícitas del usuario enforma de ratings.

Pero los ratings (explicit feedback) son difíciles de obtener.

Por otro lado, tenemos la opción de usar feedback implícito, pero con los siguientesproblemas:

·

·

·

No hay feedback negativo.

Contiene ruido.

Es difícil cuantificar preferencia y confianza en esas preferencias.

Hay una carencia de métricas de evaluación (RMSE y MAE no funcionarían bien)

-

-

-

-

2/37


2 of 37 8/21/18, 09:09

Paper 1

Hu, Y., Koren, Y., & Volinsky, C. (2008).

Collaborative filtering for implicit feedback datasets.

In ICDM'08. Eighth IEEE Internatioonal Conference on DataMining (pp. 263-272).

3/37


3 of 37 8/21/18, 09:09

Ratings : recurso escasoSi bien SVD++ considera implicit feedback, este modelo optimiza específicamente feedbackimplícito

Considera, antes que todo, valores binarios de consumo/no consumo del ítem

·

·

4/37


4 of 37 8/21/18, 09:09

Modelo Implicit Feedback - Hu et al.Se considera también la confianza de observar con la variable ( = 40, uso de CV)

es, en este caso, el implicit feedback (e.g. plays)

La función que esperamos minizar es, luego

· pui cui α

rui

·

5/37


5 of 37 8/21/18, 09:09

Modelo Implicit Feedback - Hu et al. IIAprendizaje de parámetros (factores latentes): ALS en lugar de SGD.

puede tomar distintas formas. Una alternativa es

·

· cui

De esta forma, el implicit feedback se descompone en (prefencias) y (nivel deconfianza), y

Maneja todas las combinaciones usuario-item (n * m) en tiempo lineal al explotar laestructura algebraica de las variables

· rui pui cui

·

6/37


6 of 37 8/21/18, 09:09

ExperimentoServicio de TV digital, datos recolectados de 300.000 set top boxes.

En un período de 4 semanas, 17.000 programas de TV únicos

: cuantás veces usuario vio programa en un período de 4 semanas

Luego de una agregación y limpieza de datos, : 32 millones

·

·

· rui u i· | |rui

7/37


7 of 37 8/21/18, 09:09

Evaluación y resultados : percentil-ranking de un programa en la lista de recomendación de .

Si = 0%, el programa ha sido predicho como el más relevante para el usuario , y si = 100%, el programa es el menos deseado. Expected percentile ranking : the

smaller the better

· rankui i u· rankui i u

rankui i rankˉ

8/37


8 of 37 8/21/18, 09:09

Resultados I

9/37


9 of 37 8/21/18, 09:09

Resultados II

10/37


10 of 37 8/21/18, 09:09

Paper 2

Parra, D., & Amatriain, X. (2011).Walk the Talk: Analyzing the Relation between Implicit andExplicit Feedback for Preference Elicitation.In User Modeling, Adaptation and Personalization (pp.255-268). Springer Berlin Heidelberg.

Parra, D., Karatzoglou, A., Amatriain, X., & Yavuz, I. (2011).Implicit feedback recommendation via implicit-to-explicitordinal logistic regression mapping. Proceedings of theCARS Workshop, Chicago, IL, USA, 2011.

11/37


11 of 37 8/21/18, 09:09

IntroductionIs it possible to map implicit behavior to explicit preference (ratings)?

Which variables better account for the amount of times a user listens to online albums?[Baltrunas & Amatriain CARS ‘09 workshop – RecSys 2009.]

OUR APPROACH: Study with Last.fm users

·

·

·

Part I: Ask users to rate 100 albums (how to sample)

Part II: Build a model to map collected implicit feedback and context to explicit feedback

-

-

12/37


12 of 37 8/21/18, 09:09

Walk the Talk (2011)

13/37


13 of 37 8/21/18, 09:09

Walk the Talk - IIRequisitos para participar en estudio: > 18años, scrobblings > 5000·

14/37


14 of 37 8/21/18, 09:09

Muestreo de Datos para estudio de UsuarioCuántos y qué items (álbums) deberian ver los usuarios?·

Implicit Feedback (IF): playcount for a user on a given album. Changed to scale [1-3], 3means being more listened to.

Global Popularity (GP): global playcount for all users on a given album [1-3]. Changed toscale [1-3], 3 means being more listened to.

Recentness (R) : time elapsed since user played a given album. Changed to scale [1-3], 3means being listened to more recently.

-

-

-

15/37


15 of 37 8/21/18, 09:09

Análisis de Regresión

Including Recentness increases R2 in more than 10% [ 1 -> 2]

Including GP increases R2, not much compared to RE + IF [ 1 -> 3]

Not Including GP, but including interaction between IF and RE improves the variance of theDV explained by the regression model. [ 2 -> 4 ]

·

·

·

16/37


16 of 37 8/21/18, 09:09

Análisis de Regresión 2

RMSE1: Considera los ratings = 0.

We tested conclusions of regression analysis by predicting the score, checking RMSE in 10-fold cross validation.

Results of regression analysis are supported.

·

·

·

17/37


17 of 37 8/21/18, 09:09

Conclusions of Part IUsing a linear model, Implicit feedback and recentness can help to predict explicit feedback(in the form of ratings)

Global popularity doesn’t show a significant improvement in the prediction task

Our model can help to relate implicit and explicit feedback, helping to evaluate and compareexplicit and implicit recommender systems.

·

·

·

18/37


18 of 37 8/21/18, 09:09

Parte IIImplicit Feedback Recommendation via Implicit-to-Explicit OLR Mapping (Recsys 2011, CARSWorkshop)

·

Consider ratings as ordinal variables

Use mixed-models to account for non-independence of observations

Compare with state-of-the-art implicit feedback algorithm

-

-

-

19/37


19 of 37 8/21/18, 09:09

Supuestos en el estudio ILinear Regression did not account for the nested nature of ratings·

And ratings were treated as continuous, when they are actually ordinal.·

20/37


20 of 37 8/21/18, 09:09

Modelo II: Ordinal Logistic RegressionActually Mixed-Effects Ordinal Multinomial Logistic Regression

Mixed-effects: Nested nature of ratings

We obtain a distribution over ratings (ordinal multinomial) per each pair USER, ITEM -> wepredict the rating using the expected value. … And we can compare the inferred ratings witha method that directly uses implicit information (playcounts) to recommend ( by Hu, Korenet al. 2007)

·

·

·

21/37


21 of 37 8/21/18, 09:09

Ordinal Logistic Regression MappingModel·

Predicted values·

22/37


22 of 37 8/21/18, 09:09

DatasetsD1: users, albums, if, re, gp, ratings, demographics/consumption

D2: users, albums, if, re, gp, NO RATINGS.

·

·

23/37


23 of 37 8/21/18, 09:09

Results

24/37


24 of 37 8/21/18, 09:09

Conclusions and current work

25/37


25 of 37 8/21/18, 09:09

Paper 3

Xing Yi, Liangjie Hong, Erheng Zhong, Nanthan Nan Liu,and Suju Rajan. 2014.

Beyond clicks: dwell time for personalization.

ACM RecSys 2014.

26/37


26 of 37 8/21/18, 09:09

Dwell TimeMethod to consume fine-grained dwell-time at web scale

Dwell times varies by device (correlation between)

Raw dwell time distributions change considerably on content type, but at least log-rawdistributions are bell shaped

·

Focus Blur (FB) and Last Event (LE) methods: server side methods

Focus blur closer to client side, so is the one used

-

-

·

·

27/37


27 of 37 8/21/18, 09:09

Dwell Time IIChallenge: dwell time normalization, to extract an engagement signal which is comparableacross devices -> they normalize

·

Dwell time is used in a learning to rank approach (using dwell time as target) to rankitems

Evaluation on Yahoo! logs

Option 2 is using directly dwell time in a CF-based recommendation

-

-

-

28/37


28 of 37 8/21/18, 09:09

Eventos: Server y Client-Side

29/37


29 of 37 8/21/18, 09:09

Dwell Time para Distintos Dispositivos

30/37


30 of 37 8/21/18, 09:09

Dwell Time vs. Largo del articulo

31/37


31 of 37 8/21/18, 09:09

Dwell Time vs. Número de Fotos

32/37


32 of 37 8/21/18, 09:09

Slideshows en Distintos Dispositivos

33/37


33 of 37 8/21/18, 09:09

Consumo de Videos en Distintos Dispositivos

34/37


34 of 37 8/21/18, 09:09

Features

35/37


35 of 37 8/21/18, 09:09

Evaluación

36/37


36 of 37 8/21/18, 09:09

ReferenciasHu, Y., Koren, Y., & Volinsky, C. (2008). Collaborative filtering for implicit feedback datasets. InICDM'08. Eighth IEEE Internatioonal Conference on Data Mining (pp. 263-272).

Parra, D., & Amatriain, X. (2011). Walk the Talk: Analyzing the Relation between Implicit andExplicit Feedback for Preference Elicitation. In User Modeling, Adaptation andPersonalization (pp. 255-268). Springer Berlin Heidelberg.

Xing Yi, Liangjie Hong, Erheng Zhong, Nanthan Nan Liu, and Suju Rajan. 2014. Beyond clicks:dwell time for personalization. ACM RecSys 2014.

·

·

·

37/37


37 of 37 8/21/18, 09:09

a a t t i i c c í í l l p m p m I I n n ó ó i i c c a a t...

Documents

Transcript of a a t t i i c c í í l l p m p m I I n n ó ó i i c c a a t...