© author(s) of these slides including research results from the KOM research network and TU Darmstadt; otherwise it is specified at the respective slide 21-Sep-12
Prof. Dr.-Ing. Ralf Steinmetz KOM - Multimedia Communications Lab
ECTEL__Sem_Info_rec_learning_resources_v6.0_20120921_MA.pptx
Exploiting Semantic Information for Graph-based Recommendations of Learning Resources
Mojisola Anjorin Thomas Rodenhausen Renato Domínguez García Christoph Rensing
EC-TEL 2012, Saarbrücken
Research Talk
Ranking
Algorithms
Slideshare
Tags ResourcesUsers
Prepare Talk
Read-Up on Basics
Activities
Find Related Work
Friends
Friends
FriendsBlue Group
KOM – Multimedia Communications Lab 3
Application Scenario: CROKODIL
CROKODIL is a platform offering support for resource-based learning § Semantic Tag Types § Activities § Learner Groups & Friendships § Recommendations
[Anjorin et al, 2011]
http://demo.crokodil.de
KOM – Multimedia Communications Lab 4
§ Motivation: Resource-based Learning § Application Scenario: CROKODIL § CROKODIL’s Extended Folksonomy Model § Ascore and AInheritScore § Evaluation Methodology, Metrics and Results § Conclusion & Future Work
Overview
KOM – Multimedia Communications Lab 5
A folksonomy is a quadruple F:= (U, T, R, Y), where U – Users T – Tags R – Resources Y ⊆ U × T × R - tag assignment
Folksonomy Model
Research Talk
Ranking
Algorithms
Slideshare
Tags ResourcesUsers
[Hotho et al. 2006]
KOM – Multimedia Communications Lab 6
CROKODIL Extends the Folksonomy Model …
Research Talk
Ranking
Algorithms
Slideshare
Tags ResourcesUsers
KOM – Multimedia Communications Lab 7
… with Semantic Tag Types
[Böhnstedt et al. 2009]
Research Talk
Ranking
Algorithms
Slideshare
Tags ResourcesUsers
Genre
Event
Person
Location
Other
Topic
KOM – Multimedia Communications Lab 8
… with Activities
Research Talk
Ranking
Algorithms
Slideshare
Tags ResourcesUsers
Prepare Talk
Read-Up on Basics
Activities
Find Related Work
KOM – Multimedia Communications Lab 9
… with Learner Groups and Friendships
Research Talk
Ranking
Algorithms
Slideshare
Tags ResourcesUsers
Prepare Talk
Read-Up on Basics
Activities
Find Related Work
Friends
Friends
FriendsBlue Group
KOM – Multimedia Communications Lab 10
CROKODIL‘s Extended Folksonomy
FC:= (U, TTyped, R, YT, (A, <), YA, YU, G, friends) where U – users TTyped – typed tags R – learning resources YT ⊆ U × TTyped × R – tag assignment (A, <) – activities with sub-activities YA ⊆ U × A × R – activity assignment YU ⊆ U × A – activity membership
assignment G ⊆ P(U) – groups of learners friends ⊆ U × U – friendship relation
Research Talk
Ranking
Algorithms
Slideshare
Tags ResourcesUsers
Prepare Talk
Read-Up on Basics
Activities
Find Related Work
Friends
Friends
FriendsBlue Group
KOM – Multimedia Communications Lab 11
Resource Recommendations for CROKODIL
http://demo.crokodil.de
KOM – Multimedia Communications Lab 12
Graph-based recommender techniques can be classified as neighbourhood-based collaborative filtering approaches
Graph-based Resource Recommendations
Graph-based Ranking
Algorithm
Resource Score r1 0.9 r2 0.7 r3 0.5 r4 0.2
1 1
2 1
P1
P2
P4
P3
3
4
2
1
2
Folksonomy Graph e.g. FolkRank based on “Random Walk” of PageRank
Recommendation List (ranked resources)
[Desrosiers et al. 2011]
KOM – Multimedia Communications Lab 13
§ Motivation: Resource-based Learning § Application Scenario: CROKODIL § CROKODIL’s Extended Folksonomy Model § Ascore and AInheritScore § Evaluation Methodology, Metrics and Results § Conclusion & Future Work
Overview
KOM – Multimedia Communications Lab 14
1. Add activity nodes Vc = VF ∪ A 2. Add edges: § activity assignments (u, r, a) § assignments of a user to an
activity (u, a) § activity hierarchies (asub , asuper)
4. Assign weights to edges: § w(r,a) = w(r,u) = w(u,a)
= max(|Ut,r|) § w(u, a) = max(|Ru,t|) § w(asub,asuper) = max(|Ut,r|, |Ru,t|)
5. Run graph-based ranking algorithm e.g. FolkRank
AScore
[Abel et al, 2011] Inspired by GFolkRank
Extend the Folksonomy Graph F = (V, E) with Activities
Research Talk
Ranking
Algorithms
Slideshare
Tags ResourcesUsers
Prepare Talk
Read-Up on Basics
Activities
Find Related Work
KOM – Multimedia Communications Lab 15
§ Depending on the tags of a user,
scores are “inherited” over the activity hierarchy
§ Resources and users assigned to activities influence the scores as well
§ Scores are attenuated depending on activity distance § Activity distance between two
activities: the number of hops from one activity to the other
AInheritScore
[Abel et al, 2011] Inspired by GRank
Leveraging Activity Hierarchies to Calculate Scores Research Talk Ranking
Algorithms
Research Talk Prepare Talk
Read-Up on Basics
Find Related Work
...
... ...
KOM – Multimedia Communications Lab 16
§ Motivation: Resource-based Learning § Application Scenario: CROKODIL § CROKODIL’s Extended Folksonomy Model § Ascore and AInheritScore § Evaluation Methodology, Metrics and Results § Conclusion & Future Work
Overview
KOM – Multimedia Communications Lab 17
GroupMe! dataset
Evaluation Corpus and Evaluation Metrics
[Abel et al, GroupMe!]
Elements Count Users 649 Tags 2580 Resources 1789 Groups of Resources
1143
Posts 1865 Tag assignments 4366
The mean of the Average Precision over several queries Q
Mean Normalized Precision: The mean of the Precision@k over several queries Q
MAP(Q) =1
|Q|
|Q|�
j=1
1
mj
mj�
k=1
Precision(Rjk)
Mean Average Precision:
MNP(Q,k) =1
|Q|
|Q|�
j=1
Precisionj(k)
Precisionmax,j(k)
[Manning et al 2008]
KOM – Multimedia Communications Lab 18
Tango
Buenos
Aires
DancingFestival
Tango
Buenos
Aires
Dancing Festival
A post is a Pu,r= {(u,r,t)|(u,r,t) ∈ Y} For LeavePostOut, the recommendation task with user as input is harder as with tag as input
Evaluation Methodology: LeavePostOut
[Jäschke et al. 2007]
KOM – Multimedia Communications Lab 19
RTr,t= {(u,r,t)|(u,r,t) ∈ Y} For LeaveRTOut, the recommendation task with tag as input is harder as with user as input
Evaluation Methodology: LeaveRTOut
Tango
Buenos
Aires
DancingFestival
Tango
Buenos
Aires
Dancing Festival
KOM – Multimedia Communications Lab 20
A violin plot is a combination of a box plot and a density trace
Visualization of Results with Violin Plots
[Hintze et al. 1998]
KOM – Multimedia Communications Lab 21
A violin plot is a combination of a box plot and a density trace
Visualization of Results with Violin Plots
Median
3rd Quartile
1st Quartile [Hintze et al. 1998]
KOM – Multimedia Communications Lab 22
Evaluation results with user as input
Evaluation Results for LeavePostOut
KOM – Multimedia Communications Lab 23
Evaluation results with user as input
Evaluation Results for LeavePostOut
KOM – Multimedia Communications Lab 24
Evaluation results with user as input
Evaluation Results for LeavePostOut
KOM – Multimedia Communications Lab 25
Evaluation results with user as input
Evaluation Results for LeavePostOut
KOM – Multimedia Communications Lab 26
Evaluation results with user as input
Evaluation Results for LeavePostOut
KOM – Multimedia Communications Lab 27
Evaluation results with user as input
Evaluation Results for LeavePostOut
KOM – Multimedia Communications Lab 28
Evaluation Results for LeavePostOut
Approaches MAP GFolkRank 0.70 AScore 0.70 AInheritscore 0.47 GRank 0.38 FolkRank 0.19 Popularity 0.00
KOM – Multimedia Communications Lab 29
Evaluation Results for LeaveRTOut
Evaluation results with user as input
KOM – Multimedia Communications Lab 30
Evaluation Results for LeaveRTOut
Approaches MAP AScore 0.20 GFolkRank 0.20 FolkRank 0.18 GRank 0.14 AInheritscore 0.11 Popularity 0.02
KOM – Multimedia Communications Lab 31
Exploiting hierarchical activity structures as found in CROKODIL can improve the ranking of resources for the purpose of recommending learning resources § AScore § AInheritscore
Future Work § Evaluation using a data set from CROKODIL § User Study § Hybrid approaches
Conclusion and Future Work
www.crokodil.de
KOM – Multimedia Communications Lab 33
Statistical Significance Tests – LeavePostOut
More effective than à
Popularity Folk Rank
GFolkRank
AScore GRank AInheritScore
Poularity FolkRank X GFolkRank X X X X X AScore X X X X GRank X X AInheritScore X X X
Significance matrix of pair-wise comparisons of LeavePostOut results Based on Average Precision with a significance level of p = 0.05
KOM – Multimedia Communications Lab 34
Statistical Significance Tests – LeaveRTOut
More effective than à
Popularity Folk Rank
GFolkRank
AScore GRank AInheritScore
Poularity FolkRank X X X GFolkRank X X X X AScore X X X X X GRank X X AInheritScore X
Significance matrix of pair-wise comparisons of LeaveRTOut results Based on Average Precision with a significance level of p = 0.05
KOM – Multimedia Communications Lab 35
Adapted PageRank
!
!
!"
"
"
"
"
# #
$%&'()*+,& Tango0
Buenos
Aires0
Buenos
Aires0
Dancing
Festival0
1
"-.
#-.
#-.
"-.
PageRank‘s intelligent surfer model The ranking of a node is determined by how often the surfer visits the node Adjoining edges are followed with a certain probability – determined by the edge weights The query node acts as the starting point and focus i.e. the surfer returns to this node with a certain probability – determined by the node weights
[Hotho et al. 2006]
Top Related