Slide 1

Positional Relevance Model for PseudoRelevance Feedback Yuanhua Lv & ChengXiang Zhai Department of Computer Science, UIUC Presented by Bo Man 2014/11/18 Slide 2 Positional Relevance Model for PseudoRelevance Feedback Yuanhua Lv & ChengXiang Zhai Department of Computer Science, UIUC Presented by Bo Man 2014/11/18 Slide 3 Presentation Guideline Review on Feedback Motivation Positional Relevance Model Experiments, Results and Analyses Conclusions Slide 4 Presentation Guideline Review on Feedback Motivation Positional Relevance Model Experiments, Results and Analyses Conclusions Slide 5 Review on Feedback (1)Explicit Feedback Easy for training But need users interaction Slide 6 Review on Feedback (2)Implicit Feedback Not need interaction But work more on mining Slide 7 Slide 8 Slide 9 Slide 10 Slide 11 (3)Pseudo- Relevance Feedback No need users interaction No Mining Slide 12 Problems? Traditional Pseudo-Relevance Feedback assumes that the contents of a document are incoherent (sharing the same topic). What if a document shares different topics Term-based? Or document-based? Slide 13 Presentation Guideline Review on Feedback Motivation Positional Relevance Model Experiments, Results and Analyses Conclusions Slide 14 Presentation Guideline Review on Feedback Motivation Positional Relevance Model Experiments, Results and Analyses Conclusions Slide 15 Motivation How to effectively select from feedback documents the words that are focused on the query topic based on positions of terms in feedback documents? Slide 16 Presentation Guideline Review on Feedback Motivation Positional Relevance Model Experiments, Results and Analyses Conclusions Slide 17 Presentation Guideline Review on Feedback Motivation Positional Relevance Model Experiments, Results and Analyses Conclusions Slide 18 Positional Relevance Model Relevance Model (one of the most robust) rep- resent the set of smoothed document models for the pseudo feedback documents. p(D) is a prior on documents and is often assumed to be uniform without any additional prior knowledge about document D. After the relevance model is estimated, the estimated P (w|Q) can then be interpolated with the original query model Q to improve performance. is a parameter to control the amount of feedback. Slide 19 Positional Relevance Model(PRM) i indicates a position in document D F is the set of feedback documents (assumed to be relevant) Challenge? How to estimate joint probability? Positional Relevance Model Slide 20 Methods (1) i.i.d. Sampling (2) conditional Sampling estimating P (w, Q, D, i) Positional Relevance Model Slide 21 i.i.d. sampling Positional Relevance Model Slide 22 i.i.d. sampling derivation(1) P(D) can be interpreted as a document prior and set to a uniform distribution with no prior knowledge about document D. assume that every position is equally likely but it is possible to estimate P(i|D) based on document structures assume that the generation of word w and that of query Q are independent Positional Relevance Model Slide 23 i.i.d. sampling derivation(2) In the above equation, P (w|D, i) is the probability of sampling word w at position i in document D. To improve the efficiency of PRM, we simplify P (w|D, i) as: QUESTION: HOW to estimate ? The query likelihood at position i of document D. Positional Relevance Model Slide 24 conditional sampling QUESTION: HOW to estimate ? The query likelihood at position i of document D. Positional Relevance Model Slide 25 estimate the query likelihood at position i of document D (1)Use Positional Language Model (2)Use Gaussian kernel function (3)Approximate (4)Set parameters Positional Relevance Model Slide 26 estimate the query likelihood at position i of document D (5)Use JM Smoothing (6)Compute The computation of positional query likelihood is the most time-consuming part in estimating PRM. Positional Relevance Model Slide 27 Presentation Guideline Review on Feedback Motivation Positional Relevance Model Experiments, Results and Analyses Conclusions Slide 28 Presentation Guideline Review on Feedback Motivation Positional Relevance Model Experiments, Results and Analyses Conclusions Slide 29 Experiments and Results Evaluation methods. (1) The basic retrieval model is the KL-divergence retrieval model, and we chose the Dirichlet smoothing method [33] for smoothing document language models, where the smoothing parameter was set empirically to 1500. This method was labeled as NoFB. (2) The baseline pseudo feedback method is the relevance model RM3, which is one of the most effective and robust pseudo feedback methods un- der language modeling framework. (3) Another baseline pseudo feedback method is a standard passage- based feed- back model, labeled as RM3-p, which estimates the RM3 relevance model based on the best matching passage of each feedback document. (4) We have two variations of PRM, i.e., PRM1 and PRM2, which are based on the two estimation methods described in Section 3.2, respectively. (5) In addition, we also used PRM1 and PRM2 for passage feed- back in a way as RM3-p does. Specifically, we first computed a PLM for each position of the document, and then we estimate a PRM based on a passage of size 2 centered at the position with the maximum positional query likelihood score Slide 30 Results Experiments and Results Slide 31 More results Robustness Analysis Experiments and Results Slide 32 Presentation Guideline Review on Feedback Motivation Positional Relevance Model Experiments, Results and Analyses Conclusions Slide 33 Presentation Guideline Review on Feedback Motivation Positional Relevance Model Experiments, Results and Analyses Conclusions Slide 34 Propose a novel positional relevance model(PRM) PRM exploits term position and proximity to assign more weights to words closer to query words, based on the intuitionwords closer to query words are more likely to be consistent with the query topic. Experiments results show that PRM is quite effective and performs significantly better than others based on document or passage. Conclusions Slide 35 Questions?