Dynamic Information Retrieval Tutorial

download Dynamic Information Retrieval Tutorial

of 214

  • date post

    23-Aug-2014
  • Category

    Science

  • view

    902
  • download

    22

Embed Size (px)

description

Dynamic aspects of Information Retrieval (IR), including changes found in data, users and systems, are increasingly being utilized in search engines and information filtering systems. Examples include large datasets containing sequential data capturing document dynamics and modern IR systems observing user dynamics through interactivity. Existing IR techniques are limited in their ability to optimize over changes, learn with minimal computational footprint and be responsive and adaptive. The objective of this tutorial is to provide a comprehensive and up-to-date introduction to Dynamic Information Retrieval Modeling. Dynamic IR Modeling is the statistical modeling of IR systems that can adapt to change. It is a natural follow-up to previous statistical IR modeling tutorials with a fresh look on state-of-the-art dynamic retrieval models and their applications including session search and online advertising. The tutorial covers techniques ranging from classic relevance feedback to the latest applications of partially observable Markov decision processes (POMDPs) and presents to fellow researchers and practitioners a handful of useful algorithms and tools for solving IR problems incorporating dynamics. http://www.dynamic-ir-modeling.org/ @inproceedings{Yang:2014:DIR:2600428.2602297, author = {Yang, Hui and Sloan, Marc and Wang, Jun}, title = {Dynamic Information Retrieval Modeling}, booktitle = {Proceedings of the 37th International ACM SIGIR Conference on Research \&\#38; Development in Information Retrieval}, series = {SIGIR '14}, year = {2014}, isbn = {978-1-4503-2257-7}, location = {Gold Coast, Queensland, Australia}, pages = {1290--1290}, numpages = {1}, url = {http://doi.acm.org/10.1145/2600428.2602297}, doi = {10.1145/2600428.2602297}, acmid = {2602297}, publisher = {ACM}, address = {New York, NY, USA}, keywords = {dynamic information retrieval modeling, probabilistic relevance model, reinforcement learning}, }

Transcript of Dynamic Information Retrieval Tutorial

  • SIGIRTutorial July 7th 2014 Grace Hui Yang Marc Sloan JunWang Guest Speaker: EmineYilmaz Dynamic Information Retrieval Modeling
  • Dynamic Information Retrieval ModelingTutorial 20142
  • Age of Empire Dynamic Information Retrieval ModelingTutorial 20143
  • Dynamic Information Retrieval Dynamic Information Retrieval ModelingTutorial 20144 Documents to explore Information need Observed documents User Devise a strategy for helping the user explore the information space in order to learn which documents are relevant and which arent, and satisfy their information need.
  • Evolving IR Dynamic Information Retrieval ModelingTutorial 20145 Paradigm shifts in IR as new models emerge e.g.VSM BM25 Language Model Different ways of defining relationship between query and document Static Interactive Dynamic Evolution in modeling user interaction with search engine
  • Outline Dynamic Information Retrieval ModelingTutorial 20146 Introduction Static IR Interactive IR Dynamic IR Theory and Models Session Search Reranking GuestTalk: Evaluation
  • Conceptual Model Static IR Dynamic Information Retrieval ModelingTutorial 20147 Static IR Interactive IR Dynamic IR No feedback
  • Characteristics of Static IR Dynamic Information Retrieval ModelingTutorial 20148 Does not learn directly from user Parameters updated periodically
  • Static Information Retrieval Model Dynamic Information Retrieval ModelingTutorial 20149 Learning to Rank
  • Dynamic Information Retrieval ModelingTutorial 201410 Commonly Used Static IR Models BM25 PageRank Language Model
  • Feedback in IR Dynamic Information Retrieval ModelingTutorial 201411
  • Outline Dynamic Information Retrieval ModelingTutorial 201412 Introduction Static IR Interactive IR Dynamic IR Theory and Models Session Search Reranking GuestTalk: Evaluation
  • Conceptual Model Interactive IR Dynamic Information Retrieval ModelingTutorial 201413 Static IR Interactive IR Dynamic IR Exploit Feedback
  • Interactive User Feedback Dynamic Information Retrieval ModelingTutorial 201414 Like, dislike, pause, skip
  • Learn the users taste interactively! At the same time, provide good recommendations! Dynamic Information Retrieval ModelingTutorial 201415 Interactive Recommender Systems
  • Example - Multi Page Search Dynamic Information Retrieval ModelingTutorial 201416 Ambiguous Query
  • Example - Multi Page Search Dynamic Information Retrieval ModelingTutorial 201417 Topic: Car
  • Example - Multi Page Search Dynamic Information Retrieval ModelingTutorial 201418 Topic:Animal
  • Example Interactive Search Dynamic Information Retrieval ModelingTutorial 201419 Click on car webpage
  • Example Interactive Search Dynamic Information Retrieval ModelingTutorial 201420 Click on Next Page
  • Example Interactive Search Dynamic Information Retrieval ModelingTutorial 201421 Page 2 results: Cars
  • Example Interactive Search Dynamic Information Retrieval ModelingTutorial 201422 Click on animal webpage
  • Example Interactive Search Dynamic Information Retrieval ModelingTutorial 201423 Page 2 results: Animals
  • Example Dynamic Search Dynamic Information Retrieval ModelingTutorial 201424 Topic: Guitar
  • Example Dynamic Search Dynamic Information Retrieval ModelingTutorial 201425 Diversified Page 1 Topics: Cars, animals, guitars
  • Toy Example Dynamic Information Retrieval ModelingTutorial 201426 Multi-Page search scenario User image searches for jaguar Rank two of the four results over two pages: = 0.5 = 0.51 = 0.9 = 0.49
  • Toy Example Static Ranking Dynamic Information Retrieval ModelingTutorial 201427 Ranked according to PRP Page 1 Page 2 1. 2. = 0.9 = 0.51 1. 2. = 0.5 = 0.49
  • Toy Example Relevance Feedback Dynamic Information Retrieval ModelingTutorial 201428 Interactive Search Improve 2nd page based on feedback from 1st page Use clicks as relevance feedback Rocchio1 algorithm on terms in image webpage = + | | New query closer to relevant documents and different to non-relevant documents 1Rocchio, J. J., 71, Baeza-Yates & Ribeiro-Neto99
  • Toy Example Relevance Feedback Dynamic Information Retrieval ModelingTutorial 201429 Ranked according to PRP and Rocchio Page 1 Page 2 2. = 0.9 = 0.51 1. 2. = 0.5 = 0.49 1. * * Click
  • Toy Example Relevance Feedback Dynamic Information Retrieval ModelingTutorial 201430 No click when searching for animals Page 1 Page 2 2. = 0.9 = 0.51 1. 2. 1. ? ?
  • Toy Example Value Function Dynamic Information Retrieval ModelingTutorial 201431 Optimize both pages using dynamic IR Bellman equation for value function Simplified example: , = max + ( +1 +1 , +1 ) , = relevance and covariance of documents for page = clicks on page =value of ranking on page Maximize value over all pages based on estimating feedback
  • 1 0.8 0.1 0 0.8 1 0.1 0 0.1 0.1 1 0.95 0 0 0.95 1 Toy Example - Covariance Dynamic Information Retrieval ModelingTutorial 201432 Covariance matrix represents similarity between images
  • Toy Example Myopic Value Dynamic Information Retrieval ModelingTutorial 201433 For myopic ranking, 2 = 16.380 Page 1 2. 1.
  • Toy Example Myopic Ranking Dynamic Information Retrieval ModelingTutorial 201434 Page 2 ranking stays the same regardless of clicks Page 1 Page 2 2. 1. 2. 1.
  • Toy Example Optimal Value Dynamic Information Retrieval ModelingTutorial 201435 For optimal ranking, 2 = 16.528 Page 1 2. 1.
  • Toy Example Optimal Ranking Dynamic Information Retrieval ModelingTutorial 201436 If car clicked, Jaguar logo is more relevant on next page Page 1 Page 2 2. 1. 2. 1.
  • Toy Example Optimal Ranking Dynamic Information Retrieval ModelingTutorial 201437 In all other scenarios, rank animal first on next page Page 1 Page 2 2. 1. 2. 1.
  • Interactive vs Dynamic IR Dynamic Information Retrieval ModelingTutorial 201438 Treats interactions independently Responds to immediate feedback Static IR used before feedback received Optimizes over all interaction Long term gains Models future user feedback Also used at beginning of interaction Interactive Dynamic
  • Outline Dynamic Information Retrieval ModelingTutorial 201439 Introduction Static IR Interactive IR Dynamic IR Theory and Models Session Search Reranking GuestTalk: Evaluation
  • Conceptual Model Dynamic IR Dynamic Information Retrieval ModelingTutorial 201440 Static IR Interactive IR Dynamic IR Explore and exploit Feedback
  • Characteristics of Dynamic IR Dynamic Information Retrieval ModelingTutorial 201441 Rich interactions Query formulation Document clicks Document examination eye movement mouse movements etc.
  • Characteristics of Dynamic IR Dynamic Information Retrieval ModelingTutorial 201442 Temporal dependency clicked documentsquery D1 ranked documents q1 C1 D2 q2 C2 Dn qn Cn I information need iteration 1 iteration 2 iteration n
  • Characteristics of Dynamic IR Dynamic Information Retrieval ModelingTutorial 201443 Overall goal Optimize over all iterations for goal IR metric or user satisfaction Optimal policy
  • Dynamic IR Dynamic Information Retrieval ModelingTutorial 201444 Dyna