Understanding User Goals in Web Search

Daniel E. RoseDanny Levinson

Not only what, but also why

● previous work on information-seeking behavior

● authors' hierarchy of search goals

● how to classify queries

● analysis of results

● applicability of model

LIS precedents● Bates

○ "Information Search Tactics" (1979)

○ "The Design of Browsing and Berrypicking Techniques for the Online Search Interface" (1989)

Diagrams are recreations of figures that appear in "The Design of Browsing and Berrypicking Techniques for the Online Search Interface."

Above: Bates' berrypicking model Left: the traditional model of document recovery

● Belkin, Oddy, and Brooks ○ "ASK for Information Retrieval" (1982)

■ from "Part I. Background and Theory"

■ from "Part II. Results of a Design Study"

● Broder○ "A Taxonomy of Web Search" (2002)

Process

● developed flat list of user goals

● used list to classify queries in test set

● revised categories as necessary--developing hierarchical classification

● manually classified three sets of queries

○ Each set of approximately 500 queries was randomly selected from AltaVista query logs on a different day, and at a different time of the day, than the other two.

Search goal hierarchy

Search goal hierarchy (continued)

Manual query classification

"We need to know the relative prevalence of various goals. And if we hope to infer goals automatically in the future, we need to know that it is possible to do so manually" (16).

Objects of consideration:● ��the query itself● results returned by the

search engine● results clicked on by the

user● further searches or other

actions by the userDo these documents provide "sufficient information for a human to consistently classify queries according to our goal framework" (16)?

"Once we could successfully classify queries manually, we would be able to provide training data for a future automatic classification system" (16).

Final Fantasy example

What kind of a site did the user who searched for "final fantasy" intend to find--a site that sells a version of the game, one that lists the game's "official" rules, or one that provides less exact information about the game?

Time Delta t Event Details

36 36 result click pg 1, pos 1 http://www.ffonline.com

113 77 query pg 1 final fantasy

118 5 result click pg 1, pos 8 http://www.eyesonff.com

147 29 result click pg 1, pos 8 http://www.eyesonff.com

Examining the results returned by two search engines for this query, and the user's subsequent clicks, the authors conclude that the goal of this search was "undirected" information.

Results

Results (continued)

● "One issue is that we have no way of knowing conclusively whether the goal we inferred for a query is in fact the user’s actual goal. In the future, we would like to combine our work with user studies, including qualitative data such as diary reports of user goals" (19).

Applications and limitations

● "If our findings about the relatively small number of navigational queries are accurate, they suggest that much of the attention in the commercial search engine world may be misdirected" (18).

Questions● How does Rose and Levinson's user goals paradigm

compare to the idea of document relevance? Does a result need to meet a search goal in order for the user to perceive it as relevant, or simply convey information related to the search query?

● In what ways does their user goals model take into account query refinement and/or query expansion?

● What is the user's goal in the Aloha/American Airlines scenario (outlined on slides 8 and 9)?

● How would you apply Rose and Levinson's research to the following questions:

○ Why does a search occur?○ What is its purpose?○ How does it proceed?