Using Metadata to Improve Search User Interfaces Marti Hearst UC Berkeley FLINT Workshop, August...
-
date post
21-Dec-2015 -
Category
Documents
-
view
217 -
download
0
Transcript of Using Metadata to Improve Search User Interfaces Marti Hearst UC Berkeley FLINT Workshop, August...
Using Metadata to Improve Search User Interfaces
Marti Hearst
UC Berkeley
FLINT Workshop, August 2001
My Claims
Web Search is OK Gets people to the right starting points
Web SITE search is NOT ok The best way to improve site search is
NOT to make new fancy algorithms Instead …
The best wasy to improve search:
Improve the User Interface
Recent Study by Vividence Research
Spring 2001, 69 web sites The most common problems:
53% had poorly organized search results 32% had poor information architecture 32% had slow performance 27% had cluttered home pages
Organizing Search Results
There is a lot of prior work on this Cha-Cha (Chen et al. 1999)
Scatter-Gather clustering (Cutting et al. 93, Hearst et al. 1996)
Becoming more prevalent in web search too. Teoma Visimo Northern Light
Putting Results into Categories
Usually doesn’t work well when drilling down, however
Web Search Results Grouping
Drill down one category Cannot mix and match categories Probably isn’t all that useful But …what about site search?
A Taxonomy of WebSites
Catalog SitesWeb-based Information
Systems
Web-Presence Sites
Service-Oriented Sites
low
low
high
high
Complexity of Applications
Complexity of Data
From Mecca et al.,WebDB’99
An Important Trend
Web sites generated from databases Implications:
Web sites can adapt to user actions Web sites can be instrumented
Navigation on the Web
Web search engines are good at getting people to the right site.
But … what happens when the user reaches the site?
Follow Links … or …
Search
An Analogy
text searchhypertext
Goal: An All-Tertrain Vehicle
The best of both techniques A vehicle that magically lays down track to
suggest choices of where you want to go next based on what you’ve done so far and what you are trying to do
The tracks follow the lay of the land and go everywhere, but cross over the crevasses
The tracks allow you to back up easily
New interfaces are mixing and matching thesaurus-style metadata
Time/Date Topic RoleGeoRegion
The question: how to do this effectively?
Goals for Metadata Usage
Well-integrated with search Provides useful hints of where to go next Tailored to task as it develops Personalized Dynamic
The FLAMENCO ProjectFLexible Access using MEtadata in Novel Combinations
Main goal: Perform systematic studies to determine how
metadata should be incorporated into search Answer questions such as:
Given a set of user goals and a set of information: How many metadata combinations to show? What level of detail to show? How best to preview and postview choices?
Evaluation Methodology
Regression Test Select a set of tasks
Use these throughout the evaluation Start with a baseline system
Evaluate using the test tasks Add a feature
Evaluation again Compare to baseline Only retain those changes that improve results
Recipe Example
soar.berkeley.edu/recipes
soar.berkeley.edu/recipes
soar.berkeley.edu/recipes
www.epicurious.com
www.epicurious.com
www.epicurious.com
www.epicurious.com
Epicurious Metadata Usage
Advantages Creates combinations of metadata on the fly Different metadata choices show the same information in
different ways Previews show how many recipes will result Easy to back up Supports several task types
``Help me find a summer pasta,'' (ingredient type with event type), ``How can I use an avocado in a salad?'' (ingredient type with dish type), ``How can I bake sea-bass'' (preparation type and ingredient type)
Metadata usage in Epicurious
PrepareCuisineIngredient Dish
Recipe
Metadata usage in Epicurious
PrepareCuisineIngredient Dish
PrepareCuisineDishISelect
Metadata usage in Epicurious
PrepareCuisineIngredient Dish
I >
Group by
PrepareCuisineDish
Metadata usage in Epicurious
PrepareCuisineIngredient Dish
PrepareCuisineDishI >
Group by
Metadata usage in Epicurious
PrepareCuisineIngredient Dish
PrepareCuisineDishI >
Group by
PrepareCuisineISelect
Metadata Usage in Epicurious
Can choose category types in any order But categories never more than one level deep And can never use more than one instance of a
category Even though items may be assigned more than one
of each category type Items (recipes) are dead-ends
Don’t link to “more like this” Not fully integrated with search
Epicurious Basic SearchLacks integration with metadata
The FLAMENCO PROJECTThe FLAMENCO PROJECTUsability Studies to determine how Usability Studies to determine how
to improve website searchto improve website searchMarti Hearst
Rashmi Sinha
Ame Elliott
Jen English
Kirsten Swearington
Ping Yee
http://bailando.sims.berkeley.edu/flamenco.html
The FLAMENCO Project
Usability studies to determine how to improve site search using metadata
Researchers: Marti Hearst, Rashmi Sinha Ame Elliott, Jennifer English, Kirstin Swearington,
Ping Yee So far:
Epicurious usability study Image search interface framework and study design
Epicurious Usability Study 9 participants so far
Independent Variables: 1) Epicurious Interface (Basic vs. Enhanced vs. Browse) 2) Task type (known-item search vs. browsing for inspiration) 3) Degree of constraint of query 4) Number of results required (1 vs. many)
Dependent Variables: 1) Time to find satisfactory recipe(s) 2) Navigation path (backtracking, starting over, revising queries) 3) Satisfaction with results of search 4) Satisfaction with individual system features (e.g. breadcrumbs,
query previews, refine by hyperlinks) 5) Likelihood of using each interface in the future.
Epicurious Usability Study
Participants were asked to: Do 3 pre-specified searches in advance In the lab:
Specify a cooking scenario of interest to them Search for 3 recipes for this recipe Search for each recipe using each of the interfaces
Complete several structured tasks Along the way, answer questions about
Getting closer or farther away from goal Satisfaction with search results Satisfaction with the interace
Usability Study: Preliminary Results, Preference Data
Site Basic Enhanced BrowseTotal "Very Likely" to Use: 7 2 4 7
Total "Likely" to Use: 0 1 1 0Total "Not Likely" to Use: 2 6 4 2
PERCENTAGES Basic Enhanced BrowseVery Satisfied 32 43 35
Satisfied 50 43 52Middle 9 4 4
Dissatisfied 9 9 9
FavoriteSubject_JG: EnhancedSubject_NS: EnhancedSubject_SP: Browse
Subject_RM: Browse
Subject_LA: Enhanced
Subject_MC: BrowseSubject_MW: BrowseSubject_NM: EnhancedSubject_CG: Browse
Query previews and navigation. Options to refine by course or season. Choose how you view the results
Searching within made all the difference. I could see how many results I was getting in each Very specific. I can choose more than 1 detail with search for recipe I'm looking for.Likes the way it narrows things down. And it gives you the numbers.
Found it simpler, more readable. Helped you hone in on the season.Liked the kid friendly, low fat optionWhy?
Can narrow down when you're stuck. You can always refine [your search].
Allowed me to make specific selections. I liked Browse too. Gave lots to choose from. Depends on what you’re looking for that day
Can limit and unlimit and limit again in a different way. Prioritize your criteria--change the first thing I clicked and go in a different direction. Easy to back up.
Usability StudyPreliminary Results: Feature Preference
Subject_SP Subject_RM Subject_LA Subject_MC Subject_MW Subject_NM Subject_CGQuery previews +1 DN +1 0 +1 +1 +1
Having a complete list of ingredients (enhanced and browse) +1 +1 +1
enhanced - 0, browse - +1 +1 +1 +1
Search within results +1 +1 0 +1 +1 +1 +1Refine using hyperlinks (browse) +1 +1 DN +1 +1 DN +1Set all criteria from one screen (enhanced) 0 +1 +1 -1 0 +1 -1"May include" and "must include" options 0 0 DN 0 +1 +1 0"All words," "any words," "exact phrase," and "Boolean" options 0 0 0 DN DN +1 DN
KEY+1 - Helpful0 - Not helpful-1 - InterferedDN - Didn't notice
Usability StudyPreliminary Results: Quantitative
Search Metrics Basic Enhanced Browse
No Facets / Keywords per search
Total per search 4.3 7.7 ~2.11
During Step 1 of Search 2.3 always 1
For Each Step of Search 2.2 3.5 ~1
Ave. times 0 Results
During Step 1 0.0% 15.8% 0.0%
Overall 12.2% 31.4% 0.0%
Ave. Time ( in seconds)
Time per Search 98.6 130.3 108.11
Time per Step 50.9 60.6 41.38
Ave. No Steps 1.86 2.1 2.81
Median No Results
Overall (excluding 0 results) 186 17 704
in Step 1 13 10 579
In Final Step 32 14 13
Usability StudyPreliminary Results: Constraint-based Preferences
# of Results High LowEnhanced (LA) Browse (LA)Enhanced (MC) Browse (MC)Browse (MW) Browse (MW)Enhanced (NM) Enhanced (NM)Basic (CG) Browse (CG)Enhanced (LA) Browse (LA)Enhanced (MC) Browse (MC)Enhanced (MW) Browse (MW)Enhanced (NM) Enhanced (NM)Enhanced (CG) Browse (CG)
Constraint
1 result needed
Many results needed
Usability Study Results: Summary
People liked the browsing-style metadata-based search and found it helpful
People sometimes preferred the search-style metadata search when the task was more constrained But zero results are frustrating This can be alleviated with query previews
People disprefer the standard simple search
Application to Image Search
Image Search: What is the task?
Illustrate my slides? “Find a crevasse” Keyword match works pretty
well Find inspiration for an
architectural design? Needs richer search support
Faceted Metadata
Planalto Palace Parti Communiste Francais Pantheon
Oscar Neimeyer Oscar Neimeyer Jacques-Gabriel Soufflot
20th Century 20th Century 17th & 18th C.
Brasilia Paris Paris
Stone Curvilinear Stone
Image:
Architect:
Period:
Location:
Concept:
Planalto Palace Parti Communiste Francais Pantheon
Oscar Neimeyer Oscar Neimeyer Jaques-Gabriel Soufflot
20th Century 20th Century 17th & 18th C.
Brasilia Paris Paris
Stone Curvilinear Stone
Image:
Architect:
Period:
Location:
Concept:
Faceted Metadata
SPIRO Query Form
SP
IRO
que
ry o
n S
ubje
ct: c
hurc
h
Results of a pilot study with Archictects: Metadata is Helpful
Very positive feedback about the general approach All 9 participants named the metadata in
the search results area as their favorite aspect of Flamenco
Metadata was successful at giving hints about where to go next Perceived as useful “These are places I
can go from here.”
Results: More Metadata Please
Participants asked for more metadata Although there were complaints about the contents of
the metadata, users still wanted more Longer lists of options (more hints) Users wanted more control to make very specific searches
Half the participants requested the ability to control order of results with metadata Juxtapose visible images 2 different ways:
Overview (one image from each project) vs. like together ( all images of a project next to each other)
Different than ranking for text retrieval (precision, recall), but ordering does matter
Results: Complaints The UI was not successful at clarifying
searching within results vs. starting a new search Only 2 of the 9 participants understood the
distinction without discussion – but they want to do both
The 1/3 of the participants who couldn’t find a treasure hunt image felt that Flamenco was slow Corroborates findings that perceived system
speed is about finding what you want (Spool ‘00)
Summary Web site search needs improvement
Users want more organized results Our approach: integrate navigation with search
Metadata is being mixed and matched in interesting ways, but there are no guidelines on what works We are investigating how to design websites containing large sets
of items Architectural images Biomedical text
Preliminary results indicate that metadata organization is useful in some situations Depends on the type of search need
SummaryOur goals
Systematically determine what works, with the following emphases:
Task-centric Integrate metadata with search Dynamic previews Easily retrace steps
Develop recommendations that reflect both the task structure and the richness of the information structure
In future: integrate with more sophisticated displays
http://bailando.sims.berkeley.edu