COMP3410 DB32: Technologies for Knowledge Management Lecture 7: Query Broadening to improve IR By...
-
Upload
katelyn-murphy -
Category
Documents
-
view
221 -
download
0
Transcript of COMP3410 DB32: Technologies for Knowledge Management Lecture 7: Query Broadening to improve IR By...
COMP3410 DB32:Technologies for Knowledge Management
Lecture 7:
Query Broadening to improve IR
By Eric Atwell, School of Computing, University of Leeds
(including re-use of teaching resources from other sources, esp. Stuart Roberts, School of Computing, Univ of Leeds)
Module Objectives
“On completion of this module, students should be able to:
… describe classical and emerging information retrieval techniques, and their relevance to knowledge management; …”
Today’s objectives• first we look at a method for query broadening that
required input from the user
• then we look at an automatic method for query broadening using a thesaurus
• by the end of the lecture you should understand what a thesaurus, terminology-bank, ontology are, and how they are used to broaden queries
Some issues to be resolved• Synonyms
– football / soccer, tap / faucet: search for one, find both?
• homonyms– lead (metal or leash?), tap: find both, only want one?
• local/global contexts determine “good” terms– football articles: won’t mention word ‘football’;
will have particular meaning for the word ‘goal’
• Precoordination (proximity query): multi-word terms– “Venetian blind” vs “blind Venetian”
Evaluation/Effectiveness measures• effort - required by the users in formulation of queries
• time - between receipt of user query and production of list of ‘hits’
• presentation - of the output
• coverage - of the collection
• recall - the fraction of relevant items retrieved
• precision - the fraction of retrieved items that are relevant
• user satisfaction – with the retrieved items
Better hits: Query Broadening• User unaware of collection characteristics is likely to
formulate a ‘naïve’ query
• query broadening aims to replace the initial query with a new one featuring one or other of:– new index terms– adjusted term weights
• One method uses feedback information from the user
• Another method uses a thesaurus / term-bank / ontology
From response to initial query, gather relevance informationHR = RH = set of retrieved, relevant hitsHNR = H-R = set of retrieved, non-relevant hits
replace query q with replacement query q' :q' = q
di / |HR|
di / |HNR|
note: this moves the query vector closer to the centroid of the “relevant retrieved” document vectors and further from the centroid of the “non-relevant retrieved” documents.
di HNR
di HR
Relevance Feedback
Using terms from relevant documents• We expect documents that are similar to one another in
meaning (or usefulness) to have similar index terms.
• The system creates a replacement query (q’) based on q, but adds index terms that have been used to index known relevant documents, increases the relative weight of index terms in q that are also found in relevant documents, and reduces the weight of terms found in non-relevant documents.
How does this help?• It could help if documents were being missed because of the
synonym problem. The user uses the word ‘jam’, but some recipes use ‘jelly’ instead. Once a hit that uses ‘jelly’ has been recognized as relevant, then ‘jelly’ will appear n the next version of the query. Now hits may use ‘jelly’ but not ‘jam’.
• Conversely, it can help with the homonym problem. If the user wants references to ‘lead’ (the metal), and gets documents relating to dog-walking, then by marking the dog-walking references as not relevant, key words associated with dog-walking will be reduced in weight
pros and cons of feedback• If is set = 0, ignore non-relevant hits, a positive
feedback system; often preferred
• the feedback formula can be applied repeatedly, asking user for relevance information at each iteration
• relevance feedback is generally considered to be very effective for “high-use” systems
• one drawback is that it is not fully automatic.
Simple feedback example:
T = {pudding, jam, traffic, lane, treacle}
d1 = (0.8, 0.8, 0.0, 0.0, 0.4),
d2 = (0.0, 0.0, 0.9, 0.8, 0.0),
d3 = (0.8, 0.0, 0.0, 0.0, 0.8)
d4 = (0.6, 0.9, 0.5, 0.6, 0.0)
Recipe for jam pudding
DoT report on traffic lanes
Radio item on traffic jam in Pudding Lane
Recipe for treacle pudding
Display first 2 documents that match the following query:q = (1.0, 0.6, 0.0, 0.0, 0.0)
r = (0.91, 0.0, 0.6, 0.73)
Retrieved documents are:
d1 : Recipe for jam pudding
d4 : Radio item on traffic jam
relevant
not relevant
Suppose we set and to 0.5, to 0.2
q' = q di / | HR | di / | HNR|
= 0.5 q + 0.5 d1 0.2 d4
= 0.5 (1.0, 0.6, 0.0, 0.0, 0.0)+ 0.5 (0.8, 0.8, 0.0, 0.0, 0.4) 0.2 (0.6, 0.9, 0.5, 0.6, 0.0)
= (0.78, 0.52, 0.1, 0.12, 0.2)
(Note |Hn| = 1 and |Hnr| = 1)
di HR di HNR
Positive and Negative Feedback
Simple feedback example:
T = {pudding, jam, traffic, lane, treacle}
d1 = (0.8, 0.8, 0.0, 0.0, 0.4),
d2 = (0.0, 0.0, 0.9, 0.8, 0.0),
d3 = (0.8, 0.0, 0.0, 0.0, 0.8)
d4 = (0.6, 0.9, 0.5, 0.6, 0.0)
Display first 2 documents that match the following query:q’ = (0.78, 0.52, 0.1, 0.12, 0.2)
r’ = (0.96, 0.0, 0.86, 0.63) Retrieved documents are:
d1 : Recipe for jam pudding
d3 : Recipe for treacle pud
relevant
relevant
Thesaurus• a thesaurus or ontology may contain
– controlled vocabulary of terms or phrases describing a specific restricted topic,
– synonym classes, – hierarchy defining broader terms (hypernyms) and narrower
terms (hyponyms)– classes of ‘related’ terms.
• a thesaurus or ontology may be:– generic (as Roget’s thesaurus, or WordNet)– specific to a certain domain of knowledge, eg medical
Language normalisation
Content analysis
Uncontrolled keywords
Thesaurus
Index terms
User query
Normalised query
match
by replacing words from documents and query words with synonyms from a controlled language, we can improve precision and recall:
Thesaurus / Ontology construction
• Include terms likely to be of value in content analysis
• for each term, form classes of related words (separate classes for synonyms, hypernyms, hyponyms)
• form separate classes for each relevant meaning of the word
• terms in a class should occur with roughly equal frequency (not easy – NL has Zipf’s law word-freq )
• avoid high-frequency terms• it involves some expert judgment that will not be
easy to automate.
Example thesaurusA public-domain thesaurus (WORDNET) is available from:
http://www.cogsci.princeton.edu/~wn/
/home/cserv1_a/staff/nlplib/WordNet/2.0
/home/cserv1_a/staff/extras/nltk/1.4.2/corpora/wordnet
computer
data processor electronic computer
information processing system
synonyms (sense 1):
Example thesaurusA public-domain thesaurus (WORDNET) is available from:
http://www.cogsci.princeton.edu/~wn/
computercalculator
reckonerfigurer
estimator
synonyms (sense 2):
Hypernym is the generic term used to designate a whole class of specific instances. Y is a hypernym of X if X is a (kind of) Y.
Hyponym is the generic term used to designate a member of a class. X is a hyponym of Y if X is a (kind of) Y.
Coordinate words are words that have the same hypernym.
Hypernym synsets are preceded by "->", and hyponym synsets are preceded by "=>".
Terminology (from WordNet Help)
HypernymsSense 1computer, data processor, electronic computer, information processing system-> machine -> device -> instrumentality, instrumentation -> artifact, artefact -> object, physical object -> entity, something
Hypernym synsets are preceded by "->", and hyponym synsets are preceded by "=>".
HyponymsSense 1
computer, data processor, electronic computer, information processing system=> analog computer, analogue computer=> digital computer=> node, client, guest=> number cruncher=> pari-mutuel machine, totalizer, totaliser, totalizator, totalisator=> server, host
Hypernym synsets are preceded by "->", and hyponym synsets are preceded by "=>".
Sense 1computer, data processor, electronic computer, information processing system-> machine=> assembly=> calculator, calculating machine=> calendar=> cash machine, cash dispenser, automated teller machine, automatic teller machine, automated teller, automatic teller, ATM=> computer, data processor, electronic computer, information processing system=> concrete mixer, cement mixer=> corker=> cotton gin, gin=> decoder
Coordinate terms
Thesaurus use • replace term in document and/or query with term in
controlled language• replace term in query with related or broader term to
increase recall• suggest to user narrower terms to increase precision
Doc: <data processor>
Query: < electronic computer>
Thesaurus
computer (sense 1)
computer (sense 1)
match
S
Thesaurus use• replace term in document and/or query with term in
controlled language• replace term in query with related or broader term to
increase recall• suggest to user narrower terms to increase precision
Thesaurus
Query: <computer (sense 1)>
match
All collection
Query: <node(sense 6)>
match
All collectionB
Thesaurus use• replace term in document and/or query with term in
controlled language• replace term in query with related or broader term to
increase recall• suggest to user narrower terms to increase precision
Thesaurus
Query: client
match
All collection
match
All collectionN
Query: <computer (sense 1)>
User
Key points• a thesaurus or ontology can be used to normalise a
vocabulary and queries (?or documents?)
• it can be used (with some human intervention) to increase recall and precision
• generic thesaurus/ontology may not be effective in specialized collections and/or queries
• Semi-automatic construction of thesaurus/ontology based on the retrieved set of documents has produced some promising results.