Jiuling Zhang2009-03-04

Why perform query expansion?WordNet based Word Sense

Disambiguation WordNet Word Sense DisambiguationConceptual Query ExpansionExperimentsConclusionFuture workReferences

queries are baffled with:

Incompleteness lack of enough knowledge

Vaguenessuncertainties inherent to natural

languages: synonymous & polysemous

Spelling errors

How to handle this problem?

Query expansion/reformulation with a thesaurus

Query expansion by automatic thesaurus generation

Spelling correction

Examples & recent researches:

Qiu introduced query expansion based on concept[1]

Smeaton tried to expand weighting and word sense disambiguation techniques[2]

Hoeber manually constructed a concept network to expand[3]

A large manually constructed comprehensive thesaurus developed at Princeton[4]

WordNet is organized into a network of synonyms(synsets)

A synset is basic element in WordNet and words of the same synset are exchangeable in some context

Example: word vs. synset

wn good -synsnSense 1good => advantage, vantageSense 2good, goodness => moralitySense 3good, goodness => qualitySense 4commodity, trade good, good => artifact, artifact

Word Sense Disambiguationsupervised Word Sense Disambiguation:

relies on a sense-tagged corpusunsupervised Word Sense

Disambiguation: relies on a machine readable thesaurus instead of sense-tagged corpus[5]

Our method belongs to the latter one

Components:

WordNet

WordNet modules:WordNet::SenseRelate::AllWords[6] WordNet::QueryData[7] Lingua::WordNet[8]

The adapted Lesk measure is employed.

Adapted Lesk algorithm is higher version of the Lesk method by counting the number of overlaps not only of glosses of synonymy but also of glosses of other related synsets, hyponymy, meronymy, troponymy e.g. [5]

Procedure:

I. PreprocessII. Perform WSD to queryIII. Combine new terms to obtain new queriesIV. Perform WSD to obtained queriesV. Compare synset array & select expanded

queries

The fl

ow

chart o

f query

expansio

n

pro

cedu

re

Short queries are avoided

Expanded queries are to the Google and evaluated by 10 persons

Precision@10 are recorded

Results:

Query_num Original query Expanded query Increment

1 0.6 0.6625 0.0625

2 0.675 0.6125 -0.0625

3 0.65 0.70 0.05

4 0.5125 0.5375 0.025

5 0.625 0.7625 0.1365

Avg@10 0.6125 0.655 0.0425

Propose a new concept based query expansion using WordNet and WSD

Experimental results show it can improve effectiveness

Apply the similar idea to sentences in documents to perform documents expansion

Employing Markov language model to modify newly generated queries[9]

Experiments on TREC Web Track collections

[1] Qiu, Y., Frei, H.-P.: Concept based query expansion. In Proceedings of the 16th annual international ACM SIGIR conference on Research and development in information retrieval. ACM Press, Pittsburgh, Pennsylvania, USA (1993) 160-169

[2] R. Richardson, AF Smeaton.: Using WordNet in a Knowledge-Based Approach to Information Retrieval. Proceedings of the BCS-IRSG Colloquium, Crewe(1995)

[3] Hoeber, X.-D. Yang, and Y. Yao.: Conceptual query expansion. In Proceedings of the Atlantic Web Intelligence Conference (2005)

[4] Miller, G., R. Beckwith, C. Fellbaum, D. Gross, and K. Miller.: Five papers on WordNet. CSL Report 43, Cognitive Science Laboratory, Princeton University(1990)

[5] Patwardhan, S., Banerjee, S., Pedersen, T.: UMND1: Unsupervised Word Sense Disambiguation Using Contextual Semantic Relatedness. In the Proceedings of SemEval-2007: 4th International Workshop on Semantic Evaluations(2007) 390-393

[6] http://search.cpan.org/~jrennie/WordNet-SenseRelate-AllWords/

[7] http://search.cpan.org/~jrennie/WordNet-QueryData-1.47/

[8] http://search.cpan.org/~dbrian/Lingua-Wordnet-0.74/

[9] Shuang Liu, Fang Liu, Clement Yu Weiyi Meng.: An Effective Approach to Document Retrieval via Utilizing WordNet and Recognizing Phrases. In Proceedings of the 27th Annual International ACM/SIGIR Conference on Research and development in information retrieval, Sheffield, Yorkshire, UK(2004)

Thanks!