Listeners use descriptive contrast to disambiguate novel ...
Align, Disambiguate, and Walk A Unified Approach for Measuring Semantic Similarity
description
Transcript of Align, Disambiguate, and Walk A Unified Approach for Measuring Semantic Similarity
![Page 1: Align, Disambiguate, and Walk A Unified Approach for Measuring Semantic Similarity](https://reader035.fdocuments.in/reader035/viewer/2022062521/568168ed550346895ddfeebe/html5/thumbnails/1.jpg)
Align, Disambiguate, and WalkA Unified Approach for Measuring Semantic Similarity
Mohammad Taher PilehvarDavid JurgensRoberto Navigli
![Page 2: Align, Disambiguate, and Walk A Unified Approach for Measuring Semantic Similarity](https://reader035.fdocuments.in/reader035/viewer/2022062521/568168ed550346895ddfeebe/html5/thumbnails/2.jpg)
Semantic Similarity; how
similar are a pair of lexical items?
![Page 3: Align, Disambiguate, and Walk A Unified Approach for Measuring Semantic Similarity](https://reader035.fdocuments.in/reader035/viewer/2022062521/568168ed550346895ddfeebe/html5/thumbnails/3.jpg)
Semantic Similarity
Sentence Level Word Level Sense Level
![Page 4: Align, Disambiguate, and Walk A Unified Approach for Measuring Semantic Similarity](https://reader035.fdocuments.in/reader035/viewer/2022062521/568168ed550346895ddfeebe/html5/thumbnails/4.jpg)
Semantic Similarity
Sentence Level Word Level Sense Level
![Page 5: Align, Disambiguate, and Walk A Unified Approach for Measuring Semantic Similarity](https://reader035.fdocuments.in/reader035/viewer/2022062521/568168ed550346895ddfeebe/html5/thumbnails/5.jpg)
Semantic Similarity
The worker was terminated
The boss fired him
Sentence level
Applications• Paraphrase recognition
(Tsatsaronis et al., 2010)
• MT evaluation (Kauchak and Barzilay, 2006)
• Question Answering (Surdeanu et al., 2011)
• Textual Entailment (Dagan et al., 2006)
![Page 6: Align, Disambiguate, and Walk A Unified Approach for Measuring Semantic Similarity](https://reader035.fdocuments.in/reader035/viewer/2022062521/568168ed550346895ddfeebe/html5/thumbnails/6.jpg)
Semantic Similarity
Sentence Level Word Level Sense Level
![Page 7: Align, Disambiguate, and Walk A Unified Approach for Measuring Semantic Similarity](https://reader035.fdocuments.in/reader035/viewer/2022062521/568168ed550346895ddfeebe/html5/thumbnails/7.jpg)
Semantic SimilarityWord level
fireplace
Applications• Lexical simplification (Biran et al., 2011)
Locuacious → Talkative
• Lexical substitution (McCarthy and Navigli, 2009)
heater
![Page 8: Align, Disambiguate, and Walk A Unified Approach for Measuring Semantic Similarity](https://reader035.fdocuments.in/reader035/viewer/2022062521/568168ed550346895ddfeebe/html5/thumbnails/8.jpg)
Semantic Similarity
Sentence Level Word Level Sense Level
![Page 9: Align, Disambiguate, and Walk A Unified Approach for Measuring Semantic Similarity](https://reader035.fdocuments.in/reader035/viewer/2022062521/568168ed550346895ddfeebe/html5/thumbnails/9.jpg)
Semantic Similarityfire sense #1 Sense level
fire sense #8
Applications• Coarsening sense inventories
(Snow et al., 2007)
• Semantic priming (Neely et al., 1989)
![Page 10: Align, Disambiguate, and Walk A Unified Approach for Measuring Semantic Similarity](https://reader035.fdocuments.in/reader035/viewer/2022062521/568168ed550346895ddfeebe/html5/thumbnails/10.jpg)
Exisiting Similarity Measures
Patwardan (2003)
Banerjee and Pederson (2003)
Hirst and St-Onge (1998)
Lin (1998)
Jiang and Conrath (1997)
Resnik (1995)
Sussna (1993, 1997)
Wu and Palmer (1994)
Leacock and Chodorow (1998)
Salton and McGill (1983)
Gabrilovich and Markovitch (2007)
Radinsky et al. (2011)
Ramage et al. (2009)
Yeh et al., (2009)
Turney (2007)
Landauer et al. (1998)
Allison and Dix (1986)Gusfield (1997)
Wise (1996)Keselj et al. (2003)
Sentence Word Sense
![Page 11: Align, Disambiguate, and Walk A Unified Approach for Measuring Semantic Similarity](https://reader035.fdocuments.in/reader035/viewer/2022062521/568168ed550346895ddfeebe/html5/thumbnails/11.jpg)
Exisiting Similarity Measures
Sense Word Sentence
Patwardan (2003)
Banerjee and Pederson (2003)
Hirst and St-Onge (1998)
Lin (1998)
Jiang and Conrath (1997)
Resnik (1995)
Sussna (1993, 1997)
Wu and Palmer (1994)
Leacock and Chodorow (1998)
Salton and McGill (1983)
Gabrilovich and Markovitch (2007)
Radinsky et al. (2011)
Ramage et al. (2009)
Yeh et al., (2009)
Turney (2007)
Landauer et al. (1998)
Allison and Dix (1986)Gusfield (1997)
Wise (1996)Keselj et al. (2003)
But
Different internal representationswhich are not comparable to each other
None directly covers all levels at the same time
Different output scales
![Page 12: Align, Disambiguate, and Walk A Unified Approach for Measuring Semantic Similarity](https://reader035.fdocuments.in/reader035/viewer/2022062521/568168ed550346895ddfeebe/html5/thumbnails/12.jpg)
Contribution
Sense Word sentence
State-of-the-art performance in each level Using only WordNet
A unified representationfor any lexical item
![Page 13: Align, Disambiguate, and Walk A Unified Approach for Measuring Semantic Similarity](https://reader035.fdocuments.in/reader035/viewer/2022062521/568168ed550346895ddfeebe/html5/thumbnails/13.jpg)
Advantage 1Unified representation
sense
word
sentencetext word
sense
All lexical itemsthis way
![Page 14: Align, Disambiguate, and Walk A Unified Approach for Measuring Semantic Similarity](https://reader035.fdocuments.in/reader035/viewer/2022062521/568168ed550346895ddfeebe/html5/thumbnails/14.jpg)
sense
word
Advantage 2Cross-level semantic similarity
A large and imposing house Mansion Residence#3
sentence
![Page 15: Align, Disambiguate, and Walk A Unified Approach for Measuring Semantic Similarity](https://reader035.fdocuments.in/reader035/viewer/2022062521/568168ed550346895ddfeebe/html5/thumbnails/15.jpg)
sentence
Advantage 3Sense-level operation
set of sensessense
word
sense
![Page 16: Align, Disambiguate, and Walk A Unified Approach for Measuring Semantic Similarity](https://reader035.fdocuments.in/reader035/viewer/2022062521/568168ed550346895ddfeebe/html5/thumbnails/16.jpg)
Outline
sense
word
text
Introduction Methodology Experiments
![Page 17: Align, Disambiguate, and Walk A Unified Approach for Measuring Semantic Similarity](https://reader035.fdocuments.in/reader035/viewer/2022062521/568168ed550346895ddfeebe/html5/thumbnails/17.jpg)
How Does it work?
lexical item 1 lexical item 2
semanticsignature 1
semanticsignature 2
![Page 18: Align, Disambiguate, and Walk A Unified Approach for Measuring Semantic Similarity](https://reader035.fdocuments.in/reader035/viewer/2022062521/568168ed550346895ddfeebe/html5/thumbnails/18.jpg)
Semantic Signature
sense word
Unified semantic signature
sentence
![Page 19: Align, Disambiguate, and Walk A Unified Approach for Measuring Semantic Similarity](https://reader035.fdocuments.in/reader035/viewer/2022062521/568168ed550346895ddfeebe/html5/thumbnails/19.jpg)
Semantic Signature
sense word
Unified semantic signature
Sense or set of senses
Alignment-based disambiguation
sentence
![Page 20: Align, Disambiguate, and Walk A Unified Approach for Measuring Semantic Similarity](https://reader035.fdocuments.in/reader035/viewer/2022062521/568168ed550346895ddfeebe/html5/thumbnails/20.jpg)
A woman is frying food
Semantic Signature
![Page 21: Align, Disambiguate, and Walk A Unified Approach for Measuring Semantic Similarity](https://reader035.fdocuments.in/reader035/viewer/2022062521/568168ed550346895ddfeebe/html5/thumbnails/21.jpg)
Semantic SignatureDistributional representation
over all synsets in WordNet
Importance of this synset (syn_4) for our lexical item
syn_1syn_2
syn_3syn_4
syn_5syn_6
syn_7syn_8
syn_9syn_n
. . .
![Page 22: Align, Disambiguate, and Walk A Unified Approach for Measuring Semantic Similarity](https://reader035.fdocuments.in/reader035/viewer/2022062521/568168ed550346895ddfeebe/html5/thumbnails/22.jpg)
Semantic Signaturea woman is frying food
syn_1syn_2
syn_3syn_4
syn_5syn_6
syn_7syn_8
syn_9syn_n
cooking#1
table#3frying_pan#1
oil#4kitchen#3
carpet#2
sugar#2
natural_gas#2
physics#1
ship#1
![Page 23: Align, Disambiguate, and Walk A Unified Approach for Measuring Semantic Similarity](https://reader035.fdocuments.in/reader035/viewer/2022062521/568168ed550346895ddfeebe/html5/thumbnails/23.jpg)
Personalized PageRank
![Page 24: Align, Disambiguate, and Walk A Unified Approach for Measuring Semantic Similarity](https://reader035.fdocuments.in/reader035/viewer/2022062521/568168ed550346895ddfeebe/html5/thumbnails/24.jpg)
Personalized PageRank
![Page 25: Align, Disambiguate, and Walk A Unified Approach for Measuring Semantic Similarity](https://reader035.fdocuments.in/reader035/viewer/2022062521/568168ed550346895ddfeebe/html5/thumbnails/25.jpg)
Personalized PageRank
vfry 2
nfood 1 nwoman 1
a woman is frying food{ , , }nwoman 1
nfood 1vfry 2
![Page 26: Align, Disambiguate, and Walk A Unified Approach for Measuring Semantic Similarity](https://reader035.fdocuments.in/reader035/viewer/2022062521/568168ed550346895ddfeebe/html5/thumbnails/26.jpg)
vfry2
nfood 1
ncooking 1
vcook 3
nfat 1
nfrench_fries 1
ndish 2nnutriment 1
nfood 2
nbeverage1
These weights form a semantic signature
![Page 27: Align, Disambiguate, and Walk A Unified Approach for Measuring Semantic Similarity](https://reader035.fdocuments.in/reader035/viewer/2022062521/568168ed550346895ddfeebe/html5/thumbnails/27.jpg)
Comparing Semantic Signatures
![Page 28: Align, Disambiguate, and Walk A Unified Approach for Measuring Semantic Similarity](https://reader035.fdocuments.in/reader035/viewer/2022062521/568168ed550346895ddfeebe/html5/thumbnails/28.jpg)
Comparing Semantic Signatures
• Parametric– Cosine
• Non-parametric– Weighted Overlap– Top-k Jaccard
![Page 29: Align, Disambiguate, and Walk A Unified Approach for Measuring Semantic Similarity](https://reader035.fdocuments.in/reader035/viewer/2022062521/568168ed550346895ddfeebe/html5/thumbnails/29.jpg)
Comparing Semantic SignaturesWeighted Overlap
syn_1 syn_2 syn_3 syn_4 syn_5 syn_6
syn_1 syn_2 syn_3 syn_4 syn_5 syn_6
![Page 30: Align, Disambiguate, and Walk A Unified Approach for Measuring Semantic Similarity](https://reader035.fdocuments.in/reader035/viewer/2022062521/568168ed550346895ddfeebe/html5/thumbnails/30.jpg)
Comparing Semantic SignaturesWeighted Overlap
syn_4 syn_5 syn_2 syn_6 syn_1 syn_3
syn_4 syn_5 syn_2 syn_6 syn_1 syn_3
![Page 31: Align, Disambiguate, and Walk A Unified Approach for Measuring Semantic Similarity](https://reader035.fdocuments.in/reader035/viewer/2022062521/568168ed550346895ddfeebe/html5/thumbnails/31.jpg)
Comparing Semantic SignaturesTop- Jaccard
syn_1 syn_2 syn_3 syn_4 syn_5 syn_6 syn_7 syn_8 syn_9 syn_10
syn_1 syn_2 syn_3 syn_4 syn_5 syn_6 syn_7 syn_8 syn_9 syn_10
𝑘=4
![Page 32: Align, Disambiguate, and Walk A Unified Approach for Measuring Semantic Similarity](https://reader035.fdocuments.in/reader035/viewer/2022062521/568168ed550346895ddfeebe/html5/thumbnails/32.jpg)
Comparing Semantic SignaturesTop- Jaccard
syn_1 syn_2 syn_3 syn_4 syn_5 syn_6 syn_7 syn_8 syn_9 syn_10
syn_1 syn_2 syn_3 syn_4 syn_5 syn_6 syn_7 syn_8 syn_9 syn_10
𝑘=4
![Page 33: Align, Disambiguate, and Walk A Unified Approach for Measuring Semantic Similarity](https://reader035.fdocuments.in/reader035/viewer/2022062521/568168ed550346895ddfeebe/html5/thumbnails/33.jpg)
Alignment-based disambiguation
sense word
Unified semantic signature
Sense or set of senses
Alignment-based disambiguation
sentence
![Page 34: Align, Disambiguate, and Walk A Unified Approach for Measuring Semantic Similarity](https://reader035.fdocuments.in/reader035/viewer/2022062521/568168ed550346895ddfeebe/html5/thumbnails/34.jpg)
Alignment-based disambiguation
sense word
Unified semantic signature
Sense or set of senses
Alignment-based disambiguation
sentencesentence
![Page 35: Align, Disambiguate, and Walk A Unified Approach for Measuring Semantic Similarity](https://reader035.fdocuments.in/reader035/viewer/2022062521/568168ed550346895ddfeebe/html5/thumbnails/35.jpg)
Why is disambiguation needed?
The worker was fired
He was terminated
![Page 36: Align, Disambiguate, and Walk A Unified Approach for Measuring Semantic Similarity](https://reader035.fdocuments.in/reader035/viewer/2022062521/568168ed550346895ddfeebe/html5/thumbnails/36.jpg)
A manager fired the worker.An employee was terminated from work by his boss.
Alignment-based disambiguation
![Page 37: Align, Disambiguate, and Walk A Unified Approach for Measuring Semantic Similarity](https://reader035.fdocuments.in/reader035/viewer/2022062521/568168ed550346895ddfeebe/html5/thumbnails/37.jpg)
A manager fired the worker.An employee was terminated from work by his boss.
Alignment-based disambiguation
![Page 38: Align, Disambiguate, and Walk A Unified Approach for Measuring Semantic Similarity](https://reader035.fdocuments.in/reader035/viewer/2022062521/568168ed550346895ddfeebe/html5/thumbnails/38.jpg)
managern
employeen
firev workern terminatev bossnworkn
Alignment-based disambiguation
![Page 39: Align, Disambiguate, and Walk A Unified Approach for Measuring Semantic Similarity](https://reader035.fdocuments.in/reader035/viewer/2022062521/568168ed550346895ddfeebe/html5/thumbnails/39.jpg)
manager terminate work boss
bosswork
. . .
terminate
terminate
terminate
employee
work
. . .
nmanager 2
n1
n1
v1
v2
v3
v4
n2
n1
n3
n1
n2
fire worker
fire
fire
worker
. . .
v1
v2
v3
n1
n2
managern
employeen
firev workern terminatev bossnworkn
. . .
n
Sentence 1 Sentence 2
Alignment-based disambiguation
![Page 40: Align, Disambiguate, and Walk A Unified Approach for Measuring Semantic Similarity](https://reader035.fdocuments.in/reader035/viewer/2022062521/568168ed550346895ddfeebe/html5/thumbnails/40.jpg)
manager terminate work boss
bosswork
. . .
terminate
terminate
terminate
employee
work
. . .
nmanager 2
n1
n1
v1
v2
v3
v4
n2
n1
n3
n1
n2
fire worker
fire
fire
worker
. . .
v1
v2
v3
n1
n2
managern
employeen
firev workern terminatev bossnworkn
. . .
n
Sentence 1 Sentence 2
Alignment-based disambiguation
![Page 41: Align, Disambiguate, and Walk A Unified Approach for Measuring Semantic Similarity](https://reader035.fdocuments.in/reader035/viewer/2022062521/568168ed550346895ddfeebe/html5/thumbnails/41.jpg)
manager
terminate work boss
bosswork
. . .
terminate
terminate
terminate
employee
work
. . .
nmanager 2
n1
n1
v1
v2
v3
v4
n2
n1
n3
n1
n2
n
Alignment-based disambiguation
0.50.3
Tversky (1977)Markman and Gentner (1993)
![Page 42: Align, Disambiguate, and Walk A Unified Approach for Measuring Semantic Similarity](https://reader035.fdocuments.in/reader035/viewer/2022062521/568168ed550346895ddfeebe/html5/thumbnails/42.jpg)
terminate work boss
bosswork
. . .
terminate
terminate
terminate
employee
work
. . .
n1
v1
v2
v3
v4
n2
n1
n3
n1
n2
Alignment-based disambiguation
0.3
nmanager 2n
managern1
0.50.3
![Page 43: Align, Disambiguate, and Walk A Unified Approach for Measuring Semantic Similarity](https://reader035.fdocuments.in/reader035/viewer/2022062521/568168ed550346895ddfeebe/html5/thumbnails/43.jpg)
manager terminate work boss
bosswork
. . .
terminate
terminate
terminate
employee
work
. . .
nmanager 2
n1
n1
v1
v2
v3
v4
n2
n1
n3
n1
n2
fire worker
fire
fire
worker
. . .
v1
v2
v3
n1
n2
managern
employeen
firev workern terminatev bossnworkn
. . .
n
Sentence 1 Sentence 2
fire v4
Alignment-based disambiguation
![Page 44: Align, Disambiguate, and Walk A Unified Approach for Measuring Semantic Similarity](https://reader035.fdocuments.in/reader035/viewer/2022062521/568168ed550346895ddfeebe/html5/thumbnails/44.jpg)
manager terminate work boss
bosswork
. . .
terminate
terminate
terminate
employee
work
. . .
nmanager 2
n1
n1
v1
v2
v3
v4
n2
n1
n3
n1
n2
fire worker
fire
fire
worker
. . .
v1
v2
v3
n1
n2
managern
employeen
firev workern terminatev bossnworkn
. . .
n
Sentence 1 Sentence 2
fire v4
Alignment-based disambiguation
![Page 45: Align, Disambiguate, and Walk A Unified Approach for Measuring Semantic Similarity](https://reader035.fdocuments.in/reader035/viewer/2022062521/568168ed550346895ddfeebe/html5/thumbnails/45.jpg)
Outline
sense
word
text
Introduction Methodology Experiments
![Page 46: Align, Disambiguate, and Walk A Unified Approach for Measuring Semantic Similarity](https://reader035.fdocuments.in/reader035/viewer/2022062521/568168ed550346895ddfeebe/html5/thumbnails/46.jpg)
Experiments• Sentence level– Semantic Textual Similarity (SemEval-2012)
![Page 47: Align, Disambiguate, and Walk A Unified Approach for Measuring Semantic Similarity](https://reader035.fdocuments.in/reader035/viewer/2022062521/568168ed550346895ddfeebe/html5/thumbnails/47.jpg)
Experiments• Sentence level– Semantic Textual Similarity (SemEval-2012)
• Word level– Synonymy recognition (TOEFL dataset)– Correlation-based (RG-65 dataset)
![Page 48: Align, Disambiguate, and Walk A Unified Approach for Measuring Semantic Similarity](https://reader035.fdocuments.in/reader035/viewer/2022062521/568168ed550346895ddfeebe/html5/thumbnails/48.jpg)
Experiments• Sentence level– Semantic Textual Similarity (SemEval-2012)
• Word level– Synonymy recognition (TOEFL dataset)– Correlation-based (RG-65 dataset)
• Sense level– Coarsening WordNet sense inventory
![Page 49: Align, Disambiguate, and Walk A Unified Approach for Measuring Semantic Similarity](https://reader035.fdocuments.in/reader035/viewer/2022062521/568168ed550346895ddfeebe/html5/thumbnails/49.jpg)
Experiment 1Similarity at Sentence level
• Semantic Textual Similarity (STS-12)– 5 datasets – Three evaluation measures
• ALL, ALLnrm, and Mean
• Top-ranking systems– UKP2 (Bär et al., 2012)– TLSim and TLSyn (Šarić et al., 2012)
![Page 50: Align, Disambiguate, and Walk A Unified Approach for Measuring Semantic Similarity](https://reader035.fdocuments.in/reader035/viewer/2022062521/568168ed550346895ddfeebe/html5/thumbnails/50.jpg)
Experiment 1Similarity at Sentence level
Features– Main features
• Cosine• Weighted Overlap• Top-k Jaccard
– String-based features• Longest common substring• Longest common subsequence• Greedy string tiling• Character/word n-grams
![Page 51: Align, Disambiguate, and Walk A Unified Approach for Measuring Semantic Similarity](https://reader035.fdocuments.in/reader035/viewer/2022062521/568168ed550346895ddfeebe/html5/thumbnails/51.jpg)
STS Results
Experiment 1Similarity at Sentence level
ALL
ALLnrm
Mean
0.6 0.65 0.7 0.75 0.8 0.85 0.9
ADW
ADW
ADW
UKP2
UKP2
UKP2
TLsyn
TLsyn
TLsyn
TLsim
TLsim
TLsim
+0.034
+0.007
+0.043
![Page 52: Align, Disambiguate, and Walk A Unified Approach for Measuring Semantic Similarity](https://reader035.fdocuments.in/reader035/viewer/2022062521/568168ed550346895ddfeebe/html5/thumbnails/52.jpg)
Experiments• Sentence level– Semantic Textual Similarity (Semeval-12)
• Word level– Synonymy recognition (TOEFL dataset)– Correlation-based (RG-65 dataset)
• Sense level– Coarsening WordNet sense inventory
![Page 53: Align, Disambiguate, and Walk A Unified Approach for Measuring Semantic Similarity](https://reader035.fdocuments.in/reader035/viewer/2022062521/568168ed550346895ddfeebe/html5/thumbnails/53.jpg)
Experiment 2Similarity at Word Level
Synonymy recognition Correlating word similarity judgments
![Page 54: Align, Disambiguate, and Walk A Unified Approach for Measuring Semantic Similarity](https://reader035.fdocuments.in/reader035/viewer/2022062521/568168ed550346895ddfeebe/html5/thumbnails/54.jpg)
Similarity at Word LevelSynonym Recognition
• TOEFL dataset (Landauer and Dumais, 1997)
– 80 multiple choice questions– Human test takers: 64.5% only
![Page 55: Align, Disambiguate, and Walk A Unified Approach for Measuring Semantic Similarity](https://reader035.fdocuments.in/reader035/viewer/2022062521/568168ed550346895ddfeebe/html5/thumbnails/55.jpg)
Similarity at Word LevelSynonym Recognition
Accuracy on TOEFL dataset
PPMIC (Bullinaria and Levy, 2007)
GLSA (Matveeva et al., 2005)
LSA (Rapp, 2003)
ADW_Jaccard
ADW_WO
ADW_Cosine
PR (Turney et al., 2003)
PCCP (Bullinaria and Levy, 2012)
75 80 85 90 95 100
![Page 56: Align, Disambiguate, and Walk A Unified Approach for Measuring Semantic Similarity](https://reader035.fdocuments.in/reader035/viewer/2022062521/568168ed550346895ddfeebe/html5/thumbnails/56.jpg)
Similarity at Word LevelJudgment Correlation
• Dataset: RG-65 (Rubenstein and Goodenough, 1965)
– 65 word pairs • judged by 51 human subjects
– Scale of 0 → 4
![Page 57: Align, Disambiguate, and Walk A Unified Approach for Measuring Semantic Similarity](https://reader035.fdocuments.in/reader035/viewer/2022062521/568168ed550346895ddfeebe/html5/thumbnails/57.jpg)
Similarity at Word LevelJudgment Correlation
Spearman correlation, RG-65 dataset
ADW_Cosine
Agirre et al. (2009)
Hughes and Ramage (2007)
Zesch et al. (2008)
ADW_Jaccard
ADW_WO
0.76 0.78 0.8 0.82 0.84 0.86 0.88 0.9
+0.03
![Page 58: Align, Disambiguate, and Walk A Unified Approach for Measuring Semantic Similarity](https://reader035.fdocuments.in/reader035/viewer/2022062521/568168ed550346895ddfeebe/html5/thumbnails/58.jpg)
Experiments• Text level– Semantic Textual Similarity (Semeval-12)
• Word level– Synonymy recognition (TOEFL dataset)– Correlation-based (RG-65 dataset)
• Sense level– Coarsening WordNet sense inventory
![Page 59: Align, Disambiguate, and Walk A Unified Approach for Measuring Semantic Similarity](https://reader035.fdocuments.in/reader035/viewer/2022062521/568168ed550346895ddfeebe/html5/thumbnails/59.jpg)
Experiment 3Similarity at Sense Level
• Coarse-graining WordNet
Snow et al. (2007)Navigli (2006)
![Page 60: Align, Disambiguate, and Walk A Unified Approach for Measuring Semantic Similarity](https://reader035.fdocuments.in/reader035/viewer/2022062521/568168ed550346895ddfeebe/html5/thumbnails/60.jpg)
Experiment 3Similarity at Sense Level
• Binary classification: Merged or not-merged
bass#1
bass#3
bass#2
bass#4
Senseval-2 English Lexical Sample WSD
task(Kilgarriff, 2001)
OntoNotes (Hovy et al., 2006)
about 3500 sense pairs (Noun)about 5000 sense pairs (Verb)
about 16000 sense pairs (Noun)about 31000 sense pairs (Verb)
![Page 61: Align, Disambiguate, and Walk A Unified Approach for Measuring Semantic Similarity](https://reader035.fdocuments.in/reader035/viewer/2022062521/568168ed550346895ddfeebe/html5/thumbnails/61.jpg)
Experiment 3Similarity at Sense Level
• Binary classification: Merge or not-merged
Merged
Not-merged
Tuned using 10% of the dataset
if similarity ≥ 𝑡
h𝑜𝑡 𝑒𝑟𝑤𝑖𝑠𝑒
![Page 62: Align, Disambiguate, and Walk A Unified Approach for Measuring Semantic Similarity](https://reader035.fdocuments.in/reader035/viewer/2022062521/568168ed550346895ddfeebe/html5/thumbnails/62.jpg)
Experiment 3Similarity at Sense Level
F-score on OntoNotes dataset
ADW_Cosine
ADW_WO
ADW_Jaccard
Snow et al. (2007)
Navigli (2006)
0 0.2 0.4 0.6
0.41
0.42
0.42
0.37
0.22
Noun
0 0.2 0.4 0.6
0.52
0.54
0.53
0.45
0.37
Verb
![Page 63: Align, Disambiguate, and Walk A Unified Approach for Measuring Semantic Similarity](https://reader035.fdocuments.in/reader035/viewer/2022062521/568168ed550346895ddfeebe/html5/thumbnails/63.jpg)
Experiment 3Similarity at Sense Level
F-score on Senseval-2 dataset
ADW_Cosine
ADW_WO
ADW_Jaccard
Snow et al. (2007)
Navigli (2006)
0 0.2 0.4 0.6
0.44
0.47
0.47
0.42
0.37
Noun
0 0.1 0.2 0.3 0.4 0.5 0.6
0.49
0.5
0.49
0.43
0.29
Verb
![Page 64: Align, Disambiguate, and Walk A Unified Approach for Measuring Semantic Similarity](https://reader035.fdocuments.in/reader035/viewer/2022062521/568168ed550346895ddfeebe/html5/thumbnails/64.jpg)
Conclusions• A unified approach for computing semantic
similarity for any pair of lexical items
• Experiments with SOA performance– Sense level (sense coarsening)– Word level (synonymy recognition and judgment)– Sentence level (Semantic Textual Similarity)
![Page 65: Align, Disambiguate, and Walk A Unified Approach for Measuring Semantic Similarity](https://reader035.fdocuments.in/reader035/viewer/2022062521/568168ed550346895ddfeebe/html5/thumbnails/65.jpg)
Future Direction• Larger sense inventories (e.g., BabelNet)
• Cross-level semantic similarity
• Create datasets for cross-level similarity– Future Semeval task?
![Page 66: Align, Disambiguate, and Walk A Unified Approach for Measuring Semantic Similarity](https://reader035.fdocuments.in/reader035/viewer/2022062521/568168ed550346895ddfeebe/html5/thumbnails/66.jpg)
nlistening 1
nthank_you 1
ngratitude 1
nthanks 1
nexpression 3
vheed 1nsensing 2
vlisten 2
nauscultation 1near 2
nhearing 6
jaudio-lingual 1
vlisten 1
Thank you for listening!