September 2003 1 PROBABILISTIC CFGs & PROBABILISTIC PARSING Universita’ di Venezia 3 Ottobre 2003.
-
Upload
derick-douglas -
Category
Documents
-
view
214 -
download
0
Transcript of September 2003 1 PROBABILISTIC CFGs & PROBABILISTIC PARSING Universita’ di Venezia 3 Ottobre 2003.
![Page 1: September 2003 1 PROBABILISTIC CFGs & PROBABILISTIC PARSING Universita’ di Venezia 3 Ottobre 2003.](https://reader034.fdocuments.in/reader034/viewer/2022051517/56649cfa5503460f949cbf0f/html5/thumbnails/1.jpg)
September 2003 1
PROBABILISTIC CFGs &PROBABILISTIC PARSING
Universita’ di Venezia
3 Ottobre 2003
![Page 2: September 2003 1 PROBABILISTIC CFGs & PROBABILISTIC PARSING Universita’ di Venezia 3 Ottobre 2003.](https://reader034.fdocuments.in/reader034/viewer/2022051517/56649cfa5503460f949cbf0f/html5/thumbnails/2.jpg)
September 2003 2
Probabilistic CFGs
Context-Free Grammar Rules are of the form:– S NP VP
In a Probabilistic CFG, we assign a probability to these rules:– S NP VP, P(SNP,VP|S)
![Page 3: September 2003 1 PROBABILISTIC CFGs & PROBABILISTIC PARSING Universita’ di Venezia 3 Ottobre 2003.](https://reader034.fdocuments.in/reader034/viewer/2022051517/56649cfa5503460f949cbf0f/html5/thumbnails/3.jpg)
September 2003 3
Why PCFGs?
DISAMBIGUATION: with a PCFG, probabilities can be used to choose the most likely parse
ROBUSTNESS: rather than excluding things, a PCFG may assign them a very low probability
LEARNING: CFGs cannot be learned from positive data only
![Page 4: September 2003 1 PROBABILISTIC CFGs & PROBABILISTIC PARSING Universita’ di Venezia 3 Ottobre 2003.](https://reader034.fdocuments.in/reader034/viewer/2022051517/56649cfa5503460f949cbf0f/html5/thumbnails/4.jpg)
September 2003 4
An example of PCFG
![Page 5: September 2003 1 PROBABILISTIC CFGs & PROBABILISTIC PARSING Universita’ di Venezia 3 Ottobre 2003.](https://reader034.fdocuments.in/reader034/viewer/2022051517/56649cfa5503460f949cbf0f/html5/thumbnails/5.jpg)
September 2003 5
PCFGs in Prolog (courtesy Doug Arnold)
s(P0, [s,NP,VP] ) --> np(P1,NP),
vp(P2,VP),{ P0 is 1.0*P1*P2 }.
….vp(P0, [vp,V,NP] ) -->
v(P1,V),np(P2,NP ),{ P0 is 0.7*P1*P2 }.
![Page 6: September 2003 1 PROBABILISTIC CFGs & PROBABILISTIC PARSING Universita’ di Venezia 3 Ottobre 2003.](https://reader034.fdocuments.in/reader034/viewer/2022051517/56649cfa5503460f949cbf0f/html5/thumbnails/6.jpg)
September 2003 6
Notation and assumptions
![Page 7: September 2003 1 PROBABILISTIC CFGs & PROBABILISTIC PARSING Universita’ di Venezia 3 Ottobre 2003.](https://reader034.fdocuments.in/reader034/viewer/2022051517/56649cfa5503460f949cbf0f/html5/thumbnails/7.jpg)
September 2003 7
Independence assumptions
PCFGs specify a language model, just like n-grams
We need however to make some independence assumptions yet again: the probability of a subtree is independent of:
![Page 8: September 2003 1 PROBABILISTIC CFGs & PROBABILISTIC PARSING Universita’ di Venezia 3 Ottobre 2003.](https://reader034.fdocuments.in/reader034/viewer/2022051517/56649cfa5503460f949cbf0f/html5/thumbnails/8.jpg)
September 2003 8
The language model defined by PCFGs
![Page 9: September 2003 1 PROBABILISTIC CFGs & PROBABILISTIC PARSING Universita’ di Venezia 3 Ottobre 2003.](https://reader034.fdocuments.in/reader034/viewer/2022051517/56649cfa5503460f949cbf0f/html5/thumbnails/9.jpg)
September 2003 9
Using PCFGs to disambiguate: “Astronomers saw stars with ears”
![Page 10: September 2003 1 PROBABILISTIC CFGs & PROBABILISTIC PARSING Universita’ di Venezia 3 Ottobre 2003.](https://reader034.fdocuments.in/reader034/viewer/2022051517/56649cfa5503460f949cbf0f/html5/thumbnails/10.jpg)
September 2003 10
A second parse
![Page 11: September 2003 1 PROBABILISTIC CFGs & PROBABILISTIC PARSING Universita’ di Venezia 3 Ottobre 2003.](https://reader034.fdocuments.in/reader034/viewer/2022051517/56649cfa5503460f949cbf0f/html5/thumbnails/11.jpg)
September 2003 11
Choosing among the parses, and the sentence’s probability
![Page 12: September 2003 1 PROBABILISTIC CFGs & PROBABILISTIC PARSING Universita’ di Venezia 3 Ottobre 2003.](https://reader034.fdocuments.in/reader034/viewer/2022051517/56649cfa5503460f949cbf0f/html5/thumbnails/12.jpg)
September 2003 12
Parsing with PCFGs:A comparison with HMMs
An HMM defines a REGULAR GRAMMAR:
![Page 13: September 2003 1 PROBABILISTIC CFGs & PROBABILISTIC PARSING Universita’ di Venezia 3 Ottobre 2003.](https://reader034.fdocuments.in/reader034/viewer/2022051517/56649cfa5503460f949cbf0f/html5/thumbnails/13.jpg)
September 2003 13
Parsing with CFGs: A comparison with HMMs
![Page 14: September 2003 1 PROBABILISTIC CFGs & PROBABILISTIC PARSING Universita’ di Venezia 3 Ottobre 2003.](https://reader034.fdocuments.in/reader034/viewer/2022051517/56649cfa5503460f949cbf0f/html5/thumbnails/14.jpg)
September 2003 14
Inside and outside probabilities(cfr. forward and backward probabilities for HMMs)
![Page 15: September 2003 1 PROBABILISTIC CFGs & PROBABILISTIC PARSING Universita’ di Venezia 3 Ottobre 2003.](https://reader034.fdocuments.in/reader034/viewer/2022051517/56649cfa5503460f949cbf0f/html5/thumbnails/15.jpg)
September 2003 15
Parsing with probabilistic CFGs
![Page 16: September 2003 1 PROBABILISTIC CFGs & PROBABILISTIC PARSING Universita’ di Venezia 3 Ottobre 2003.](https://reader034.fdocuments.in/reader034/viewer/2022051517/56649cfa5503460f949cbf0f/html5/thumbnails/16.jpg)
September 2003 16
The algorithm
![Page 17: September 2003 1 PROBABILISTIC CFGs & PROBABILISTIC PARSING Universita’ di Venezia 3 Ottobre 2003.](https://reader034.fdocuments.in/reader034/viewer/2022051517/56649cfa5503460f949cbf0f/html5/thumbnails/17.jpg)
September 2003 17
Example
![Page 18: September 2003 1 PROBABILISTIC CFGs & PROBABILISTIC PARSING Universita’ di Venezia 3 Ottobre 2003.](https://reader034.fdocuments.in/reader034/viewer/2022051517/56649cfa5503460f949cbf0f/html5/thumbnails/18.jpg)
September 2003 18
Initialization
![Page 19: September 2003 1 PROBABILISTIC CFGs & PROBABILISTIC PARSING Universita’ di Venezia 3 Ottobre 2003.](https://reader034.fdocuments.in/reader034/viewer/2022051517/56649cfa5503460f949cbf0f/html5/thumbnails/19.jpg)
September 2003 19
Example
![Page 20: September 2003 1 PROBABILISTIC CFGs & PROBABILISTIC PARSING Universita’ di Venezia 3 Ottobre 2003.](https://reader034.fdocuments.in/reader034/viewer/2022051517/56649cfa5503460f949cbf0f/html5/thumbnails/20.jpg)
September 2003 20
Example
![Page 21: September 2003 1 PROBABILISTIC CFGs & PROBABILISTIC PARSING Universita’ di Venezia 3 Ottobre 2003.](https://reader034.fdocuments.in/reader034/viewer/2022051517/56649cfa5503460f949cbf0f/html5/thumbnails/21.jpg)
September 2003 21
Learning the probabilities: the Treebank
![Page 22: September 2003 1 PROBABILISTIC CFGs & PROBABILISTIC PARSING Universita’ di Venezia 3 Ottobre 2003.](https://reader034.fdocuments.in/reader034/viewer/2022051517/56649cfa5503460f949cbf0f/html5/thumbnails/22.jpg)
September 2003 22
Learning probabilities
Reconstruct the rules used in the analysis of the Treebank
Estimate probabilities by:
P(AB) = C(AB) / C(A)
![Page 23: September 2003 1 PROBABILISTIC CFGs & PROBABILISTIC PARSING Universita’ di Venezia 3 Ottobre 2003.](https://reader034.fdocuments.in/reader034/viewer/2022051517/56649cfa5503460f949cbf0f/html5/thumbnails/23.jpg)
September 2003 23
Probabilistic lexicalised PCFGs(Collins, 1997; Charniak, 2000)
![Page 24: September 2003 1 PROBABILISTIC CFGs & PROBABILISTIC PARSING Universita’ di Venezia 3 Ottobre 2003.](https://reader034.fdocuments.in/reader034/viewer/2022051517/56649cfa5503460f949cbf0f/html5/thumbnails/24.jpg)
September 2003 24
Parsing evaluation
![Page 25: September 2003 1 PROBABILISTIC CFGs & PROBABILISTIC PARSING Universita’ di Venezia 3 Ottobre 2003.](https://reader034.fdocuments.in/reader034/viewer/2022051517/56649cfa5503460f949cbf0f/html5/thumbnails/25.jpg)
September 2003 25
Performance of current parsers
![Page 26: September 2003 1 PROBABILISTIC CFGs & PROBABILISTIC PARSING Universita’ di Venezia 3 Ottobre 2003.](https://reader034.fdocuments.in/reader034/viewer/2022051517/56649cfa5503460f949cbf0f/html5/thumbnails/26.jpg)
September 2003 26
Readings
Manning and Schütze, chapters 11 and 12
![Page 27: September 2003 1 PROBABILISTIC CFGs & PROBABILISTIC PARSING Universita’ di Venezia 3 Ottobre 2003.](https://reader034.fdocuments.in/reader034/viewer/2022051517/56649cfa5503460f949cbf0f/html5/thumbnails/27.jpg)
September 2003 27
Acknowledgments
Some slides and the Prolog code are borrowed from Doug Arnold
Thanks also to Chris Manning & Diego Molla