Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type...
Transcript of Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type...
![Page 1: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/1.jpg)
Context Dependent Fine Grained Entity Typing
Rajarshi Das
June 29, 2016Reading Group Presentation
![Page 2: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/2.jpg)
Entity Type Classification
![Page 3: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/3.jpg)
Entity Type Classification• can be defined as the task of assigning semantic labels
to mentions of entities in documents (such as ‘person’, ‘location’, ‘organization’, ‘misc.’)
![Page 4: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/4.jpg)
Entity Type Classification• can be defined as the task of assigning semantic labels
to mentions of entities in documents (such as ‘person’, ‘location’, ‘organization’, ‘misc.’)
• Entity types are useful for a variety of related natural language tasks such as coreference resolution (Recasens et al., 2013), relation extraction (Yao et al. 2010; Ling and Weld 2012)
![Page 5: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/5.jpg)
Entity Type Classification• can be defined as the task of assigning semantic labels
to mentions of entities in documents (such as ‘person’, ‘location’, ‘organization’, ‘misc.’)
• Entity types are useful for a variety of related natural language tasks such as coreference resolution (Recasens et al., 2013), relation extraction (Yao et al. 2010; Ling and Weld 2012)
• They have also increased the performance of several downstream applications such as Question Answering (Lin et al., 2012) and Knowledge Base Completion (Carlson et al., 2010; Das et al., 2016)
![Page 6: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/6.jpg)
Fine-grained Entity Types
![Page 7: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/7.jpg)
• Standard type classification task just use few coarse labels (such as person, location, organization, etc)
Fine-grained Entity Types
![Page 8: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/8.jpg)
• Standard type classification task just use few coarse labels (such as person, location, organization, etc)
• Recent work has focused on much larger set of fine grained labels (Ling and Weld, 2012; Yosef et al., 2012; Del Corro et al, 2015; inter-alia)
Fine-grained Entity Types
![Page 9: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/9.jpg)
• Standard type classification task just use few coarse labels (such as person, location, organization, etc)
• Recent work has focused on much larger set of fine grained labels (Ling and Weld, 2012; Yosef et al., 2012; Del Corro et al, 2015; inter-alia)
• Today I will discuss few recent paper about context dependent fine grained entity types…
Fine-grained Entity Types
![Page 10: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/10.jpg)
Dan Gillick, Nevena Lazic, Kuzman Ganchev, Jesse Kirchner, David Huynh Google
Context-Dependent Fine-Grained Entity Type Tagging
![Page 11: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/11.jpg)
Why context dependent types?
![Page 12: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/12.jpg)
Why context dependent types?
![Page 13: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/13.jpg)
Why context dependent types?
Types in YAGO
![Page 14: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/14.jpg)
Why context dependent types?
![Page 15: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/15.jpg)
Why context dependent types?
![Page 16: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/16.jpg)
Why context dependent types?
Types in YAGO
![Page 17: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/17.jpg)
Why context dependent types?
![Page 18: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/18.jpg)
Why context dependent types?
Are all of these types always relevant?
![Page 19: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/19.jpg)
Why context dependent types?
Are all of these types always relevant?In April 1986, he won election as mayor (a
nonpartisan position) of his adopted hometown, Carmel-by-the-Sea, California
![Page 20: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/20.jpg)
Why context dependent types?
Are all of these types always relevant?In April 1986, he won election as mayor (a
nonpartisan position) of his adopted hometown, Carmel-by-the-Sea, California
Relevant Types: mayor, politician, legislator
![Page 21: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/21.jpg)
Why context dependent types?
Are all of these types always relevant?In April 1986, he won election as mayor (a
nonpartisan position) of his adopted hometown, Carmel-by-the-Sea, California
Eastwood has been recognized with multiple awards and nominations for his work in film,
television, and music.
Relevant Types: mayor, politician, legislator
![Page 22: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/22.jpg)
Why context dependent types?
Are all of these types always relevant?In April 1986, he won election as mayor (a
nonpartisan position) of his adopted hometown, Carmel-by-the-Sea, California
Eastwood has been recognized with multiple awards and nominations for his work in film,
television, and music.
Relevant Types: mayor, politician, legislator
Relevant Types: film director, producer, entertainer
![Page 23: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/23.jpg)
Context Dependent Types
![Page 24: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/24.jpg)
Context Dependent Types
• Context Dependent Fine Type tagging: set of acceptable types for an entity mention are only those that are relevant to the local context
![Page 25: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/25.jpg)
Context Dependent Types
• Context Dependent Fine Type tagging: set of acceptable types for an entity mention are only those that are relevant to the local context
If a hostile predator emerges for Saatchi & Saatchi Co., cofounders
Charles and Maurice Saatchi will lead…..
![Page 26: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/26.jpg)
Context Dependent Types
• Context Dependent Fine Type tagging: set of acceptable types for an entity mention are only those that are relevant to the local context
Entity Mentions: predators, Saatchi & Saatchi, Charles & Mau…
If a hostile predator emerges for Saatchi & Saatchi Co., cofounders
Charles and Maurice Saatchi will lead…..
![Page 27: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/27.jpg)
Context Dependent Types
• Context Dependent Fine Type tagging: set of acceptable types for an entity mention are only those that are relevant to the local context
Fine Types: predator, Saatchi & Saatchi Co. : organization/companyCharles and Maurice Saatchi : person/business
Entity Mentions: predators, Saatchi & Saatchi, Charles & Mau…
If a hostile predator emerges for Saatchi & Saatchi Co., cofounders
Charles and Maurice Saatchi will lead…..
![Page 28: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/28.jpg)
Context Dependent Types
• Context Dependent Fine Type tagging: set of acceptable types for an entity mention are only those that are relevant to the local context
Fine Types: predator, Saatchi & Saatchi Co. : organization/companyCharles and Maurice Saatchi : person/business
Entity Mentions: predators, Saatchi & Saatchi, Charles & Mau…
If a hostile predator emerges for Saatchi & Saatchi Co., cofounders
Charles and Maurice Saatchi will lead…..
Note: Without the context “predator” would not be
identified as an “organization”
![Page 29: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/29.jpg)
Type Labels and Manual Annotation
![Page 30: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/30.jpg)
Type Labels and Manual Annotation• The type labels are derived from Freebase
• They are also organized into a hierarchy. For examples an “athlete” type is “person/athlete”
![Page 31: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/31.jpg)
Type Labels and Manual Annotation• The type labels are derived from Freebase
• They are also organized into a hierarchy. For examples an “athlete” type is “person/athlete”
![Page 32: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/32.jpg)
Type Labels and Manual Annotation
![Page 33: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/33.jpg)
Type Labels and Manual Annotation
4 commonly used coarse types
![Page 34: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/34.jpg)
Type Labels and Manual Annotation
![Page 35: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/35.jpg)
Type Labels and Manual AnnotationLevel II types - person/athlete
organiztion/company
![Page 36: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/36.jpg)
Type Labels and Manual Annotation
![Page 37: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/37.jpg)
Type Labels and Manual AnnotationLevel III types -
person/artist/actor other/event/election
![Page 38: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/38.jpg)
Manual Annotation
![Page 39: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/39.jpg)
Manual Annotation
• All news documents from the OntoNotes test set were manually annotated for context dependent types.
![Page 40: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/40.jpg)
Manual Annotation
• All news documents from the OntoNotes test set were manually annotated for context dependent types.
![Page 41: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/41.jpg)
Manual Annotation
• All news documents from the OntoNotes test set were manually annotated for context dependent types.
• Each document annotated by 6 annotators
• More annotations at the top level • More disagreement at the bottom level
(Specificity) • Some disagreements at type level too
(Type)
![Page 42: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/42.jpg)
Distant Supervision for Training
![Page 43: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/43.jpg)
Distant Supervision for Training• The data described before was the test data
![Page 44: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/44.jpg)
Distant Supervision for Training• The data described before was the test data
• For training:
![Page 45: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/45.jpg)
Distant Supervision for Training• The data described before was the test data
• For training:
• 133K news docs as training corpus.
![Page 46: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/46.jpg)
Distant Supervision for Training• The data described before was the test data
• For training:
• 133K news docs as training corpus.
• Run standard NLP pipeline to detect entity mentions.
![Page 47: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/47.jpg)
Distant Supervision for Training• The data described before was the test data
• For training:
• 133K news docs as training corpus.
• Run standard NLP pipeline to detect entity mentions.
• POS tagger -> Dependency Parser -> NP extractor
![Page 48: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/48.jpg)
Distant Supervision for Training• The data described before was the test data
• For training:
• 133K news docs as training corpus.
• Run standard NLP pipeline to detect entity mentions.
• POS tagger -> Dependency Parser -> NP extractor
• Assign Freebase types to the resolved entities.
![Page 49: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/49.jpg)
Distant Supervision for Training• The data described before was the test data
• For training:
• 133K news docs as training corpus.
• Run standard NLP pipeline to detect entity mentions.
• POS tagger -> Dependency Parser -> NP extractor
• Assign Freebase types to the resolved entities.
• Problem with this approach?
![Page 50: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/50.jpg)
Distant Supervision for Training• The data described before was the test data
• For training:
• 133K news docs as training corpus.
• Run standard NLP pipeline to detect entity mentions.
• POS tagger -> Dependency Parser -> NP extractor
• Assign Freebase types to the resolved entities.
• Problem with this approach?Ty
![Page 51: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/51.jpg)
Distant Supervision for Training• The data described before was the test data
• For training:
• 133K news docs as training corpus.
• Run standard NLP pipeline to detect entity mentions.
• POS tagger -> Dependency Parser -> NP extractor
• Assign Freebase types to the resolved entities.
• Problem with this approach?Ty
• Barack Obama will have all the types associated with him i.e. mentions will not have context dependent types
![Page 52: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/52.jpg)
Distant Supervision for Training• The data described before was the test data
• For training:
• 133K news docs as training corpus.
• Run standard NLP pipeline to detect entity mentions.
• POS tagger -> Dependency Parser -> NP extractor
• Assign Freebase types to the resolved entities.
• Problem with this approach?Ty
• Barack Obama will have all the types associated with him i.e. mentions will not have context dependent types
• Reduce the mismatch between training and test set by applying some heuristics.
![Page 53: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/53.jpg)
Distant Supervision for Training• The data described before was the test data
• For training:
• 133K news docs as training corpus.
• Run standard NLP pipeline to detect entity mentions.
• POS tagger -> Dependency Parser -> NP extractor
• Assign Freebase types to the resolved entities.
• Problem with this approach?Ty
• Barack Obama will have all the types associated with him i.e. mentions will not have context dependent types
• Reduce the mismatch between training and test set by applying some heuristics.
•
![Page 54: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/54.jpg)
Heuristics
![Page 55: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/55.jpg)
Heuristics• Sibling Pruning: remove sibling types associated with a single entity,
leaving only the parent type i.e. a mention with types ‘person/athlete’ and ‘person/political-figure’ will be pruned to just ‘person’.
![Page 56: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/56.jpg)
Heuristics• Sibling Pruning: remove sibling types associated with a single entity,
leaving only the parent type i.e. a mention with types ‘person/athlete’ and ‘person/political-figure’ will be pruned to just ‘person’.
• Motivation: Uncommon for several sibling types to be relevant in the same context.
![Page 57: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/57.jpg)
Heuristics• Sibling Pruning: remove sibling types associated with a single entity,
leaving only the parent type i.e. a mention with types ‘person/athlete’ and ‘person/political-figure’ will be pruned to just ‘person’.
• Motivation: Uncommon for several sibling types to be relevant in the same context.
• Less common entities associated with few freebase types are better as training data as they are annotated with types relevant from context.
![Page 58: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/58.jpg)
Heuristics• Sibling Pruning: remove sibling types associated with a single entity,
leaving only the parent type i.e. a mention with types ‘person/athlete’ and ‘person/political-figure’ will be pruned to just ‘person’.
• Motivation: Uncommon for several sibling types to be relevant in the same context.
• Less common entities associated with few freebase types are better as training data as they are annotated with types relevant from context.
• Coarse Type Pruning: Remove types that do not agree with the output of a standard coarse grained classifier trained on {‘person’, ‘location’, ‘organization’, ‘other’}
![Page 59: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/59.jpg)
Heuristics• Sibling Pruning: remove sibling types associated with a single entity,
leaving only the parent type i.e. a mention with types ‘person/athlete’ and ‘person/political-figure’ will be pruned to just ‘person’.
• Motivation: Uncommon for several sibling types to be relevant in the same context.
• Less common entities associated with few freebase types are better as training data as they are annotated with types relevant from context.
• Coarse Type Pruning: Remove types that do not agree with the output of a standard coarse grained classifier trained on {‘person’, ‘location’, ‘organization’, ‘other’}
• Motivation: Removes conflicting types of mentions such as location & organization
![Page 60: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/60.jpg)
Heuristics• Sibling Pruning: remove sibling types associated with a single entity,
leaving only the parent type i.e. a mention with types ‘person/athlete’ and ‘person/political-figure’ will be pruned to just ‘person’.
• Motivation: Uncommon for several sibling types to be relevant in the same context.
• Less common entities associated with few freebase types are better as training data as they are annotated with types relevant from context.
• Coarse Type Pruning: Remove types that do not agree with the output of a standard coarse grained classifier trained on {‘person’, ‘location’, ‘organization’, ‘other’}
• Motivation: Removes conflicting types of mentions such as location & organization
• Minimum Count Pruning: Remove types which appear less then ‘k’ times.
![Page 61: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/61.jpg)
Features, Model & Inference
![Page 62: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/62.jpg)
Features, Model & Inference• Sparse feature based model
![Page 63: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/63.jpg)
Features, Model & Inference• Sparse feature based model
She is married to the 44th and current President of the United States, Barack Obama, and is the first African-American First Lady…..
![Page 64: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/64.jpg)
Features, Model & Inference• Sparse feature based model
She is married to the 44th and current President of the United States, Barack Obama, and is the first African-American First Lady…..
![Page 65: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/65.jpg)
Features, Model & Inference• Sparse feature based model
She is married to the 44th and current President of the United States, Barack Obama, and is the first African-American First Lady…..
![Page 66: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/66.jpg)
Features, Model & Inference• Sparse feature based model
She is married to the 44th and current President of the United States, Barack Obama, and is the first African-American First Lady…..
• For each mention, we have a sparse feature vector
![Page 67: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/67.jpg)
Features, Model & Inference• Sparse feature based model
She is married to the 44th and current President of the United States, Barack Obama, and is the first African-American First Lady…..
• For each mention, we have a sparse feature vector • and a sparse label vector
![Page 68: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/68.jpg)
Models & Inference
![Page 69: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/69.jpg)
Models & Inference• The problem can be viewed as structured multi-label classification
problem.
![Page 70: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/70.jpg)
Models & Inference• The problem can be viewed as structured multi-label classification
problem.0
1
1
0
0
/location/city
/person
/person/political_figure
/other/food
/organization/business
>
![Page 71: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/71.jpg)
Models & Inference• The problem can be viewed as structured multi-label classification
problem.0
1
1
0
0
/location/city
/person
/person/political_figure
/other/food
/organization/business
>
Training Data
![Page 72: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/72.jpg)
Models & Inference• The problem can be viewed as structured multi-label classification
problem.0
1
1
0
0
/location/city
/person
/person/political_figure
/other/food
/organization/business
>
• +ve examples: For a type, +ve examples are all mentions marked as itself and also its descendants i.e. a mention labeled ‘person/artist’ is a +ve example for ‘person’
Training Data
![Page 73: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/73.jpg)
Models & Inference• The problem can be viewed as structured multi-label classification
problem.0
1
1
0
0
/location/city
/person
/person/political_figure
/other/food
/organization/business
>
• +ve examples: For a type, +ve examples are all mentions marked as itself and also its descendants i.e. a mention labeled ‘person/artist’ is a +ve example for ‘person’
Training Data
• -ve examples:
![Page 74: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/74.jpg)
Models & Inference• The problem can be viewed as structured multi-label classification
problem.0
1
1
0
0
/location/city
/person
/person/political_figure
/other/food
/organization/business
>
• +ve examples: For a type, +ve examples are all mentions marked as itself and also its descendants i.e. a mention labeled ‘person/artist’ is a +ve example for ‘person’
Training Data
• -ve examples: 1. all other types with the same parent (‘person/artist’ will be -
ve for ‘person/author’)
![Page 75: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/75.jpg)
Models & Inference• The problem can be viewed as structured multi-label classification
problem.0
1
1
0
0
/location/city
/person
/person/political_figure
/other/food
/organization/business
>
• +ve examples: For a type, +ve examples are all mentions marked as itself and also its descendants i.e. a mention labeled ‘person/artist’ is a +ve example for ‘person’
Training Data
• -ve examples: 1. all other types with the same parent (‘person/artist’ will be -
ve for ‘person/author’)2. all other types at the same depth (‘organization/business’
will be -ve for ‘person/author’)
![Page 76: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/76.jpg)
Models & Inference• The problem can be viewed as structured multi-label classification
problem.0
1
1
0
0
/location/city
/person
/person/political_figure
/other/food
/organization/business
>
• +ve examples: For a type, +ve examples are all mentions marked as itself and also its descendants i.e. a mention labeled ‘person/artist’ is a +ve example for ‘person’
Training Data
• -ve examples: 1. all other types with the same parent (‘person/artist’ will be -
ve for ‘person/author’)2. all other types at the same depth (‘organization/business’
will be -ve for ‘person/author’)3. all other types
![Page 77: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/77.jpg)
Models & Inference Local Model
![Page 78: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/78.jpg)
Models & Inference Local Model
• Local classifiers: A binary logistic regression classifier is trained for each label (i.e. there are T classifiers ) and label consistency is enforced at inference time
![Page 79: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/79.jpg)
Models & Inference Local Model
• During inference:
• Independent: Assign all types that exceeds some decision threshold
• Conditional: Multiply each label with the probability of its parent. This strategy ensures that if a label is selected then its parent would also be selected
• Marginalization: (if I understand correctly) Probability of a label is the sum of configurations in which it appears.
• Local classifiers: A binary logistic regression classifier is trained for each label (i.e. there are T classifiers ) and label consistency is enforced at inference time
![Page 80: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/80.jpg)
Models & Inference Global/Flat Model
![Page 81: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/81.jpg)
• Flat classifier: A flat softmax classifier to discriminate between all types
Models & Inference Global/Flat Model
![Page 82: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/82.jpg)
• Flat classifier: A flat softmax classifier to discriminate between all types • Classifier expects single type label to each instance.
Models & Inference Global/Flat Model
![Page 83: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/83.jpg)
• Flat classifier: A flat softmax classifier to discriminate between all types • Classifier expects single type label to each instance.• To account for that each multi-label instance, will be treated as
multiple single label instance
Models & Inference Global/Flat Model
![Page 84: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/84.jpg)
• Flat classifier: A flat softmax classifier to discriminate between all types • Classifier expects single type label to each instance.• To account for that each multi-label instance, will be treated as
multiple single label instance
Models & Inference Global/Flat Model
![Page 85: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/85.jpg)
• Flat classifier: A flat softmax classifier to discriminate between all types • Classifier expects single type label to each instance.• To account for that each multi-label instance, will be treated as
multiple single label instance
• Inference time: assign all labels whose probability exceeds a threshold; not just the max.
Models & Inference Global/Flat Model
![Page 86: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/86.jpg)
Results
![Page 87: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/87.jpg)
Results
Different -ve example strategies:
![Page 88: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/88.jpg)
Results
Different -ve example strategies:
![Page 89: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/89.jpg)
Results
Different -ve example strategies:
Different inference schemes:
![Page 90: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/90.jpg)
Results
Different -ve example strategies:
Different inference schemes:
![Page 91: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/91.jpg)
Results
![Page 92: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/92.jpg)
Results
Comparison between local and flat classifiers among different levels
![Page 93: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/93.jpg)
Results
Comparison between local and flat classifiers among different levels
![Page 94: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/94.jpg)
Conclusions
• Strives to make fine grained typing meaningful by requiring context dependence
• Introduce several distant supervision heuristics aimed at pruning irrelevant labels from the training data and match the gold data.
• Introduce new dataset 11,304 manually annotated mentions in 77 OntoNotes news documents.
![Page 95: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/95.jpg)
-Dani Yogatama, Dan Gillick, Nevena Lazic CMU, Google
Embedding Methods for Fine Grained Entity Type Classification
![Page 96: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/96.jpg)
Key Contribution
![Page 97: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/97.jpg)
Key Contribution
• Learn low dimensional embeddings of entity types and mentions instead of sparse binary embeddings.
![Page 98: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/98.jpg)
Key Contribution
• Learn low dimensional embeddings of entity types and mentions instead of sparse binary embeddings.
• Previously, each mention was one big sparse feature vector
![Page 99: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/99.jpg)
Key Contribution
• Learn low dimensional embeddings of entity types and mentions instead of sparse binary embeddings.
• Previously, each mention was one big sparse feature vector
If a hostile predator emerges for Saatchi & Saatchi Co., cofounders
Charles and Maurice Saatchi will lead…..
![Page 100: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/100.jpg)
Key Contribution
• Learn low dimensional embeddings of entity types and mentions instead of sparse binary embeddings.
• Previously, each mention was one big sparse feature vector
![Page 101: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/101.jpg)
Key Contribution
• Learn low dimensional embeddings of entity types and mentions instead of sparse binary embeddings.
• Previously, each mention was one big sparse feature vector
• Now instead, learn a low dimensional representation for each mention and types.
![Page 102: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/102.jpg)
Key Contribution
• Learn low dimensional embeddings of entity types and mentions instead of sparse binary embeddings.
• Previously, each mention was one big sparse feature vector
• Now instead, learn a low dimensional representation for each mention and types.
![Page 103: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/103.jpg)
Key Contribution
• Learn low dimensional embeddings of entity types and mentions instead of sparse binary embeddings.
• Previously, each mention was one big sparse feature vector
• Now instead, learn a low dimensional representation for each mention and types.
![Page 104: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/104.jpg)
Key Contribution
• Learn low dimensional embeddings of entity types and mentions instead of sparse binary embeddings.
• Previously, each mention was one big sparse feature vector
• Now instead, learn a low dimensional representation for each mention and types.
• Motivation: Learning low dimensional embeddings allows information sharing among related labels. For example: person/author would be more closer to person/artist than location/city.
![Page 105: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/105.jpg)
Model
![Page 106: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/106.jpg)
Model
f(x) : RD ! RH
8t 2 {1, 2, . . . ,>}, g(yt) : {0, 1}> ! RH
f and gInterested in learning mapping function
![Page 107: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/107.jpg)
Model
f(x) : RD ! RH
8t 2 {1, 2, . . . ,>}, g(yt) : {0, 1}> ! RH
f and gInterested in learning mapping function Low dim. embedding for mentions
![Page 108: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/108.jpg)
Model
Low dim. embedding for each entity type (label)
f(x) : RD ! RH
8t 2 {1, 2, . . . ,>}, g(yt) : {0, 1}> ! RH
f and gInterested in learning mapping function Low dim. embedding for mentions
![Page 109: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/109.jpg)
Model
Low dim. embedding for each entity type (label)
f(x) : RD ! RH
8t 2 {1, 2, . . . ,>}, g(yt) : {0, 1}> ! RH
f and gInterested in learning mapping function Low dim. embedding for mentions
Learning: Score of a label (represented as one hot vector ) and a feature vector
t ytx
![Page 110: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/110.jpg)
Model
Low dim. embedding for each entity type (label)
f(x) : RD ! RH
8t 2 {1, 2, . . . ,>}, g(yt) : {0, 1}> ! RH
f and gInterested in learning mapping function Low dim. embedding for mentions
s(x,yt,;A,B) = f(x,A) · g(yt,B) = Ax ·Byt
A 2 RH⇥D,B 2 RH⇥>
Learning: Score of a label (represented as one hot vector ) and a feature vector
t ytx
![Page 111: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/111.jpg)
Model
Low dim. embedding for each entity type (label)
f(x) : RD ! RH
8t 2 {1, 2, . . . ,>}, g(yt) : {0, 1}> ! RH
f and gInterested in learning mapping function Low dim. embedding for mentions
s(x,yt,;A,B) = f(x,A) · g(yt,B) = Ax ·Byt
A 2 RH⇥D,B 2 RH⇥>
Learning: Score of a label (represented as one hot vector ) and a feature vector
t ytx
f and g are linear mappings and score is calculated as dot product
![Page 112: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/112.jpg)
Kernel Extension
![Page 113: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/113.jpg)
Kernel Extension
s(x, yt) = Ax ·Byt
=X
d
(Adxd)>Bt
![Page 114: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/114.jpg)
Kernel Extension
s(x, yt) = Ax ·Byt
=X
d
(Adxd)>Bt
Kernel Extension: X
d
Kd,t(Adxd)>Bt
![Page 115: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/115.jpg)
Kernel Extension
s(x, yt) = Ax ·Byt
=X
d
(Adxd)>Bt
Kernel Extension: X
d
Kd,t(Adxd)>Bt
K is the kernel matrix
Kd,t = 1, if Ad is one of the nearest neighbor of Bt
![Page 116: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/116.jpg)
Kernel Extension
s(x, yt) = Ax ·Byt
=X
d
(Adxd)>Bt
Kernel Extension: X
d
Kd,t(Adxd)>Bt
K is the kernel matrix
Kd,t = 1, if Ad is one of the nearest neighbor of Bt
• It implicitly plays the role of label-dependent feature selector, learning which features can interact with which labels and turning the appropriate noisy ones off
![Page 117: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/117.jpg)
Kernel Extension
s(x, yt) = Ax ·Byt
=X
d
(Adxd)>Bt
Kernel Extension: X
d
Kd,t(Adxd)>Bt
K is the kernel matrix
Kd,t = 1, if Ad is one of the nearest neighbor of Bt
• It implicitly plays the role of label-dependent feature selector, learning which features can interact with which labels and turning the appropriate noisy ones off
• It is also a way of introducing non-linearity!
![Page 118: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/118.jpg)
Results
![Page 119: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/119.jpg)
Results
![Page 120: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/120.jpg)
Results
![Page 121: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/121.jpg)
Sonse Shimaoka, Pontus Stenetorp, Kentaro Inui, Sebastian Riedel Tohoku University, University College London
An Attentive Neural Architecture for Fine-grained Entity Type Classification
![Page 122: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/122.jpg)
Key Contribution
![Page 123: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/123.jpg)
Key Contribution
• Previous work modeled context as sparse features
![Page 124: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/124.jpg)
Key Contribution
• Previous work modeled context as sparse features
![Page 125: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/125.jpg)
Key Contribution
• Previous work modeled context as sparse features
• This has severe limitations:
• Cannot model larger context; will lead to feature explosion.
![Page 126: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/126.jpg)
Key Contribution
• Previous work modeled context as sparse features
• This has severe limitations:
• Cannot model larger context; will lead to feature explosion.
• Similar context do not share statistical strengths
![Page 127: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/127.jpg)
Key Contribution
• Previous work modeled context as sparse features
• This has severe limitations:
• Cannot model larger context; will lead to feature explosion.
River Monongahela flows through Pittsburgh
Pittsburgh has 3 rivers Allegheny, Monongahela, Ohio running through it
• Similar context do not share statistical strengths
![Page 128: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/128.jpg)
Key Contribution
![Page 129: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/129.jpg)
• Recurrent Neural Networks to the rescue!
Key Contribution
![Page 130: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/130.jpg)
• Recurrent Neural Networks to the rescue!
Key Contribution
• Given an entity mention and its left and right context words….
l1, . . . , lC ,m1, . . . ,mM , r1, . . . , rC
![Page 131: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/131.jpg)
• Recurrent Neural Networks to the rescue!
Key Contribution
• Given an entity mention and its left and right context words….
l1, . . . , lC ,m1, . . . ,mM , r1, . . . , rC
������!l1, . . . , lC ,m1, . . . ,mM , ������r1, . . . , rC
LSTM LSTM
• Encode the left and right context using a LSTM.
![Page 132: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/132.jpg)
• Recurrent Neural Networks to the rescue!
Key Contribution
• Given an entity mention and its left and right context words….
l1, . . . , lC ,m1, . . . ,mM , r1, . . . , rC
������!l1, . . . , lC ,m1, . . . ,mM , ������r1, . . . , rC
LSTM LSTM
• Encode the left and right context using a LSTM.
• Side Note: They averaged the mention embeddings to get one vector
vm =
PMi=1 u (mi)
M
![Page 133: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/133.jpg)
Attentive Encoder
![Page 134: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/134.jpg)
• The extend their LSTM model to incorporate attention over the context words.
Attentive Encoder
![Page 135: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/135.jpg)
• The extend their LSTM model to incorporate attention over the context words.
Attentive Encoder
![Page 136: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/136.jpg)
• The extend their LSTM model to incorporate attention over the context words.
Attentive Encoder
• For each of the intermediate context vector,
![Page 137: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/137.jpg)
• The extend their LSTM model to incorporate attention over the context words.
Attentive Encoder
• For each of the intermediate context vector,
![Page 138: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/138.jpg)
• The extend their LSTM model to incorporate attention over the context words.
Attentive Encoder
• For each of the intermediate context vector, compute an attention weight.
![Page 139: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/139.jpg)
• The extend their LSTM model to incorporate attention over the context words.
Attentive Encoder
• For each of the intermediate context vector, compute an attention weight.
![Page 140: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/140.jpg)
• The extend their LSTM model to incorporate attention over the context words.
Attentive Encoder
• For each of the intermediate context vector, compute an attention weight.
• The context representation is the weighted sum of the intermediate
representations.
![Page 141: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/141.jpg)
• The extend their LSTM model to incorporate attention over the context words.
Attentive Encoder
• For each of the intermediate context vector, compute an attention weight.
• The context representation is the weighted sum of the intermediate
representations.
![Page 142: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/142.jpg)
• The extend their LSTM model to incorporate attention over the context words.
Attentive Encoder
• For each of the intermediate context vector, compute an attention weight.
• The context representation is the weighted sum of the intermediate
representations.• The attention weights are computed using a 2 layer feed forward
network.
![Page 143: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/143.jpg)
• The extend their LSTM model to incorporate attention over the context words.
Attentive Encoder
• For each of the intermediate context vector, compute an attention weight.
• The context representation is the weighted sum of the intermediate
representations.• The attention weights are computed using a 2 layer feed forward
network.
![Page 144: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/144.jpg)
• The extend their LSTM model to incorporate attention over the context words.
Attentive Encoder
• For each of the intermediate context vector, compute an attention weight.
• The context representation is the weighted sum of the intermediate
representations.• The attention weights are computed using a 2 layer feed forward
network.
![Page 145: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/145.jpg)
• The extend their LSTM model to incorporate attention over the context words.
Attentive Encoder
• For each of the intermediate context vector, compute an attention weight.
• The context representation is the weighted sum of the intermediate
representations.• The attention weights are computed using a 2 layer feed forward
network.
Side Note: I am not sure
why don't they condition on the
mention embedding
![Page 146: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/146.jpg)
Training
![Page 147: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/147.jpg)
• Train in a multi-label setting. Each instance (mention, context) produces ‘K’ scores for each output types.
Training
![Page 148: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/148.jpg)
• Train in a multi-label setting. Each instance (mention, context) produces ‘K’ scores for each output types.
Training
y = �(W[vm;vc])W 2 RK⇥2D vm, vc 2 RD
![Page 149: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/149.jpg)
• Train in a multi-label setting. Each instance (mention, context) produces ‘K’ scores for each output types.
Training
y = �(W[vm;vc])W 2 RK⇥2D vm, vc 2 RD
• Loss is the usual cross entropy loss to maximize the likelihood of the training set.
t 2 {0, 1}K
![Page 150: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/150.jpg)
Results
![Page 151: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/151.jpg)
Results
![Page 152: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/152.jpg)
Results
![Page 153: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/153.jpg)
Conclusion/Discussion
![Page 154: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/154.jpg)
Conclusion/Discussion• presented 3 recent work on context dependent fine
grained entity typing.
![Page 155: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/155.jpg)
Conclusion/Discussion• presented 3 recent work on context dependent fine
grained entity typing.
• The models range from sparse linear ones to dense low dimensional representation to the recent NN approaches.
![Page 156: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/156.jpg)
Conclusion/Discussion• presented 3 recent work on context dependent fine
grained entity typing.
• The models range from sparse linear ones to dense low dimensional representation to the recent NN approaches.
• Open questions:
![Page 157: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/157.jpg)
Conclusion/Discussion• presented 3 recent work on context dependent fine
grained entity typing.
• The models range from sparse linear ones to dense low dimensional representation to the recent NN approaches.
• Open questions:
• Are the few hundred entity types enough?
![Page 158: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/158.jpg)
Conclusion/Discussion• presented 3 recent work on context dependent fine
grained entity typing.
• The models range from sparse linear ones to dense low dimensional representation to the recent NN approaches.
• Open questions:
• Are the few hundred entity types enough?
• Do we need to follow any taxonomy at all? Why not learn a type representation for each entity.
![Page 159: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/159.jpg)
Conclusion/Discussion• presented 3 recent work on context dependent fine
grained entity typing.
• The models range from sparse linear ones to dense low dimensional representation to the recent NN approaches.
• Open questions:
• Are the few hundred entity types enough?
• Do we need to follow any taxonomy at all? Why not learn a type representation for each entity.
• Any other way of representing context?
![Page 160: Context Dependent Fine Grained Entity Typingrajarshd.github.io/talks/Entity_Type.pdfEntity Type Classification • can be defined as the task of assigning semantic labels to mentions](https://reader035.fdocuments.in/reader035/viewer/2022071219/605524fccc3bbe70752408a1/html5/thumbnails/160.jpg)
ReferencesMarta Recasens, Marie-Catherine de Marneffe, and Christopher Potts. 2013. The life and death of discourse entities: Identifying singleton mentions.
Limin Yao, Sebastian Riedel, and Andrew McCallum. 2010. Collective cross-document relation extraction without labelled data. In Proc. of EMNLP.
Thomas Lin, Mausam, and Oren Etzioni. 2012. No noun phrase left behind: Detecting and typing unlinkable entities.In Proc. of EMNLP.
Andrew Carlson, Justin Betteridge, Richard C. Wang, Estevam R. Hruschka Jr., and Tom M. Mitchell. 2010. Coupled semi-supervised learning for information extraction. In Proc. of WSDM.
Luciano Del Corro, Abdalghani Abujabal, Rainer Gemulla, and Gerhard Weikum. 2015. Finet: Context-aware fine-grained named entity typing. In EMNLP
Mohamed Amir Yosef, Sandro Bauer, Johannes Hoffart, Marc Spaniol, and Gerhard Weikum. 2012. Hyena: Hierarchical type classification for entity names In ACL
Xiao Ling and Daniel S Weld. 2012. Fine-grained entity recognition. In AAAI
Rajarshi Das, Arvind Neelakantan, David Belanger, Andrew McCallum, Incorporating Selectional Preferences in Multihop relation extraction. In NAACL, AKBC.