Chains of Reasoning over Entities, Relations, and Text...
Transcript of Chains of Reasoning over Entities, Relations, and Text...
![Page 1: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/1.jpg)
Chains of Reasoning over Entities, Relations, and Text using
Recurrent Neural Networks
Rajarshi Das, Arvind Neelakantan, David Belanger, Andrew McCallum
Presentation in Deep Learning Summer School,
Montreal Aug 4 2016
![Page 2: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/2.jpg)
2
Relation Extraction
January 15, 2000Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft.
Text docs
![Page 3: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/3.jpg)
2
Relation Extraction
January 15, 2000Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft.
Text docs
January 15, 2000Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft. Steve Balmer
Microsoft
Bill Gates
MicrosoftSteve Balmer
![Page 4: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/4.jpg)
3
Relation Extraction
January 15, 2000Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft.
Text docs
January 15, 2000Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft.
Steve Balmer
Microsoft
Bill Gates
Microsoft
Steve Balmer
![Page 5: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/5.jpg)
3
Relation Extraction
January 15, 2000Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft.
Text docs
January 15, 2000Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft.
Steve Balmer
Microsoft
Bill Gates
Microsoft
Steve BalmerCEO?
![Page 6: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/6.jpg)
3
Relation Extraction
January 15, 2000Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft.
Text docs
January 15, 2000Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft.
Steve Balmer
Microsoft
Bill Gates
Microsoft
Steve BalmerCEO?
Friends?
Co-workers?
![Page 7: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/7.jpg)
3
Relation Extraction
January 15, 2000Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft.
Text docs
January 15, 2000Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft.
Steve Balmer
Microsoft
Bill Gates
Microsoft
Steve BalmerCEO?
Friends?
Co-workers?
![Page 8: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/8.jpg)
3
Relation Extraction
January 15, 2000Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft.
Text docs
January 15, 2000Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft.
Steve Balmer
Microsoft
Bill Gates
Microsoft
Steve BalmerCEO?
Friends?
Co-workers?
: CEO ?
: worked_together ?
![Page 9: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/9.jpg)
3
Relation Extraction
January 15, 2000Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft.
Text docs
January 15, 2000Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft.
Steve Balmer
Microsoft
Bill Gates
Microsoft
Steve BalmerCEO?
Friends?
Co-workers?
: CEO ?
: worked_together ?
long-time friendtake over as
CEO
![Page 10: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/10.jpg)
4
Relation Extraction
January 15, 2000Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft.
Text docs
January 15, 2000Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft.
Steve Balmer
Microsoft
Bill Gates
Microsoft
Steve BalmerCEO?
Friends?
Co-workers?
: CEO ?
: worked_together ?
long-time friend
take over as
CEO
Open IE textual relation types (Banko et al, 2007)
![Page 11: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/11.jpg)
5
Universal Schema for Relation Extraction
January 15, 2000Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft.
Text docs
January 15, 2000Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft.
Steve Balmer
Microsoft
Bill Gates
Microsoft
Steve BalmerCEO?
Friends?
Co-workers?
: CEO ?
: worked_together ?
long-time friend
take over as
CEO
Open IE textual relation types (Banko et al, 2007)
Riedel, Yao, McCallum, Marlin, 2013
Union of all relation types from all structured sources and text
![Page 12: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/12.jpg)
5
Universal Schema for Relation Extraction
January 15, 2000Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft.
Text docs
January 15, 2000Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft.
Steve Balmer
Microsoft
Bill Gates
Microsoft
Steve BalmerCEO?
Friends?
Co-workers?
: CEO ?
: worked_together ?
long-time friend
take over as
CEO
Open IE textual relation types (Banko et al, 2007)
Union of all relation types from all structured sources and text
![Page 13: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/13.jpg)
5
Universal Schema for Relation Extraction
January 15, 2000Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft.
Text docs
January 15, 2000Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft.
Steve Balmer
Microsoft
Bill Gates
Microsoft
Steve BalmerCEO?
Friends?
Co-workers?
: CEO ?
: worked_together ?
long-time friend
take over as
CEO
Open IE textual relation types (Banko et al, 2007)
Union of all relation types from all structured sources and text
Predict relation between entities based on mentions in text
![Page 14: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/14.jpg)
Chains of Reasoning for Relation Extraction
6
Microsoft Seattle
isBasedIn??
January 15, 2000Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft.
headquartered
![Page 15: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/15.jpg)
Chains of Reasoning for Relation Extraction
7
Microsoft Seattle
isBasedIn??
January 15, 2000Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft.
headquarteredBill
MicrosoftSeattle
![Page 16: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/16.jpg)
Chains of Reasoning for Relation Extraction
7
Microsoft Seattle
isBasedIn??
January 15, 2000Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft.
headquartered
Gates will now focus on the charitable foundation he runs with his wife Melinda French Gates. Bill and Melinda were married in a ceremony in Hawaii, rather than her hometown of Dallas.
Bill
MicrosoftSeattle
![Page 17: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/17.jpg)
Chains of Reasoning for Relation Extraction
7
Microsoft Seattle
isBasedIn??
January 15, 2000Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft.
headquartered
Gates will now focus on the charitable foundation he runs with his wife Melinda French Gates. Bill and Melinda were married in a ceremony in Hawaii, rather than her hometown of Dallas.
Bill
MicrosoftSeattle
Melinda
![Page 18: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/18.jpg)
8
Microsoft Seattle
isBasedIn??
January 15, 2000Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft
headquartered
Gates will now focus on the charitable foundation he runs with his wife Melinda French Gates. He will continue to serve as the chairman of Microsoft. Bill and Melinda were married in a ceremony in Hawaii, rather than her
Seattle
Melinda Bill Microsoft
headquartered
chairmanspouse
Chains of Reasoning for Relation Extraction
![Page 19: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/19.jpg)
8
Microsoft Seattle
isBasedIn??
January 15, 2000Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft
headquartered
Gates will now focus on the charitable foundation he runs with his wife Melinda French Gates. He will continue to serve as the chairman of Microsoft. Bill and Melinda were married in a ceremony in Hawaii, rather than her
Seattle
Melinda Bill Microsoft
headquartered
chairmanspouse
lives in??
Chains of Reasoning for Relation Extraction
![Page 20: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/20.jpg)
8
Microsoft Seattle
isBasedIn??
January 15, 2000Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft
headquartered
Gates will now focus on the charitable foundation he runs with his wife Melinda French Gates. He will continue to serve as the chairman of Microsoft. Bill and Melinda were married in a ceremony in Hawaii, rather than her
Seattle
Melinda Bill Microsoft
headquartered
chairmanspouse
lives in??
A Path between ‘Melinda’ and ‘Seattle’
Chains of Reasoning for Relation Extraction
![Page 21: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/21.jpg)
8
Microsoft Seattle
isBasedIn??
January 15, 2000Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft
headquartered
Gates will now focus on the charitable foundation he runs with his wife Melinda French Gates. He will continue to serve as the chairman of Microsoft. Bill and Melinda were married in a ceremony in Hawaii, rather than her
Seattle
Melinda Bill Microsoft
headquartered
chairmanspouse
lives in??
A Path between ‘Melinda’ and ‘Seattle’
Multi-hopInference
Chains of Reasoning for Relation Extraction
![Page 22: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/22.jpg)
8
Microsoft Seattle
isBasedIn??
January 15, 2000Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft
headquartered
Gates will now focus on the charitable foundation he runs with his wife Melinda French Gates. He will continue to serve as the chairman of Microsoft. Bill and Melinda were married in a ceremony in Hawaii, rather than her
Seattle
Melinda Bill Microsoft
headquartered
chairmanspouse
lives in??
A Path between ‘Melinda’ and ‘Seattle’
Multi-hopInference
Single-hopInference
Chains of Reasoning for Relation Extraction
![Page 23: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/23.jpg)
8
Microsoft Seattle
isBasedIn??
January 15, 2000Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft
headquartered
Gates will now focus on the charitable foundation he runs with his wife Melinda French Gates. He will continue to serve as the chairman of Microsoft. Bill and Melinda were married in a ceremony in Hawaii, rather than her
Seattle
Melinda Bill Microsoft
headquartered
chairmanspouse
lives in??
A Path between ‘Melinda’ and ‘Seattle’
Multi-hopInference
Single-hopInference
Spouse(A,B) & Chairman(B,C) & HQ-in(C,D) ➞ Lives-in(A,D)
Chains of Reasoning for Relation Extraction
![Page 24: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/24.jpg)
8
Microsoft Seattle
isBasedIn??
January 15, 2000Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft
headquartered
Gates will now focus on the charitable foundation he runs with his wife Melinda French Gates. He will continue to serve as the chairman of Microsoft. Bill and Melinda were married in a ceremony in Hawaii, rather than her
Seattle
Melinda Bill Microsoft
headquartered
chairmanspouse
lives in??
A Path between ‘Melinda’ and ‘Seattle’
Multi-hopInference
Single-hopInference
Spouse(A,B) & Chairman(B,C) & HQ-in(C,D) ➞ Lives-in(A,D)
• Sherlock (Schoenmackers et al. 2010) • Path Ranking Algorithm(Lao et al, 2011, 2012),
Chains of Reasoning for Relation Extraction
![Page 25: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/25.jpg)
8
Microsoft Seattle
isBasedIn??
January 15, 2000Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft
headquartered
Gates will now focus on the charitable foundation he runs with his wife Melinda French Gates. He will continue to serve as the chairman of Microsoft. Bill and Melinda were married in a ceremony in Hawaii, rather than her
Seattle
Melinda Bill Microsoft
headquartered
chairmanspouse
lives in??
A Path between ‘Melinda’ and ‘Seattle’
Multi-hopInference
Single-hopInference
Spouse(A,B) & Chairman(B,C) & HQ-in(C,D) ➞ Lives-in(A,D)
• Sherlock (Schoenmackers et al. 2010) • Path Ranking Algorithm(Lao et al, 2011, 2012),
Symbolic; Doesn't generalize
Chains of Reasoning for Relation Extraction
![Page 26: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/26.jpg)
8
Microsoft Seattle
isBasedIn??
January 15, 2000Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft
headquartered
Gates will now focus on the charitable foundation he runs with his wife Melinda French Gates. He will continue to serve as the chairman of Microsoft. Bill and Melinda were married in a ceremony in Hawaii, rather than her
Seattle
Melinda Bill Microsoft
headquartered
chairmanspouse
lives in??
A Path between ‘Melinda’ and ‘Seattle’
Multi-hopInference
Single-hopInference
Spouse(A,B) & Chairman(B,C) & HQ-in(C,D) ➞ Lives-in(A,D)
• Sherlock (Schoenmackers et al. 2010) • Path Ranking Algorithm(Lao et al, 2011, 2012),
Spouse(A,B) & CEO(B,C) & HQ-in(C,D) ➞ Lives-in(A,D)Symbolic;
Doesn't generalize
Chains of Reasoning for Relation Extraction
![Page 27: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/27.jpg)
8
Microsoft Seattle
isBasedIn??
January 15, 2000Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft
headquartered
Gates will now focus on the charitable foundation he runs with his wife Melinda French Gates. He will continue to serve as the chairman of Microsoft. Bill and Melinda were married in a ceremony in Hawaii, rather than her
Seattle
Melinda Bill Microsoft
headquartered
chairmanspouse
lives in??
A Path between ‘Melinda’ and ‘Seattle’
Multi-hopInference
Single-hopInference
Spouse(A,B) & Chairman(B,C) & HQ-in(C,D) ➞ Lives-in(A,D)
• Sherlock (Schoenmackers et al. 2010) • Path Ranking Algorithm(Lao et al, 2011, 2012),
Spouse(A,B) & CEO(B,C) & HQ-in(C,D) ➞ Lives-in(A,D)Spouse(A,B) & COO(B,C) & HQ-in(C,D) ➞ Lives-in(A,D)
Symbolic; Doesn't generalize
Chains of Reasoning for Relation Extraction
![Page 28: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/28.jpg)
8
Microsoft Seattle
isBasedIn??
January 15, 2000Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft
headquartered
Gates will now focus on the charitable foundation he runs with his wife Melinda French Gates. He will continue to serve as the chairman of Microsoft. Bill and Melinda were married in a ceremony in Hawaii, rather than her
Seattle
Melinda Bill Microsoft
headquartered
chairmanspouse
lives in??
A Path between ‘Melinda’ and ‘Seattle’
Multi-hopInference
Single-hopInference
Spouse(A,B) & Chairman(B,C) & HQ-in(C,D) ➞ Lives-in(A,D)
• Sherlock (Schoenmackers et al. 2010) • Path Ranking Algorithm(Lao et al, 2011, 2012),
Spouse(A,B) & CEO(B,C) & HQ-in(C,D) ➞ Lives-in(A,D)Spouse(A,B) & COO(B,C) & HQ-in(C,D) ➞ Lives-in(A,D)
Symbolic; Doesn't generalize
Chains of Reasoning for Relation Extraction
![Page 29: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/29.jpg)
9
SeattleMelinda Bill Microsoftheadquarteredchairmanspouse
RNNs for Chains of Reasoning
![Page 30: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/30.jpg)
9
SeattleMelinda Bill Microsoftheadquarteredchairmanspouse
RNN
RNNs for Chains of Reasoning
![Page 31: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/31.jpg)
9
SeattleMelinda Bill Microsoftheadquarteredchairmanspouse
RNN RNN
RNNs for Chains of Reasoning
![Page 32: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/32.jpg)
9
SeattleMelinda Bill Microsoftheadquarteredchairmanspouse
RNN RNN
RNNs for Chains of Reasoning
![Page 33: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/33.jpg)
9
SeattleMelinda Bill Microsoftheadquarteredchairmanspouse
RNN RNN RNN
RNNs for Chains of Reasoning
![Page 34: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/34.jpg)
9
SeattleMelinda Bill Microsoftheadquarteredchairmanspouse
RNN RNN RNN
RNNs for Chains of Reasoning
![Page 35: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/35.jpg)
9
SeattleMelinda Bill Microsoftheadquarteredchairmanspouse
RNN RNN RNN
Vector representation of the path
RNNs for Chains of Reasoning
![Page 36: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/36.jpg)
9
SeattleMelinda Bill Microsoftheadquarteredchairmanspouse
RNN RNN RNN
RNNs for Chains of Reasoning
![Page 37: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/37.jpg)
9
SeattleMelinda Bill Microsoftheadquarteredchairmanspouse
RNN RNN RNN
similarity metriclives in??
RNNs for Chains of Reasoning
![Page 38: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/38.jpg)
9
SeattleMelinda Bill Microsoftheadquarteredchairmanspouse
RNN RNN RNN
similarity metriclives in??
0.94
RNNs for Chains of Reasoning
![Page 39: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/39.jpg)
9
SeattleMelinda Bill Microsoftheadquarteredchairmanspouse
RNN RNN RNN
similarity metriclives in??
0.94
RNNs for Chains of ReasoningLogical Inference
inVector space!
![Page 40: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/40.jpg)
9
SeattleMelinda Bill Microsoftheadquarteredchairmanspouse
RNN RNN RNN
similarity metriclives in??
0.94
Neelakantan et al’ 15
RNNs for Chains of ReasoningLogical Inference
inVector space!
![Page 41: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/41.jpg)
10
Contributions
![Page 42: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/42.jpg)
10
Contributions
1. Single RNN for chains of reasoning.
![Page 43: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/43.jpg)
10
Contributions
1. Single RNN for chains of reasoning.
2. Combine evidence from multiple paths between entity pairs.
![Page 44: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/44.jpg)
10
Contributions
1. Single RNN for chains of reasoning.
2. Combine evidence from multiple paths between entity pairs.
3. Entity Aware RNN for chains of reasoning
![Page 45: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/45.jpg)
10
Contributions
1. Single RNN for chains of reasoning.
2. Combine evidence from multiple paths between entity pairs.
3. Entity Aware RNN for chains of reasoning
%MAP 64.43% 73.26%
![Page 46: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/46.jpg)
10
Contributions
1. Single RNN for chains of reasoning.
2. Combine evidence from multiple paths between entity pairs.
3. Entity Aware RNN for chains of reasoning
%MAP 64.43% 73.26%13.7%
![Page 47: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/47.jpg)
11
1. Single RNN for chains of reasoning.
![Page 48: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/48.jpg)
11
1. Single RNN for chains of reasoning.
lives in??
![Page 49: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/49.jpg)
11
1. Single RNN for chains of reasoning.
lives in?? works in??
![Page 50: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/50.jpg)
11
1. Single RNN for chains of reasoning.
lives in?? works in?? shops in??
![Page 51: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/51.jpg)
12
/ /lives in?? works in?? shops in?? /….
1. Single RNN for chains of reasoning.
![Page 52: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/52.jpg)
12
1. Single RNN for chains of reasoning.
![Page 53: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/53.jpg)
12
1. Single RNN for chains of reasoning.
SeattleMelinda Bill Microsoftheadquarteredchairmanspouse
RNN RNN RNN
similarity metric
0.94
![Page 54: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/54.jpg)
12
1. Single RNN for chains of reasoning.
SeattleMelinda Bill Microsoftheadquarteredchairmanspouse
RNN RNN RNN
similarity metric
0.94
Shared representations for relations
{
Single Composition Matrix for all query
relations
![Page 55: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/55.jpg)
12
1. Single RNN for chains of reasoning.
SeattleMelinda Bill Microsoftheadquarteredchairmanspouse
RNN RNN RNN
similarity metric
0.94
Shared representations for relations
{
Single Composition Matrix for all query
relations
{All query relation vectors
lives in??
shops in??
works in??
![Page 56: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/56.jpg)
Results
13
![Page 57: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/57.jpg)
Results
Model %MAP
PRA 64.43
PRA + Bigram 64.93
13
![Page 58: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/58.jpg)
Results
Model %MAP
PRA 64.43
PRA + Bigram 64.93
RNN-Path(Neelakantan’ 15) 65.23
13
![Page 59: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/59.jpg)
Results
Model %MAP
PRA 64.43
PRA + Bigram 64.93
RNN-Path(Neelakantan’ 15) 65.23
Single Model 68.77
13
![Page 60: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/60.jpg)
Results
Model %MAP
PRA 64.43
PRA + Bigram 64.93
5%
p < 5e-6
RNN-Path(Neelakantan’ 15) 65.23
Single Model 68.77
13
![Page 61: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/61.jpg)
14
Contributions
1. Single RNN for chains of reasoning.
2. Combine evidence from multiple paths between entity pairs.
3. Entity Aware RNN for chains of reasoning
![Page 62: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/62.jpg)
Multiple Paths
SeattleMelinda Bill Microsoftheadquarteredchairmanspouse
15
![Page 63: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/63.jpg)
Multiple Paths
SeattleMelinda Bill Microsoftheadquarteredchairmanspouse
Gates Foundation
CEOlocated in
15
![Page 64: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/64.jpg)
Multiple Paths
SeattleMelinda Bill Microsoftheadquarteredchairmanspouse
Gates Foundation
CEOlocated in
Jane Doe John Doe
friendmarried to
born in
15
![Page 65: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/65.jpg)
Multiple Paths
SeattleMelinda Bill Microsoftheadquarteredchairmanspouse
Gates Foundation
CEOlocated in
Jane Doe John Doe
friendmarried to
born in
15
![Page 66: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/66.jpg)
Multiple Paths
SeattleMelinda Bill Microsoftheadquarteredchairmanspouse
Gates Foundation
CEOlocated in
Jane Doe John Doe
friendmarried to
born in
16
![Page 67: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/67.jpg)
Multiple Paths
SeattleMelinda Bill Microsoftheadquarteredchairmanspouse
Gates Foundation
CEOlocated in
Jane Doe John Doe
friendmarried to
born in
RNNencoder
16
![Page 68: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/68.jpg)
Multiple Paths
SeattleMelinda Bill Microsoftheadquarteredchairmanspouse
Gates Foundation
CEOlocated in
Jane Doe John Doe
friendmarried to
born in
RNNencoder
16
![Page 69: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/69.jpg)
Multiple Paths
SeattleMelinda Bill Microsoftheadquarteredchairmanspouse
Gates Foundation
CEOlocated in
Jane Doe John Doe
friendmarried to
born in
RNNencoder
16
![Page 70: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/70.jpg)
Multiple Paths
SeattleMelinda Bill Microsoftheadquarteredchairmanspouse
Gates Foundation
CEOlocated in
Jane Doe John Doe
friendmarried to
born in
RNNencoder
16
![Page 71: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/71.jpg)
Multiple Paths
SeattleMelinda Bill Microsoftheadquarteredchairmanspouse
Gates Foundation
CEOlocated in
Jane Doe John Doe
friendmarried to
born in
RNNencoder
lives in??
16
![Page 72: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/72.jpg)
Multiple Paths
SeattleMelinda Bill Microsoftheadquarteredchairmanspouse
Gates Foundation
CEOlocated in
Jane Doe John Doe
friendmarried to
born in
RNNencoder
similarity metric
0.96 0.94 0.18
lives in??
17
![Page 73: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/73.jpg)
Score Pooling
0.96 0.94 0.18
lives in??
similarity metric
Path 1 Path 2 Path 3
![Page 74: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/74.jpg)
Score Pooling
0.96 0.94 0.18
lives in??
similarity metric
Path 1 Path 2 Path 3
1. Max pool: max(↵1,↵2, . . . ,↵N )
Neelakantan et al’ 15
![Page 75: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/75.jpg)
Score Pooling
0.96 0.94 0.18
lives in??
similarity metric
Path 1 Path 2 Path 3
1. Max pool: max(↵1,↵2, . . . ,↵N )
Neelakantan et al’ 15
Max
![Page 76: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/76.jpg)
Score Pooling
0.96 0.94 0.18
lives in??
similarity metric
Path 1 Path 2 Path 3
1. Max pool: max(↵1,↵2, . . . ,↵N )
Neelakantan et al’ 15
Max
0.96
![Page 77: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/77.jpg)
Score Pooling
0.96 0.94 0.18
lives in??
similarity metric
Path 1 Path 2 Path 3
1. Max pool: max(↵1,↵2, . . . ,↵N )
Neelakantan et al’ 15
Max
![Page 78: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/78.jpg)
Score Pooling
0.96 0.94 0.18
lives in??
similarity metric
Path 1 Path 2 Path 3
1. Max pool: max(↵1,↵2, . . . ,↵N )
Neelakantan et al’ 15
Max
2. Avg. pool: 1
N
NX
i=1
↵i
!
Avg.
![Page 79: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/79.jpg)
Score Pooling
0.96 0.94 0.18
lives in??
similarity metric
Path 1 Path 2 Path 3
1. Max pool: max(↵1,↵2, . . . ,↵N )
Neelakantan et al’ 15
Max
2. Avg. pool: 1
N
NX
i=1
↵i
!
Avg.
0.69
![Page 80: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/80.jpg)
Score Pooling
0.96 0.94 0.18
lives in??
similarity metric
Path 1 Path 2 Path 3
1. Max pool: max(↵1,↵2, . . . ,↵N )
Neelakantan et al’ 15
Max
2. Avg. pool: 1
N
NX
i=1
↵i
!
Avg.
![Page 81: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/81.jpg)
Score Pooling
0.96 0.94 0.18
lives in??
similarity metric
Path 1 Path 2 Path 3
1. Max pool: max(↵1,↵2, . . . ,↵N )
Neelakantan et al’ 15
Max
2. Avg. pool: 1
N
NX
i=1
↵i
!
Avg.
3. Top(k): 1
k
kX
i=1
si
!
Top(2)
![Page 82: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/82.jpg)
Score Pooling
0.96 0.94 0.18
lives in??
similarity metric
Path 1 Path 2 Path 3
1. Max pool: max(↵1,↵2, . . . ,↵N )
Neelakantan et al’ 15
Max
2. Avg. pool: 1
N
NX
i=1
↵i
!
Avg.
3. Top(k): 1
k
kX
i=1
si
!
Top(2)
0.95
![Page 83: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/83.jpg)
Score Pooling
0.96 0.94 0.18
lives in??
similarity metric
Path 1 Path 2 Path 3
4. LogSumExp: log(
NX
i=1
exp(↵i))
![Page 84: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/84.jpg)
Score Pooling
0.96 0.94 0.18
lives in??
similarity metric
Path 1 Path 2 Path 3
4. LogSumExp: log(
NX
i=1
exp(↵i))
LSE
![Page 85: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/85.jpg)
Score Pooling
0.96 0.94 0.18
lives in??
similarity metric
Path 1 Path 2 Path 3
4. LogSumExp: log(
NX
i=1
exp(↵i))
LSE
1.85
![Page 86: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/86.jpg)
Score Pooling
0.96 0.94 0.18
lives in??
similarity metric
Path 1 Path 2 Path 3
4. LogSumExp: log(
NX
i=1
exp(↵i))
LSE
1.85
@LSE
@↵i=
exp(↵i)Pi exp(↵i)
![Page 87: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/87.jpg)
Score Pooling
0.96 0.94 0.18
lives in??
similarity metric
Path 1 Path 2 Path 3
4. LogSumExp: log(
NX
i=1
exp(↵i))
LSE
1.85
@LSE
@↵i=
exp(↵i)Pi exp(↵i)
Gradient Steps #1040 0.5 1 1.5 2
Trai
ning
Los
s
0.4
0.5
0.6
0.7
0.8
0.9
1
1.1
1.2LogSumExpMaxMeanTop-k
![Page 88: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/88.jpg)
Results
20
![Page 89: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/89.jpg)
Results
Model %MAP Pooling
RNN-Path(Neelakantan ’15) 65.23 Max
20
![Page 90: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/90.jpg)
Results
Model %MAP Pooling
RNN-Path(Neelakantan ’15) 65.23 Max
RNN-Path 68.43 LogSumExp
20
![Page 91: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/91.jpg)
Results
Model %MAP Pooling
RNN-Path(Neelakantan ’15) 65.23 Max
RNN-Path 68.43 LogSumExp
Single Model 68.77 Max
Single Model 55.8 Avg.
Single Model 68.20 Top(K)
20
![Page 92: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/92.jpg)
Results
Model %MAP Pooling
RNN-Path(Neelakantan ’15) 65.23 Max
RNN-Path 68.43 LogSumExp
Single Model 68.77 Max
Single Model 55.8 Avg.
Single Model 68.20 Top(K)
Single Model 70.11 LogSumExp20
![Page 93: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/93.jpg)
21
Contributions
1. Single RNN for chains of reasoning.
2. Combine evidence from multiple paths between entity pairs.
3. Entity Aware RNN for chains of reasoning
![Page 94: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/94.jpg)
JFK NYC NY
located in located in
airport_serves_location??
![Page 95: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/95.jpg)
JFK NYC NY
located in located in
airport_serves_location??
0.9276~
![Page 96: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/96.jpg)
JFK NYC NY
located in located in
airport_serves_location??
0.9276~
Yankee Stadium NYC NY
located in located in
airport_serves_location??
![Page 97: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/97.jpg)
JFK NYC NY
located in located in
airport_serves_location??
0.9276~
Yankee Stadium NYC NY
located in located in
airport_serves_location??
~ 0.9276
![Page 98: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/98.jpg)
JFK NYC NY
located in located in
airport_serves_location??
0.9276~
Yankee Stadium NYC NY
located in located in
airport_serves_location??
~ 0.9276
![Page 99: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/99.jpg)
SeattleMelinda Bill Microsoft headquarteredchairmanspouse
Entity Representation
![Page 100: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/100.jpg)
SeattleMelinda Bill Microsoft headquarteredchairmanspouse
Entity Representation
![Page 101: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/101.jpg)
SeattleMelinda Bill Microsoft
Entity Representation
![Page 102: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/102.jpg)
SeattleMelinda Bill Microsoft
Entity Representation1. Learn separate representation for each entity.
![Page 103: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/103.jpg)
SeattleMelinda Bill Microsoft
Entity Representation1. Learn separate representation for each entity.
Jane Doe
![Page 104: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/104.jpg)
SeattleMelinda Bill Microsoft
Entity Representation1. Learn separate representation for each entity.
Jane Doe
rare occurrence; hard to learn good representations.
![Page 105: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/105.jpg)
SeattleMelinda Bill Microsoft
Entity Representation
2. Represent entities by their annotated types 1. Learn separate representation for each entity.
Jane Doe
rare occurrence; hard to learn good representations.
![Page 106: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/106.jpg)
SeattleMelinda Bill Microsoft
Entity Representation
2. Represent entities by their annotated types 1. Learn separate representation for each entity.
Jane Doe
Annotated types in KB
1. CEO2. Philanthropist3. Duke University alumni4. American Citizen
1. American Citizen2. Small business owner
Jane DoeMelinda
rare occurrence; hard to learn good representations.
![Page 107: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/107.jpg)
Melinda
Entity Representation
Jane Doe
1. CEO 2. Philanthropist 3. Duke University alumni 4. American Citizen
1. American Citizen 2. Small business owner
+ + +
+
=
=
![Page 108: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/108.jpg)
Melinda
Entity Representation
Jane Doe
1. CEO 2. Philanthropist 3. Duke University alumni 4. American Citizen
1. American Citizen 2. Small business owner
+ + +
+
=
=
Entity is represented as the sum of their annotated types
![Page 109: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/109.jpg)
Entity Aware RNNs for Chains of Reasoning
RNN RNN RNN RNN
similarity metriccountryofHQ??
0.94
SeattleMelinda Bill Microsoftchairmanspouse headquartered dummy_rel
![Page 110: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/110.jpg)
Results
![Page 111: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/111.jpg)
Results
Model %MAP
Single Model 70.11
![Page 112: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/112.jpg)
Results
Model %MAP
Single Model 70.11
Single Model + Entity 71.74
![Page 113: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/113.jpg)
Results
Model %MAP
Single Model 70.11
Single Model + Entity 71.74
Single Model + Types 73.26
![Page 114: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/114.jpg)
Results
4.4%
p < 5e-6
Model %MAP
Single Model 70.11
Single Model + Entity 71.74
Single Model + Types 73.26
![Page 115: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/115.jpg)
Results
![Page 116: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/116.jpg)
Results
Model %MAP Pooling
PRA 64.43 n/a
RNN-Path(Neelkantan et al’15) 65.23 Max
![Page 117: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/117.jpg)
Results
Model %MAP Pooling
PRA 64.43 n/a
RNN-Path(Neelkantan et al’15) 65.23 Max
Single Model 68.77 Max
![Page 118: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/118.jpg)
Results
Model %MAP Pooling
PRA 64.43 n/a
RNN-Path(Neelkantan et al’15) 65.23 Max
Single Model 68.77 Max
Single Model 70.11 LogSumExp
![Page 119: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/119.jpg)
Results
Model %MAP Pooling
PRA 64.43 n/a
RNN-Path(Neelkantan et al’15) 65.23 Max
Single Model 68.77 Max
Single Model 70.11 LogSumExp
Single Model + Types 73.26 LogSumExp
![Page 120: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/120.jpg)
Results
Model %MAP Pooling
PRA 64.43 n/a
RNN-Path(Neelkantan et al’15) 65.23 Max
13.7% Single Model 68.77 Max
Single Model 70.11 LogSumExp
Single Model + Types 73.26 LogSumExp
![Page 121: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/121.jpg)
Predictive Clauses
29
![Page 122: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/122.jpg)
Predictive Clauses
/people/person/place_of_birth(A,B) ’was born in’(A,X) & ‘commonly known as’(X,B)
29
![Page 123: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/123.jpg)
Predictive Clauses
/people/person/place_of_birth(A,B) ’was born in’(A,X) & ‘commonly known as’(X,B)
29
![Page 124: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/124.jpg)
Predictive Clauses
/people/person/place_of_birth(A,B) ’was born in’(A,X) & ‘commonly known as’(X,B)Textual relations
29
![Page 125: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/125.jpg)
Predictive Clauses
/people/person/place_of_birth(A,B) ’was born in’(A,X) & ‘commonly known as’(X,B)Textual relations
Universal Schema
29
![Page 126: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/126.jpg)
Predictive Clauses
/people/person/place_of_birth(A,B) ’was born in’(A,X) & ‘commonly known as’(X,B)Textual relations
29
![Page 127: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/127.jpg)
Predictive Clauses
/people/person/place_of_birth(A,B) ’was born in’(A,X) & ‘commonly known as’(X,B)Textual relations
/book/written_work/original_language(A,B) /book/written_work_author(A,X) & ‘address’(X,Y)
& /people/person/nationality-1(Y,Z) & people/person/languages(Z,B)
29
![Page 128: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/128.jpg)
Predictive Clauses
/people/person/place_of_birth(A,B) ’was born in’(A,X) & ‘commonly known as’(X,B)Textual relations
/book/written_work/original_language(A,B) /book/written_work_author(A,X) & ‘address’(X,Y)
& /people/person/nationality-1(Y,Z) & people/person/languages(Z,B)
Entity Aware Model
29
![Page 129: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/129.jpg)
Predictive Clauses
/people/person/place_of_birth(A,B) ’was born in’(A,X) & ‘commonly known as’(X,B)Textual relations
/book/written_work/original_language(A,B) /book/written_work_author(A,X) & ‘address’(X,Y)
& /people/person/nationality-1(Y,Z) & people/person/languages(Z,B)
Sandy_Lake_Airport Sandy_Lake_First_Nation
/aviation/airport/serves?? Entity Aware Model
29
![Page 130: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/130.jpg)
Predictive Clauses
/people/person/place_of_birth(A,B) ’was born in’(A,X) & ‘commonly known as’(X,B)Textual relations
/book/written_work/original_language(A,B) /book/written_work_author(A,X) & ‘address’(X,Y)
& /people/person/nationality-1(Y,Z) & people/person/languages(Z,B)
Sandy_Lake_Airport/location/contains
Ontario‘(in northwestern)’-1
Sandy_Lake_First_Nation
Sandy_Lake_Airport Sandy_Lake_First_Nation
/aviation/airport/serves?? Entity Aware Model
29
![Page 131: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/131.jpg)
Predictive Clauses
/people/person/place_of_birth(A,B) ’was born in’(A,X) & ‘commonly known as’(X,B)Textual relations
/book/written_work/original_language(A,B) /book/written_work_author(A,X) & ‘address’(X,Y)
& /people/person/nationality-1(Y,Z) & people/person/languages(Z,B)
Sandy_Lake_Airport Sandy_Lake_First_Nation
/aviation/airport/serves?? Entity Aware Model
29
![Page 132: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/132.jpg)
Predictive Clauses
/people/person/place_of_birth(A,B) ’was born in’(A,X) & ‘commonly known as’(X,B)Textual relations
/book/written_work/original_language(A,B) /book/written_work_author(A,X) & ‘address’(X,Y)
& /people/person/nationality-1(Y,Z) & people/person/languages(Z,B)
Sandy_Lake_Airport/location/contains
Ontario‘(and at the)’
Toronto_Royal_Music_Hall
’including the’
Canada
/geography/lake/basin_countries
Big Trout Lakeand toSandy_La
ke_First_Nation
Sandy_Lake_Airport Sandy_Lake_First_Nation
/aviation/airport/serves?? Entity Aware Model
29
![Page 133: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/133.jpg)
Conclusion
30
![Page 134: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/134.jpg)
Conclusion
30
• Introduced a high-capacity single RNN model for relation extraction
![Page 135: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/135.jpg)
Conclusion
30
• Introduced a high-capacity single RNN model for relation extraction
• Combines evidences among multiple paths and sources of evidence
![Page 136: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/136.jpg)
Conclusion
30
• Introduced a high-capacity single RNN model for relation extraction
• Combines evidences among multiple paths and sources of evidence
• Entity Aware!
![Page 137: Chains of Reasoning over Entities, Relations, and Text ...rajarshd.github.io/talks/Chains_of_Reasoning.pdf · Recurrent Neural Networks Rajarshi Das, Arvind Neelakantan, ... Presentation](https://reader034.fdocuments.in/reader034/viewer/2022050217/5f633a22c5bea72a292d62f5/html5/thumbnails/137.jpg)
Conclusion
30
• Introduced a high-capacity single RNN model for relation extraction
• Combines evidences among multiple paths and sources of evidence
• Entity Aware!
Thanks!