Post on 22-Jan-2022
Machine Reading and Reasoning with Neural Program Interpreters
Sebastian Riedel
Machine ReadingBloomsbury AI
@riedelcastro
Collaborators
2
Tim Rocktäschel
Matko BosnjakPontus Stenetorp
(now at Oxford)
(UCL)(UCL)
Johannes Welbl(UCL)
Jason Naradowsky(Johns Hopkins University)
“Should we separate meaning from language?”
3
Chris Manning @AKBC 2013
!
[Language] [Meaning]
?
[Information Need]
[Maybe not?]
End-to-End Reading and Comprehension
4
(Hermann et. al 2015, Seo et al., 2016, Rajpurkar et al., 2016, Weissenborn 2016…)
!
[Language]
?
[Information Need]
Nicola Tesla
… In January 1880, two of Tesla's uncles put together enough money to help him leave Gospić for Prague where he was to study. Unfortunately, he arrived too late to enroll at Charles-Ferdinand University; he never studied Greek, a required subject; and he was illiterate in Czech, another required subject. Tesla did, however, attend lectures at the university, although, as an auditor, he did not receive grades for the courses.
Machine Reading
5
Nicola Tesla
… In January 1880, two of Tesla's uncles put together enough money to help him leave Gospić for Prague where he was to study. Unfortunately, he arrived too late to enroll at Charles-Ferdinand University; he never studied Greek, a required subject; and he was illiterate in Czech, another required subject. Tesla did, however, attend lectures at the university, although, as an auditor, he did not receive grades for the courses.
What city did Tesla move to in 1880?
Prague
Why was he unable to enroll at the university?
arrived too late to enroll
Nicola Tesla
… In January 1880, two of Tesla's uncles put together enough money to help him leave Gospić for Prague where he was to study. Unfortunately, he arrived too late to enroll at Charles-Ferdinand University; he never studied Greek, a required subject; and he was illiterate in Czech, another required subject. Tesla did, however, attend lectures at the university, although, as an auditor, he did not receive grades for the courses.
How to read and reason?
6
Machine Reading and Reasoning
7
Which medical specialty deals with pituitary ACTH hypersecretion?
Endocrinology
Pituitary ACTH hypersecretion ... is a form of hyperpituitarism characterized by an abnormally high level of ACTH produced by the anterior pituitary …
A major organ of the endocrine system, the anterior pituitary is the glandular, anterior lobe that ...
The endocrine system is ... ... The field of study dealing with the endocrine system and its disorders is endocrinology, a branch of internal medicine.
Machine Reading and Reasoning
8
How many pictures were in each of the albums?
2
Isabel uploaded 2 pictures from her phone and 4 from her camera to facebook. She sorted the pics into 3 different albums with the same amount of pics in each album.
Can we learn this end-to-end?
9
Part 1: A Read & Reason Dataset
10
Pontus StenetorpJohannes Welbl
11
A Single Instance
12
What is the nationality of Jamie Burnett?
Candidates:
Scotland (correct)
China (incorrect)
…
Jamie Burnett (born 16 September 1975 ) is a professional snooker player from Hamilton, South Lanarkshire…He began the 2014/2015 season with a quarter-final showing at the Yixing Open…
…Hamilton is a town in South Lanarkshire, in the central Lowlands of Scotland…
The Yixing Open was a professional minor-ranking snooker tournament that took place at the Yixing Sports Centre in Yixing, China.
Dataset Construction Method
13
Unlabelled Text
Knowledge Base
Dataset Construction Method
[Conditionally Accepted to TACL]
Multihop Dataset
Dataset Construction Method
14
DocumentsEntities
Jamie Burnett
Scotland
described in
mentions
described inmentions
HamiltonJamie Burnett, citizenship, Scotland
Yixing Open
China
What is the nationality of Jamie Burnett?
KB Triple Instance
Scotland
China
Dataset Construction Method
15
Unlabelled Text
Knowledge Base
Dataset Construction Method
[Conditionally Accepted to TACL]
Multihop Dataset
Dataset Construction Method
16
Dataset Construction Method
WikiHopUnlabelled Text
Knowledge Base
Dataset Construction Method
17
Dataset Construction Method
MedHopUnlabelled Text
Knowledge Base
Baseline Results
18
Accu
racy
[%]
0
8.333
16.667
25
33.333
41.667
50
WikiHop MedHop
47.8
42.9
9
25.6
9.510.6
13.9
11.5
RandomMax-MentionTF-IDFBiDAF
Reduction to Traditional Machine Comprehension
19
Which medical specialty deals with pituitary ACTH hypersecretion?
Endocrinology
Pituitary ACTH hypersecretion ... is a form of hyperpituitarism characterized by an abnormally high level of ACTH produced by the anterior pituitary …
A major organ of the endocrine system, the anterior pituitary is the glandular, anterior lobe that ...
The endocrine system is ... ... The field of study dealing with the endocrine system and its disorders is endocrinology, a branch of internal medicine.
Reduction to Traditional Machine Comprehension
20
Which medical specialty deals with pituitary ACTH hypersecretion?
Endocrinology A major organ of the endocrine system, the anterior pituitary is the glandular, anterior lobe that ...
Pituitary ACTH hypersecretion ... is a form of hyperpituitarism characterized by an abnormally high level of ACTH produced by the anterior pituitary …
The endocrine system is ... ... The field of study dealing with the endocrine system and its disorders is endocrinology, a branch of internal medicine.
Does BiDaf Aggregate?
21
Which medical specialty deals with pituitary ACTH hypersecretion?
Endocrinology A major organ of the endocrine system, the anterior pituitary is the glandular, anterior lobe that ...
Pituitary ACTH hypersecretion ... is a form of hyperpituitarism characterized by an abnormally high level of ACTH produced by the anterior pituitary …
The endocrine system is ... ... The field of study dealing with the endocrine system and its disorders is endocrinology, a branch of internal medicine.
Removing Relevant Documents, Keep Answer Documents
22
Accu
racy
[%]
25
30
35
40
45
50
55
WikiHop MedHop
30.4
44.6
33.7
54.5
BIDAFBIDAF doc-rem 10%
Part 2: Learning to Read and Calculate
23
Tim RocktäschelMatko Bosnjak Jason Naradowsky
Machine Reading and Reasoning: Math
24
How many pictures were in each of the albums?
2
Isabel uploaded 2 pictures from her phone and 4 from her camera to facebook. She sorted the pics into 3 different albums with the same amount of pics in each album.
Differentiable Program Interpreters
25
0 1 2 3 4 5 6 7 8 9p
Code0 1 2 3 4 5 6 7 8 9
34
2
d
Stack
Program Interpreter0 1 2 3 4 5 6 7 8 9
Heap
Bosnjak et al. ICML 2017
Reader
Model
Isabel uploaded 2 pictures from her phone and 4 from her camera to facebook. She sorted the pics into 3 different albums with the same amount of pics in each album.
How many pictures were in each of the albums?
0 1 2 3 4 5 6 7 8 9
34
2
d
Stack0 1 2 3 4 5 6 7 8 9p
Code
Differentiable Program Interpreters
26
0 1 2 3 4 5 6 7 8 9p
Code0 1 2 3 4 5 6 7 8 9
42
3
d
Stack
Program Interpreter0 1 2 3 4 5 6 7 8 9
Heap
Bosnjak et al. ICML 2017
Reader
Model
Isabel uploaded 2 pictures from her phone and 4 from her camera to facebook. She sorted the pics into 3 different albums with the same amount of pics in each album.
How many pictures were in each of the albums?
0 1 2 3 4 5 6 7 8 9
34
2
d
Stack0 1 2 3 4 5 6 7 8 9p
Code
Differentiable Program Interpreters
27
0 1 2 3 4 5 6 7 8 9p
Code0 1 2 3 4 5 6 7 8 9
42
3
d
Stack
Program Interpreter0 1 2 3 4 5 6 7 8 9
Heap
Bosnjak et al. ICML 2017
Reader
Model
Isabel uploaded 2 pictures from her phone and 4 from her camera to facebook. She sorted the pics into 3 different albums with the same amount of pics in each album.
How many pictures were in each of the albums?
0 1 2 3 4 5 6 7 8 9p
Code0 1 2 3 4 5 6 7 8 9
63
d
Stack
0 1 2 3 4 5 6 7 8 9
34
2
d
Stack0 1 2 3 4 5 6 7 8 9p
Code
Differentiable Program Interpreters
28
0 1 2 3 4 5 6 7 8 9p
Code0 1 2 3 4 5 6 7 8 9
42
3
d
Stack
Program Interpreter0 1 2 3 4 5 6 7 8 9
Heap
Bosnjak et al. ICML 2017
Reader
Model
Isabel uploaded 2 pictures from her phone and 4 from her camera to facebook. She sorted the pics into 3 different albums with the same amount of pics in each album.
How many pictures were in each of the albums?
0 1 2 3 4 5 6 7 8 9p
Code0 1 2 3 4 5 6 7 8 9
63
d
Stack0 1 2 3 4 5 6 7 8 9p
Code0 1 2 3 4 5 6 7 8 9
36
d
Stack
0 1 2 3 4 5 6 7 8 9
34
2
d
Stack0 1 2 3 4 5 6 7 8 9p
Code
Differentiable Program Interpreters
29
0 1 2 3 4 5 6 7 8 9p
Code0 1 2 3 4 5 6 7 8 9
42
3
d
Stack
Program Interpreter0 1 2 3 4 5 6 7 8 9
Heap
Bosnjak et al. ICML 2017
Reader
Model
Isabel uploaded 2 pictures from her phone and 4 from her camera to facebook. She sorted the pics into 3 different albums with the same amount of pics in each album.
How many pictures were in each of the albums?
0 1 2 3 4 5 6 7 8 9p
Code0 1 2 3 4 5 6 7 8 9
63
d
Stack0 1 2 3 4 5 6 7 8 9p
Code0 1 2 3 4 5 6 7 8 9
36
d
Stack0 1 2 3 4 5 6 7 8 9p
Code0 1 2 3 4 5 6 7 8 9
2
d
Stack
0 1 2 3 4 5 6 7 8 9
34
2
d
Stack0 1 2 3 4 5 6 7 8 9p
Code
Training
30
0 1 2 3 4 5 6 7 8 9p
Code0 1 2 3 4 5 6 7 8 9
42
3
d
Stack
Program Interpreter0 1 2 3 4 5 6 7 8 9
Heap
Bosnjak et al. ICML 2017
Reader
Model
Isabel uploaded 2 pictures from her phone and 4 from her camera to facebook. She sorted the pics into 3 different albums with the same amount of pics in each album.
How many pictures were in each of the albums?
0 1 2 3 4 5 6 7 8 9p
Code0 1 2 3 4 5 6 7 8 9
63
d
Stack0 1 2 3 4 5 6 7 8 9p
Code0 1 2 3 4 5 6 7 8 9
36
d
Stack0 1 2 3 4 5 6 7 8 9p
Code0 1 2 3 4 5 6 7 8 9
2
d
Stack
0 1 2 3 4 5 6 7 8 9
34
2
d
Stack0 1 2 3 4 5 6 7 8 9p
Code
Training
31
0 1 2 3 4 5 6 7 8 9p
Code0 1 2 3 4 5 6 7 8 9
42
3
d
Stack
Program Interpreter0 1 2 3 4 5 6 7 8 9
Heap
Bosnjak et al. ICML 2017
Reader
Model
Isabel uploaded 2 pictures from her phone and 4 from her camera to facebook. She sorted the pics into 3 different albums with the same amount of pics in each album.
How many pictures were in each of the albums?
0 1 2 3 4 5 6 7 8 9p
Code0 1 2 3 4 5 6 7 8 9
63
d
Stack0 1 2 3 4 5 6 7 8 9p
Code0 1 2 3 4 5 6 7 8 9
36
d
Stack0 1 2 3 4 5 6 7 8 9p
Code0 1 2 3 4 5 6 7 8 9
2
d
Stack
0 1 2 3 4 5 6 7 8 9
34
2
d
Stack0 1 2 3 4 5 6 7 8 9p
Code
Training
32
0 1 2 3 4 5 6 7 8 9p
Code0 1 2 3 4 5 6 7 8 9
42
3
d
Stack
Program Interpreter0 1 2 3 4 5 6 7 8 9
Heap
Bosnjak et al. ICML 2017
Reader
Model
Isabel uploaded 2 pictures from her phone and 4 from her camera to facebook. She sorted the pics into 3 different albums with the same amount of pics in each album.
How many pictures were in each of the albums?
0 1 2 3 4 5 6 7 8 9p
Code0 1 2 3 4 5 6 7 8 9
63
d
Stack0 1 2 3 4 5 6 7 8 9p
Code0 1 2 3 4 5 6 7 8 9
36
d
Stack0 1 2 3 4 5 6 7 8 9p
Code0 1 2 3 4 5 6 7 8 9
2
d
Stack
0 1 2 3 4 5 6 7 8 9
34
2
d
Stack0 1 2 3 4 5 6 7 8 9p
Code
Training
33
0 1 2 3 4 5 6 7 8 9p
Code0 1 2 3 4 5 6 7 8 9
42
3
d
Stack
Program Interpreter0 1 2 3 4 5 6 7 8 9
Heap
Bosnjak et al. ICML 2017
Reader
Model
Isabel uploaded 2 pictures from her phone and 4 from her camera to facebook. She sorted the pics into 3 different albums with the same amount of pics in each album.
How many pictures were in each of the albums?
0 1 2 3 4 5 6 7 8 9p
Code0 1 2 3 4 5 6 7 8 9
63
d
Stack0 1 2 3 4 5 6 7 8 9p
Code0 1 2 3 4 5 6 7 8 9
36
d
Stack0 1 2 3 4 5 6 7 8 9p
Code0 1 2 3 4 5 6 7 8 9
2
d
Stackdef solve(x):
{+|-|%|*} solve(y)
Results on Benchmark (Accuracy)
34Bosnjak et al. ICML 2017
50
62.5
75
87.5
100
Roy & Roth (2015) Ours Seq2Seq
9596
55.5
How many pictures were in each of the albums?
(2 + 4) / 3
Seq2Seq solves simpler problem
Limitations
Continuous relaxations difficult to train
Hard to learn long programs
Hard to learn with recursive function calls
Need better gradients in presence of discrete variables
35
Part 3: Learning to Aggregate
36
Tim Rocktäschel
Neural Theorem Provers
37
Pituitary ACTH hypersecretion ... is a form of hyperpituitarism characterized by an abnormally high level of ACTH produced by the anterior pituitary …
Which medical specialty deals with pituitary ACTH hypersecretion?
A major organ of the endocrine system, the anterior pituitary is the glandular, anterior lobe that ...
The endocrine system is ... ... The field of study dealing with the endocrine system and its disorders is endocrinology, a branch of internal medicine.
Neural Theorem Provers
38
Pituitary ACTH hypersecretion ... is a form of hyperpituitarism characterized by an abnormally high level of ACTH produced by the anterior pituitary …
Which medical specialty deals with pituitary ACTH hypersecretion?
A major organ of the endocrine system, the anterior pituitary is the glandular, anterior lobe that ...
The endocrine system is ... ... The field of study dealing with the endocrine system and its disorders is endocrinology, a branch of internal medicine.
Rocktäschel and Riedel NIPS 2017
Agent
Program Interpreter
Reader
Question
Which specialty deals with pituitary ACTH hypersecretion?
0 1 2 3 4 5 6 7 8 9p
Code
0 1 2 3 4 5 6 7 8 9p
Code
Neural Theorem Provers
39
Pituitary ACTH hypersecretion ... is a form of hyperpituitarism characterized by an abnormally high level of ACTH produced by the anterior pituitary …
A major organ of the endocrine system, the anterior pituitary is the glandular, anterior lobe that ...
The endocrine system is ... ... The field of study dealing with the endocrine system and its disorders is endocrinology, a branch of internal medicine.
Rocktäschel and Riedel NIPS 2017
Agent
Program Interpreter
Reader
Question
Which specialty deals with pituitary ACTH hypersecretion?
Question
What is ACTH hypersecretion created by?
0 1 2 3 4 5 6 7 8 9p
Code
Which medical specialty deals with pituitary ACTH hypersecretion?
Question
Which specialty deals with pituitary ACTH hypersecretion?
0 1 2 3 4 5 6 7 8 9p
Code
Neural Theorem Provers
40
Pituitary ACTH hypersecretion ... is a form of hyperpituitarism characterized by an abnormally high level of ACTH produced by the anterior pituitary …
A major organ of the endocrine system, the anterior pituitary is the glandular, anterior lobe that ...
The endocrine system is ... ... The field of study dealing with the endocrine system and its disorders is endocrinology, a branch of internal medicine.
Rocktäschel and Riedel NIPS 2017
Agent
Program Interpreter
Reader
Question
Which specialty deals with pituitary ACTH hypersecretion?
0 1 2 3 4 5 6 7 8 9p
CodeQuestion
What is the pituitary a part of?
0 1 2 3 4 5 6 7 8 9p
Code
Which medical specialty deals with pituitary ACTH hypersecretion?
Question
Which specialty deals with pituitary ACTH hypersecretion?
0 1 2 3 4 5 6 7 8 9p
Code
Neural Theorem Provers
41
Pituitary ACTH hypersecretion ... is a form of hyperpituitarism characterized by an abnormally high level of ACTH produced by the anterior pituitary …
A major organ of the endocrine system, the anterior pituitary is the glandular, anterior lobe that ...
The endocrine system is ... ... The field of study dealing with the endocrine system and its disorders is endocrinology, a branch of internal medicine.
Rocktäschel and Riedel NIPS 2017
Agent
Program Interpreter
Reader
Question
Which specialty deals with pituitary ACTH hypersecretion?
0 1 2 3 4 5 6 7 8 9p
CodeQuestion
What is the pituitary a part of?
0 1 2 3 4 5 6 7 8 9p
CodeQuestion
What topic covers the endocrine system?
0 1 2 3 4 5 6 7 8 9p
Code
Which medical specialty deals with pituitary ACTH hypersecretion?
Vectors Correspond to Interpretable Rules
42
Pituitary ACTH hypersecretion ... is a form of hyperpituitarism characterized by an abnormally high level of ACTH produced by the anterior pituitary …
A major organ of the endocrine system, the anterior pituitary is the glandular, anterior lobe that ...
The endocrine system is ... ... The field of study dealing with the endocrine system and its disorders is endocrinology, a branch of internal medicine.
Rocktäschel and Riedel NIPS 2017
Agent
Program Interpreter
Reader
Question
Which specialty deals with pituitary ACTH hypersecretion?
0 1 2 3 4 5 6 7 8 9p
CodeQuestion
What is the pituitary a part of?
0 1 2 3 4 5 6 7 8 9p
CodeQuestion
What topic covers the endocrine system?
0 1 2 3 4 5 6 7 8 9p
Code
X deals with Y if Y produced by Z Z is a part of U X deals with U
Which medical specialty deals with pituitary ACTH hypersecretion?
Catch: Currently only Works on Relational Data
43
createdBy(hypersecretion, anterior pituitary)dealsWith(hypersecretion, X)
partOf(endocrine system, anterior pituitary)
dealsWith(endocrine system, endocrinology)
Rocktäschel and Riedel NIPS 2017
Agent
Program Interpreter
Reader
Question
Which specialty deals with pituitary ACTH hypersecretion?
0 1 2 3 4 5 6 7 8 9p
CodeQuestion
What is the pituitary a part of?
0 1 2 3 4 5 6 7 8 9p
CodeQuestion
covers(endocrine system, X)
0 1 2 3 4 5 6 7 8 9p
Code
X deals with Y if Y produced by Z Z is a part of U X deals with U
Catch: Currently only Works on Relational Data
44
createdBy(hypersecretion, anterior pituitary)dealsWith(hypersecretion, X)
partOf(endocrine system, anterior pituitary)
dealsWith(endocrine system, endocrinology)
Rocktäschel and Riedel NIPS 2017
Agent
Amounts to Differentiable Version of the Backward Chaining algorithm used in Prolog
Supports Soft Unification
45
createdBy(hypersecretion, anterior pituitary)dealsWith(hypersecretion, X)
partOf(endocrine system, anterior pituitary)
dealsWith(endocrine system, endocrinology)
Rocktäschel and Riedel NIPS 2017
Agent
Program Interpreter
Reader
Question
Which specialty deals with pituitary ACTH hypersecretion?
0 1 2 3 4 5 6 7 8 9p
CodeQuestion
What is the pituitary a part of?
0 1 2 3 4 5 6 7 8 9p
CodeQuestion
covers(endocrine system, X)
0 1 2 3 4 5 6 7 8 9p
Code
X deals with Y if Y produced by Z Z is a part of U X deals with U
Results on Benchmark (Rank of Correct Answer)
46
0
25
50
75
100
Countries S3 UMLS NationsComplex NTP Complex NTP Complex NTP
8999
77.386
96
48.4
Comparable or Better than Baselines, and interpretable
Rocktäschel and Riedel NIPS 2017
Interpretability: Learnt Rules
if X is located in Y and Y is located in Z then X is located in Z
if X expels diplomats of Y then X shows negative behaviour towards Y
if X interacts with Y and Y interacts with Z then X interacts with Z
47
Limitations
Scalability
Currently only works for KBs with < 10k facts
Small proof depth
Still requires relational representation
48
Conclusion
Great Progress in End-to-End Reading Comprehension
Reasoning (aggregation, calculation etc.) end-to-end is still very challenging
Our Approaches
create datasets
cast reasoning as program learning and execution
are end-to-end differentiable (can be trained on downstream loss)
are inspired and tied to traditional symbolic formalisms (Forth, Prolog/Datalog)
are learnt models are interpretable
allow injection of prior knowledge
49