Knowledge Graph Reasoning: Recent Advanceswilliam/papers/Part3_KB_Reasoning.pdf · Knowledge Graph...
Transcript of Knowledge Graph Reasoning: Recent Advanceswilliam/papers/Part3_KB_Reasoning.pdf · Knowledge Graph...
![Page 1: Knowledge Graph Reasoning: Recent Advanceswilliam/papers/Part3_KB_Reasoning.pdf · Knowledge Graph Reasoning: Recent Advances William Wang Department of Computer Science CIPS Summer](https://reader034.fdocuments.in/reader034/viewer/2022042115/5e92330dd91a25184874ddc4/html5/thumbnails/1.jpg)
Knowledge Graph Reasoning:Recent Advances
William Wang
Department of Computer Science
CIPS Summer School 2018
Beijing, China
1/20
![Page 2: Knowledge Graph Reasoning: Recent Advanceswilliam/papers/Part3_KB_Reasoning.pdf · Knowledge Graph Reasoning: Recent Advances William Wang Department of Computer Science CIPS Summer](https://reader034.fdocuments.in/reader034/viewer/2022042115/5e92330dd91a25184874ddc4/html5/thumbnails/2.jpg)
Agenda
• Motivation
• Path-Based Reasoning• Embedding-Based Reasoning• Bridging Path-Based and Embedding-Based
Reasoning: DeepPath, MINERVA, and DIVA
• Conclusions• Other Research Activities at UCSB NLP
2
![Page 3: Knowledge Graph Reasoning: Recent Advanceswilliam/papers/Part3_KB_Reasoning.pdf · Knowledge Graph Reasoning: Recent Advances William Wang Department of Computer Science CIPS Summer](https://reader034.fdocuments.in/reader034/viewer/2022042115/5e92330dd91a25184874ddc4/html5/thumbnails/3.jpg)
Knowledge Graphs are Not Complete
3
BandofBrothers
Mini-Series
HBO
tvProgramCreator tvProgramGenre
GrahamYost
writtenBy music
UnitedStates
countryOfOrigin
NealMcDonough
English
TomHanks
awardWorkWinnercastActor
...
professionpersonLanguages
CaesarsEntertain…
serviceLocation-1
Actor
MichaelKamen
![Page 4: Knowledge Graph Reasoning: Recent Advanceswilliam/papers/Part3_KB_Reasoning.pdf · Knowledge Graph Reasoning: Recent Advances William Wang Department of Computer Science CIPS Summer](https://reader034.fdocuments.in/reader034/viewer/2022042115/5e92330dd91a25184874ddc4/html5/thumbnails/4.jpg)
Benefits of Knowledge Graph
• Support various applications• Structured Search• Question Answering• Dialogue Systems• Relation Extraction• Summarization
• Knowledge Graphs can be constructed via information extraction from text, but…
• There will be a lot of missing links.• Goal: complete the knowledge graph.
4
![Page 5: Knowledge Graph Reasoning: Recent Advanceswilliam/papers/Part3_KB_Reasoning.pdf · Knowledge Graph Reasoning: Recent Advances William Wang Department of Computer Science CIPS Summer](https://reader034.fdocuments.in/reader034/viewer/2022042115/5e92330dd91a25184874ddc4/html5/thumbnails/5.jpg)
Reasoning on Knowledge Graph
Query node: Band of brothers
Query relation: tvProgramLanguage
tvProgramLanguage(Band of Brothers, ?)
5
BandofBrothers
Mini-Series
HBO
tvProgramCreator tvProgramGenre
GrahamYost
writtenBy music
UnitedStates
countryOfOrigin
NealMcDonough
English
TomHanks
awardWorkWinnercastActor
...
professionpersonLanguages
CaesarsEntertain…
serviceLocation-1
Actor
MichaelKamen
![Page 6: Knowledge Graph Reasoning: Recent Advanceswilliam/papers/Part3_KB_Reasoning.pdf · Knowledge Graph Reasoning: Recent Advances William Wang Department of Computer Science CIPS Summer](https://reader034.fdocuments.in/reader034/viewer/2022042115/5e92330dd91a25184874ddc4/html5/thumbnails/6.jpg)
KB Reasoning Tasks
• Predicting the missing link.• Given e1 and e2, predict the relation r.
• Predicting the missing entity.• Given e1 and relation r, predict the missing entity e2.
• Fact Prediction.• Given a triple, predict whether it is true or false.
6
![Page 7: Knowledge Graph Reasoning: Recent Advanceswilliam/papers/Part3_KB_Reasoning.pdf · Knowledge Graph Reasoning: Recent Advances William Wang Department of Computer Science CIPS Summer](https://reader034.fdocuments.in/reader034/viewer/2022042115/5e92330dd91a25184874ddc4/html5/thumbnails/7.jpg)
Related Work
• Path-based methods• Path-Ranking Algorithm, Lao et al. 2011• ProPPR, Wang et al, 2013 (My PhD thesis)• Subgraph Feature Extraction, Gardner et al, 2015• RNN + PRA, Neelakantan et al, 2015• Chains of Reasoning, Das et al, 2017
7
Why do we need path-based methods?
It’s accurate and explainable!
![Page 8: Knowledge Graph Reasoning: Recent Advanceswilliam/papers/Part3_KB_Reasoning.pdf · Knowledge Graph Reasoning: Recent Advances William Wang Department of Computer Science CIPS Summer](https://reader034.fdocuments.in/reader034/viewer/2022042115/5e92330dd91a25184874ddc4/html5/thumbnails/8.jpg)
Path-Ranking Algorithm (Lao et al., 2011)
• 1. Run random walk with restarts to derive many paths.
• 2. Use supervised training to rank different paths.
8
![Page 9: Knowledge Graph Reasoning: Recent Advanceswilliam/papers/Part3_KB_Reasoning.pdf · Knowledge Graph Reasoning: Recent Advances William Wang Department of Computer Science CIPS Summer](https://reader034.fdocuments.in/reader034/viewer/2022042115/5e92330dd91a25184874ddc4/html5/thumbnails/9.jpg)
ProPPR (Wang et al., 2013;2015)
• ProPPR generalizes PRA with recursive probabilistic logic programs.
• You may use other relations to jointly infer this target relation.
9
![Page 10: Knowledge Graph Reasoning: Recent Advanceswilliam/papers/Part3_KB_Reasoning.pdf · Knowledge Graph Reasoning: Recent Advances William Wang Department of Computer Science CIPS Summer](https://reader034.fdocuments.in/reader034/viewer/2022042115/5e92330dd91a25184874ddc4/html5/thumbnails/10.jpg)
Chain of Reasoning (Das et al, 2017)
10
• 1. Use PRA to derive the path.
• 2. Use RNNs to perform reasoning of the target relation.
![Page 11: Knowledge Graph Reasoning: Recent Advanceswilliam/papers/Part3_KB_Reasoning.pdf · Knowledge Graph Reasoning: Recent Advances William Wang Department of Computer Science CIPS Summer](https://reader034.fdocuments.in/reader034/viewer/2022042115/5e92330dd91a25184874ddc4/html5/thumbnails/11.jpg)
Related Work
• Embedding-based method• RESCAL, Nickel et al, 2011• TransE, Bordes et al, 2013• Neural Tensor Network, Socher et al, 2013• TransR/CTransR, Lin et al, 2015• Complex Embeddings, Trouillon et al, 2016
11
Embedding methods allow us to compare, and find similar entities in the vector space.
![Page 12: Knowledge Graph Reasoning: Recent Advanceswilliam/papers/Part3_KB_Reasoning.pdf · Knowledge Graph Reasoning: Recent Advances William Wang Department of Computer Science CIPS Summer](https://reader034.fdocuments.in/reader034/viewer/2022042115/5e92330dd91a25184874ddc4/html5/thumbnails/12.jpg)
Bridging Path-Based and Embedding-Based Reasoning with Deep Reinforcement Learning:
DeepPath (Xiong et al., 2017)
12
![Page 13: Knowledge Graph Reasoning: Recent Advanceswilliam/papers/Part3_KB_Reasoning.pdf · Knowledge Graph Reasoning: Recent Advances William Wang Department of Computer Science CIPS Summer](https://reader034.fdocuments.in/reader034/viewer/2022042115/5e92330dd91a25184874ddc4/html5/thumbnails/13.jpg)
RL for KB Reasoning: DeepPath (Xiong et al., 2017)
Ø Learning the paths with RL, instead of using random walks with restart
Ø Model the path finding as a MDPØTrain a RL agent to find pathsØ Represent the KG with pretrained KG
embeddingsØ Use the learned paths as logical formulas
13
![Page 14: Knowledge Graph Reasoning: Recent Advanceswilliam/papers/Part3_KB_Reasoning.pdf · Knowledge Graph Reasoning: Recent Advances William Wang Department of Computer Science CIPS Summer](https://reader034.fdocuments.in/reader034/viewer/2022042115/5e92330dd91a25184874ddc4/html5/thumbnails/14.jpg)
Machine Learning
14
SupervisedLearning
UnsupervisedLearning
ReinforcementLearning
![Page 15: Knowledge Graph Reasoning: Recent Advanceswilliam/papers/Part3_KB_Reasoning.pdf · Knowledge Graph Reasoning: Recent Advances William Wang Department of Computer Science CIPS Summer](https://reader034.fdocuments.in/reader034/viewer/2022042115/5e92330dd91a25184874ddc4/html5/thumbnails/15.jpg)
Supervised v.s.Reinforcement
Supervised Learning◦ Trainingbasedonsupervisor/label/annotation◦ Feedbackisinstantaneous◦ Notmuchtemporalaspects
Reinforcement Learning◦ Trainingonlybasedonreward signal◦ Feedbackisdelayed◦ Timematters◦ Agentactionsaffectsubsequent exploration
15
![Page 16: Knowledge Graph Reasoning: Recent Advanceswilliam/papers/Part3_KB_Reasoning.pdf · Knowledge Graph Reasoning: Recent Advances William Wang Department of Computer Science CIPS Summer](https://reader034.fdocuments.in/reader034/viewer/2022042115/5e92330dd91a25184874ddc4/html5/thumbnails/16.jpg)
Reinforcement Learning
16
• RL is a general purpose framework for decisionmaking• ◦ RL is for an agentwith the capacity toact• ◦ Each actioninfluences the agent’s future state• ◦ Success is measured by a scalar rewardsignal• ◦ Goal: selectactionstomaximizefuturereward
![Page 17: Knowledge Graph Reasoning: Recent Advanceswilliam/papers/Part3_KB_Reasoning.pdf · Knowledge Graph Reasoning: Recent Advances William Wang Department of Computer Science CIPS Summer](https://reader034.fdocuments.in/reader034/viewer/2022042115/5e92330dd91a25184874ddc4/html5/thumbnails/17.jpg)
Reinforcement Learning
17
Environment
𝑎𝑡𝑠$%&
𝑟$%&
𝑠$𝑟$
Agent
Agent Environment
Multi-layerneuralnetsѱ(st ) KGmodeledasaMDP
![Page 18: Knowledge Graph Reasoning: Recent Advanceswilliam/papers/Part3_KB_Reasoning.pdf · Knowledge Graph Reasoning: Recent Advances William Wang Department of Computer Science CIPS Summer](https://reader034.fdocuments.in/reader034/viewer/2022042115/5e92330dd91a25184874ddc4/html5/thumbnails/18.jpg)
DeepPath: RL for KG Reasoning
18
![Page 19: Knowledge Graph Reasoning: Recent Advanceswilliam/papers/Part3_KB_Reasoning.pdf · Knowledge Graph Reasoning: Recent Advances William Wang Department of Computer Science CIPS Summer](https://reader034.fdocuments.in/reader034/viewer/2022042115/5e92330dd91a25184874ddc4/html5/thumbnails/19.jpg)
Components of MDP
• Markov decision process < 𝑆, 𝐴, 𝑃, 𝑅 >• 𝑆: continuousstatesrepresentedwithembeddings• 𝐴:actionspace(relations)• 𝑃 𝑆$%& = 𝑠F 𝑆$ = 𝑠, 𝐴$ = 𝑎 :transitionprobability• 𝑅 𝑠, 𝑎 : rewardreceivedforeachtakenstep
• Withpretrained KGembeddings• 𝑠$ = 𝑒$ ⊕ (𝑒$MNOP$ − 𝑒$)• 𝐴 = 𝑟&, 𝑟R, … , 𝑟T ,allrelationsintheKG
19
![Page 20: Knowledge Graph Reasoning: Recent Advanceswilliam/papers/Part3_KB_Reasoning.pdf · Knowledge Graph Reasoning: Recent Advances William Wang Department of Computer Science CIPS Summer](https://reader034.fdocuments.in/reader034/viewer/2022042115/5e92330dd91a25184874ddc4/html5/thumbnails/20.jpg)
Reward Functions
• Global Accuracy
• Path Efficiency
• Path Diversity
20
![Page 21: Knowledge Graph Reasoning: Recent Advanceswilliam/papers/Part3_KB_Reasoning.pdf · Knowledge Graph Reasoning: Recent Advances William Wang Department of Computer Science CIPS Summer](https://reader034.fdocuments.in/reader034/viewer/2022042115/5e92330dd91a25184874ddc4/html5/thumbnails/21.jpg)
Training with Policy Gradient
• Monte-Carlo Policy Gradient (REINFORCE, William, 1992)
21
![Page 22: Knowledge Graph Reasoning: Recent Advanceswilliam/papers/Part3_KB_Reasoning.pdf · Knowledge Graph Reasoning: Recent Advances William Wang Department of Computer Science CIPS Summer](https://reader034.fdocuments.in/reader034/viewer/2022042115/5e92330dd91a25184874ddc4/html5/thumbnails/22.jpg)
Challenge
ØTypical RL problemsqAtari games (Mnih et al., 2015): 4~18 valid actionsqAlphaGo (Silver et al. 2016): ~250 valid actionsq Knowledge Graph reasoning: >= 400 actions
Issue: q large action (search) space -> poor convergence
properties
22
![Page 23: Knowledge Graph Reasoning: Recent Advanceswilliam/papers/Part3_KB_Reasoning.pdf · Knowledge Graph Reasoning: Recent Advances William Wang Department of Computer Science CIPS Summer](https://reader034.fdocuments.in/reader034/viewer/2022042115/5e92330dd91a25184874ddc4/html5/thumbnails/23.jpg)
Supervised (Imitation) Policy Learning
§ Use randomized BFS to retrieve a few paths
§ Do imitation learning using the retrieved paths§ All the paths are assigned with +1 reward
23
![Page 24: Knowledge Graph Reasoning: Recent Advanceswilliam/papers/Part3_KB_Reasoning.pdf · Knowledge Graph Reasoning: Recent Advances William Wang Department of Computer Science CIPS Summer](https://reader034.fdocuments.in/reader034/viewer/2022042115/5e92330dd91a25184874ddc4/html5/thumbnails/24.jpg)
Datasets and Preprocessing
Dataset #ofEntities #ofRelations #of Triples #ofTasks
FB15k-237 14,505 237 310,116 20
NELL-995 75,492 200 154,213 12
24
Ø Datasetprocessingq Removeuselessrelations:haswikipediaurl,generalizations,etcq Addinverserelationlinkstotheknowledgegraphq Removethetripleswithtaskrelations
FB15k-237:SampledfromFB15k(Bordes etal.,2013),redundantrelationsremovesNELL-995:Sampledfromthe995th iterationofNELLsystem(Carlsonetal.,2010b)
![Page 25: Knowledge Graph Reasoning: Recent Advanceswilliam/papers/Part3_KB_Reasoning.pdf · Knowledge Graph Reasoning: Recent Advances William Wang Department of Computer Science CIPS Summer](https://reader034.fdocuments.in/reader034/viewer/2022042115/5e92330dd91a25184874ddc4/html5/thumbnails/25.jpg)
Effect of Supervised Policy Learning
25
• x-axis: numberoftrainingepochs• y-axis: successratio(probabilityofreachingthetarget)ontestset
->Re-traintheagentusingrewardfunctions
![Page 26: Knowledge Graph Reasoning: Recent Advanceswilliam/papers/Part3_KB_Reasoning.pdf · Knowledge Graph Reasoning: Recent Advances William Wang Department of Computer Science CIPS Summer](https://reader034.fdocuments.in/reader034/viewer/2022042115/5e92330dd91a25184874ddc4/html5/thumbnails/26.jpg)
Inference Using Learned Paths
§ Path as logical formula§ FilmCountry: actionFilm-1 -> personNationality§ PersonNationality: placeOfBirth -> locationContains-1
§ etc …
§ Bi-directional path-constrained search§ Check whether the formulas hold for entity pairs
26
… …
Uni-directionalsearch bi-directionalsearch
![Page 27: Knowledge Graph Reasoning: Recent Advanceswilliam/papers/Part3_KB_Reasoning.pdf · Knowledge Graph Reasoning: Recent Advances William Wang Department of Computer Science CIPS Summer](https://reader034.fdocuments.in/reader034/viewer/2022042115/5e92330dd91a25184874ddc4/html5/thumbnails/27.jpg)
Link Prediction Result
27
Tasks PRA Ours TransE TransRworksFor 0.681 0.711 0.677 0.692
atheletPlaysForTeam
0.987 0.955 0.896 0.784
athletePlaysInLeague
0.841 0.960 0.773 0.912
athleteHomeStadium
0.859 0.890 0.718 0.722
teamPlaysSports 0.791 0.738 0.761 0.814orgHirePerson 0.599 0.742 0.719 0.737personLeadsOrg 0.700 0.795 0.751 0.772
…
Overall 0.675 0.796 0.737 0.789
MeanaverageprecisiononNELL-995
![Page 28: Knowledge Graph Reasoning: Recent Advanceswilliam/papers/Part3_KB_Reasoning.pdf · Knowledge Graph Reasoning: Recent Advances William Wang Department of Computer Science CIPS Summer](https://reader034.fdocuments.in/reader034/viewer/2022042115/5e92330dd91a25184874ddc4/html5/thumbnails/28.jpg)
Qualitative Analysis
Path length distributions
28
![Page 29: Knowledge Graph Reasoning: Recent Advanceswilliam/papers/Part3_KB_Reasoning.pdf · Knowledge Graph Reasoning: Recent Advances William Wang Department of Computer Science CIPS Summer](https://reader034.fdocuments.in/reader034/viewer/2022042115/5e92330dd91a25184874ddc4/html5/thumbnails/29.jpg)
Qualitative Analysis
29
ExamplePaths
personNationality:
placeOfBirth ->locationContains-1
peoplePlaceLived ->locationContains-1
placeOfBirth ->locationContains
peopleMariage ->locationOfCeremony ->locationContains-1
tvProgramLanguage:
tvCountryOfOrigin ->countryOfficialLanguage
tvCountryOfOrigin ->filmReleaseRegion-1-> filmLanguage
tvCastActor ->personLanguage
athletePlaysForTeam:
athleteHomeStadium ->teamHomeStadium-1
atheleteLedSportsTeam
athletePlaysSports ->teamPlaysSports-1
![Page 30: Knowledge Graph Reasoning: Recent Advanceswilliam/papers/Part3_KB_Reasoning.pdf · Knowledge Graph Reasoning: Recent Advances William Wang Department of Computer Science CIPS Summer](https://reader034.fdocuments.in/reader034/viewer/2022042115/5e92330dd91a25184874ddc4/html5/thumbnails/30.jpg)
Bridging Path-Finding and Reasoning w. Variational Inference (teaser):
DIVA (Chen et al., NAACL 2018)
30
![Page 31: Knowledge Graph Reasoning: Recent Advanceswilliam/papers/Part3_KB_Reasoning.pdf · Knowledge Graph Reasoning: Recent Advances William Wang Department of Computer Science CIPS Summer](https://reader034.fdocuments.in/reader034/viewer/2022042115/5e92330dd91a25184874ddc4/html5/thumbnails/31.jpg)
DIVA: Variational KB Reasoning(Chen et al., NAACL 2018)
• Inferring latent paths connecting entity nodes.
31
UnitedStates
English
Condition(𝑒U, 𝑒V)
countrySpeakLanguage
ObservedVariable𝑟
�̅� = 𝑎𝑟𝑔𝑚𝑎𝑥\ log 𝑝(𝑟|𝑒U, 𝑒V)
𝑝(𝑟|𝑒U, 𝑒V)
![Page 32: Knowledge Graph Reasoning: Recent Advanceswilliam/papers/Part3_KB_Reasoning.pdf · Knowledge Graph Reasoning: Recent Advances William Wang Department of Computer Science CIPS Summer](https://reader034.fdocuments.in/reader034/viewer/2022042115/5e92330dd91a25184874ddc4/html5/thumbnails/32.jpg)
DIVA: Variational KB Reasoning(Chen et al., NAACL 2018)
• Inferring latent paths connecting entity nodes by parameterizing likelihood (path reasoning) and prior (path finding) with neural network modules.
32
tvProgramLanguage
𝑂𝑏𝑠𝑒𝑟𝑣𝑒𝑑𝑉𝑎𝑟𝑖𝑎𝑏𝑙𝑒
𝑝 = 𝑎𝑟𝑔𝑚𝑎𝑥\𝑝(𝑟|𝑒U, 𝑒V) = 𝑎𝑟𝑔𝑚𝑎𝑥\ loge 𝑝 𝑟 𝐿 𝑝(𝐿|𝑒U, 𝑒V)g
h
?
UnobservedVariableL
𝑟BandofBrothers
English
Condition
𝑒V
𝑒U
𝑝(𝐿|𝑒U, 𝑒V) 𝑝 𝑟 𝐿prior likelihood
![Page 33: Knowledge Graph Reasoning: Recent Advanceswilliam/papers/Part3_KB_Reasoning.pdf · Knowledge Graph Reasoning: Recent Advances William Wang Department of Computer Science CIPS Summer](https://reader034.fdocuments.in/reader034/viewer/2022042115/5e92330dd91a25184874ddc4/html5/thumbnails/33.jpg)
• Marginal likelihoodlog∫ 𝑝 𝑟 𝐿 𝑝(𝐿|𝑒U, 𝑒V)�h is
intractable
• We resort to Variational Bayes by introduce a posterior distribution 𝑞 𝐿 𝑒U, 𝑒V, 𝑟
33DPKingma et al. 2013
log p r 𝑒U, 𝑒V
𝔼m 𝐿 𝑒U, 𝑒V, 𝑟 log 𝑝 𝑟 𝐿
𝐾𝐿(𝑞 𝐿 𝑒U, 𝑒V, 𝑟 ||𝑝(𝐿|𝑒U, 𝑒V))
≥ −𝐸𝐿𝐵𝑂
DIVA: Variational KB Reasoning(Chen et al., NAACL 2018)
![Page 34: Knowledge Graph Reasoning: Recent Advanceswilliam/papers/Part3_KB_Reasoning.pdf · Knowledge Graph Reasoning: Recent Advances William Wang Department of Computer Science CIPS Summer](https://reader034.fdocuments.in/reader034/viewer/2022042115/5e92330dd91a25184874ddc4/html5/thumbnails/34.jpg)
Parameterization – Path-finder
• Approximate posterior 𝑞r 𝐿 𝑒U, 𝑒V, 𝑟 and prior 𝑝s 𝐿 𝑒U, 𝑒V : parameterize with RNN
34
𝑒U 𝑒t 𝑒V𝑎t
𝑒t
𝑒& 𝑒R𝑒u
𝒂𝟏 𝒂R
𝒂uKG
𝑒&
𝑒R
𝑒u
TransitionProbability:𝑝(𝑎t%&, 𝑒t%&|𝑎&:t, 𝑒&:t)
𝒂R
![Page 35: Knowledge Graph Reasoning: Recent Advanceswilliam/papers/Part3_KB_Reasoning.pdf · Knowledge Graph Reasoning: Recent Advances William Wang Department of Computer Science CIPS Summer](https://reader034.fdocuments.in/reader034/viewer/2022042115/5e92330dd91a25184874ddc4/html5/thumbnails/35.jpg)
Parameterization – Path Reasoner
• Likelihood 𝑝x(𝑟|𝐿) : parameterize with CNN
35
𝑒U 𝑒V𝑒&
𝑒R
𝑒u
𝑒U
𝑒U
𝑒V
𝑒V
𝑟𝑒𝑙𝑎𝑡𝑖𝑜𝑛&
𝑟𝑒𝑙𝑎𝑡𝑖𝑜𝑛R
𝑟𝑒𝑙𝑎𝑡𝑖𝑜𝑛u
𝐶𝑁𝑁 𝑆𝑜𝑓𝑡𝑚𝑎𝑥
![Page 36: Knowledge Graph Reasoning: Recent Advanceswilliam/papers/Part3_KB_Reasoning.pdf · Knowledge Graph Reasoning: Recent Advances William Wang Department of Computer Science CIPS Summer](https://reader034.fdocuments.in/reader034/viewer/2022042115/5e92330dd91a25184874ddc4/html5/thumbnails/36.jpg)
• Training
36
relation
𝑒V𝑒U
𝑝x(𝑟|𝐿)𝑞r(𝐿)
KGconnectedPath
𝑒U 𝑒V
𝑝s(𝐿)
KL-divergence
KGconnectedPath
𝑒U 𝑒V
𝔼m~ 𝐿 𝑒U, 𝑒V, 𝑟 log 𝑝x 𝑟 𝐿
relation
𝐾𝐿(𝑞r 𝐿 𝑒U, 𝑒V, 𝑟 ||𝑝s(𝐿|𝑒U, 𝑒V))
𝑝𝑜𝑠𝑡𝑒𝑟𝑖𝑜𝑟: 𝑞r, likelihood: 𝑝x 𝑟 𝐿 , prior: 𝑝s(𝐿|𝑒U, 𝑒V)
DIVA: Variational KB Reasoning(Chen et al., NAACL 2018)
![Page 37: Knowledge Graph Reasoning: Recent Advanceswilliam/papers/Part3_KB_Reasoning.pdf · Knowledge Graph Reasoning: Recent Advances William Wang Department of Computer Science CIPS Summer](https://reader034.fdocuments.in/reader034/viewer/2022042115/5e92330dd91a25184874ddc4/html5/thumbnails/37.jpg)
• Testing
37
𝑒V𝑒U
𝑝s(𝐿)
KGconnectedPath
𝑒U 𝑒V
relation
𝑝𝑜𝑠𝑡𝑒𝑟𝑖𝑜𝑟: 𝑞r, likelihood: 𝑝x 𝑟 𝐿 , prior: 𝑝s(𝐿|𝑒U, 𝑒V)
DIVA: Variational KB Reasoning(Chen et al., NAACL 2018)
![Page 38: Knowledge Graph Reasoning: Recent Advanceswilliam/papers/Part3_KB_Reasoning.pdf · Knowledge Graph Reasoning: Recent Advances William Wang Department of Computer Science CIPS Summer](https://reader034.fdocuments.in/reader034/viewer/2022042115/5e92330dd91a25184874ddc4/html5/thumbnails/38.jpg)
Conclusions
• Embedding-based methods are very scalable and robust.
• Path-based methods are more interpretable.• There are some recent efforts in unifying
embedding and path-based approaches.
• DIVA integrates path-finding and reasoning in a principled variational inference framework.
38
![Page 39: Knowledge Graph Reasoning: Recent Advanceswilliam/papers/Part3_KB_Reasoning.pdf · Knowledge Graph Reasoning: Recent Advances William Wang Department of Computer Science CIPS Summer](https://reader034.fdocuments.in/reader034/viewer/2022042115/5e92330dd91a25184874ddc4/html5/thumbnails/39.jpg)
UCSB NLP
39
•Natural Language Processing• Information Extraction: relation extraction, and distant supervision.• Summarization: abstractive summarization.• Social Media: non-standard English expressions.• Language & Vision: action/relation detection, and video captioning.• Spoken Language Processing: task-oriented neural dialogue systems.
•Machine Learning• Statistical Relational Learning: neural symbolic reasoning.• Deep Learning: sequence-to-sequence models.• Structure Learning: learning the structures for neural models.• Reinforcement Learning: efficient and effective methods for DRL
and NLP.•Artificial Intelligence
• Knowledge Representation & Reasoning: beyond Freebase/OpenIE.• Knowledge Graphs: construction, completion, and reasoning.
![Page 40: Knowledge Graph Reasoning: Recent Advanceswilliam/papers/Part3_KB_Reasoning.pdf · Knowledge Graph Reasoning: Recent Advances William Wang Department of Computer Science CIPS Summer](https://reader034.fdocuments.in/reader034/viewer/2022042115/5e92330dd91a25184874ddc4/html5/thumbnails/40.jpg)
Other Research Activities at UCSB’s NLP Group
40
![Page 41: Knowledge Graph Reasoning: Recent Advanceswilliam/papers/Part3_KB_Reasoning.pdf · Knowledge Graph Reasoning: Recent Advances William Wang Department of Computer Science CIPS Summer](https://reader034.fdocuments.in/reader034/viewer/2022042115/5e92330dd91a25184874ddc4/html5/thumbnails/41.jpg)
Natural Language Generation
41
![Page 42: Knowledge Graph Reasoning: Recent Advanceswilliam/papers/Part3_KB_Reasoning.pdf · Knowledge Graph Reasoning: Recent Advances William Wang Department of Computer Science CIPS Summer](https://reader034.fdocuments.in/reader034/viewer/2022042115/5e92330dd91a25184874ddc4/html5/thumbnails/42.jpg)
Reinforced Conditional Variational Autoencoder for Generating Emotional Sentences (Zhou and Wang, ACL 2018)
https://arxiv.org/abs/1711.04090
42
![Page 43: Knowledge Graph Reasoning: Recent Advanceswilliam/papers/Part3_KB_Reasoning.pdf · Knowledge Graph Reasoning: Recent Advances William Wang Department of Computer Science CIPS Summer](https://reader034.fdocuments.in/reader034/viewer/2022042115/5e92330dd91a25184874ddc4/html5/thumbnails/43.jpg)
Controlling Emotions for RC-VAE Generated Sentences
43
Generated by Seq2Seq Baseline
i ’m sorry you ’re going to be
missed it
i ’m sorry for your loss
User’s Inputsorry guys , was gunna stream
tonight but i ’m still feeling sick
Designated Emojis
Generated By MojiTalk
hope you are okay hun !
hi jason , i ’ll be praying for you
![Page 44: Knowledge Graph Reasoning: Recent Advanceswilliam/papers/Part3_KB_Reasoning.pdf · Knowledge Graph Reasoning: Recent Advances William Wang Department of Computer Science CIPS Summer](https://reader034.fdocuments.in/reader034/viewer/2022042115/5e92330dd91a25184874ddc4/html5/thumbnails/44.jpg)
Hierarchical Deep Reinforcement Learning for Video Captioning (Wang et al., CVPR 2018)
44
![Page 45: Knowledge Graph Reasoning: Recent Advanceswilliam/papers/Part3_KB_Reasoning.pdf · Knowledge Graph Reasoning: Recent Advances William Wang Department of Computer Science CIPS Summer](https://reader034.fdocuments.in/reader034/viewer/2022042115/5e92330dd91a25184874ddc4/html5/thumbnails/45.jpg)
Deep Multimodal Video Captioning(Wang et al., NAACL 2018)
45
![Page 46: Knowledge Graph Reasoning: Recent Advanceswilliam/papers/Part3_KB_Reasoning.pdf · Knowledge Graph Reasoning: Recent Advances William Wang Department of Computer Science CIPS Summer](https://reader034.fdocuments.in/reader034/viewer/2022042115/5e92330dd91a25184874ddc4/html5/thumbnails/46.jpg)
Adversarial Reward Learning for Visual Storytelling (Wang et al., ACL 2018)
46
![Page 47: Knowledge Graph Reasoning: Recent Advanceswilliam/papers/Part3_KB_Reasoning.pdf · Knowledge Graph Reasoning: Recent Advances William Wang Department of Computer Science CIPS Summer](https://reader034.fdocuments.in/reader034/viewer/2022042115/5e92330dd91a25184874ddc4/html5/thumbnails/47.jpg)
Computational Social Science
47
![Page 48: Knowledge Graph Reasoning: Recent Advanceswilliam/papers/Part3_KB_Reasoning.pdf · Knowledge Graph Reasoning: Recent Advances William Wang Department of Computer Science CIPS Summer](https://reader034.fdocuments.in/reader034/viewer/2022042115/5e92330dd91a25184874ddc4/html5/thumbnails/48.jpg)
Learning to Generate Slang Explanations (Ke Ni, IJCNLP 2017)
48
![Page 49: Knowledge Graph Reasoning: Recent Advanceswilliam/papers/Part3_KB_Reasoning.pdf · Knowledge Graph Reasoning: Recent Advances William Wang Department of Computer Science CIPS Summer](https://reader034.fdocuments.in/reader034/viewer/2022042115/5e92330dd91a25184874ddc4/html5/thumbnails/49.jpg)
Automatic Generation of Slang Words(Kulkarni and Wang, NAACL 2018)
49
![Page 50: Knowledge Graph Reasoning: Recent Advanceswilliam/papers/Part3_KB_Reasoning.pdf · Knowledge Graph Reasoning: Recent Advances William Wang Department of Computer Science CIPS Summer](https://reader034.fdocuments.in/reader034/viewer/2022042115/5e92330dd91a25184874ddc4/html5/thumbnails/50.jpg)
Leveraging Intra-Speaker and Inter-Speaker Representation Learning for Hate Speech
Detection (Qian et al., NAACL 2018)
50
![Page 51: Knowledge Graph Reasoning: Recent Advanceswilliam/papers/Part3_KB_Reasoning.pdf · Knowledge Graph Reasoning: Recent Advances William Wang Department of Computer Science CIPS Summer](https://reader034.fdocuments.in/reader034/viewer/2022042115/5e92330dd91a25184874ddc4/html5/thumbnails/51.jpg)
Deep Reinforcement Learning
51
![Page 52: Knowledge Graph Reasoning: Recent Advanceswilliam/papers/Part3_KB_Reasoning.pdf · Knowledge Graph Reasoning: Recent Advances William Wang Department of Computer Science CIPS Summer](https://reader034.fdocuments.in/reader034/viewer/2022042115/5e92330dd91a25184874ddc4/html5/thumbnails/52.jpg)
Scheduled Policy Optimization(Xiong et al., IJCAI 2018)
52
![Page 53: Knowledge Graph Reasoning: Recent Advanceswilliam/papers/Part3_KB_Reasoning.pdf · Knowledge Graph Reasoning: Recent Advances William Wang Department of Computer Science CIPS Summer](https://reader034.fdocuments.in/reader034/viewer/2022042115/5e92330dd91a25184874ddc4/html5/thumbnails/53.jpg)
Combining Model-Free and Model Based Deep Reinforcement Learning
(Wang et al., ECCV 2018)
53
![Page 54: Knowledge Graph Reasoning: Recent Advanceswilliam/papers/Part3_KB_Reasoning.pdf · Knowledge Graph Reasoning: Recent Advances William Wang Department of Computer Science CIPS Summer](https://reader034.fdocuments.in/reader034/viewer/2022042115/5e92330dd91a25184874ddc4/html5/thumbnails/54.jpg)
Structure Learning
54
![Page 55: Knowledge Graph Reasoning: Recent Advanceswilliam/papers/Part3_KB_Reasoning.pdf · Knowledge Graph Reasoning: Recent Advances William Wang Department of Computer Science CIPS Summer](https://reader034.fdocuments.in/reader034/viewer/2022042115/5e92330dd91a25184874ddc4/html5/thumbnails/55.jpg)
Learning Activation Functions in Deep Neural Networks (Conner Vercellino, NIPS MetaLearning)
55
![Page 56: Knowledge Graph Reasoning: Recent Advanceswilliam/papers/Part3_KB_Reasoning.pdf · Knowledge Graph Reasoning: Recent Advances William Wang Department of Computer Science CIPS Summer](https://reader034.fdocuments.in/reader034/viewer/2022042115/5e92330dd91a25184874ddc4/html5/thumbnails/56.jpg)
Acknowledgment
56
Sponsors: Adobe, Amazon, ByteDance, DARPA, Facebook, Google, IBM, LogMeIn, NVIDIA, Tencent.
![Page 57: Knowledge Graph Reasoning: Recent Advanceswilliam/papers/Part3_KB_Reasoning.pdf · Knowledge Graph Reasoning: Recent Advances William Wang Department of Computer Science CIPS Summer](https://reader034.fdocuments.in/reader034/viewer/2022042115/5e92330dd91a25184874ddc4/html5/thumbnails/57.jpg)
57
Thanks!Questions?nlp.cs.ucsb.edu
DeepPath Sourcecode:https://github.com/xwhan/DeepPathKBGANSourcecode:https://github.com/cai-lw/KBGANScheduledPolicyOptimization:https://github.com/xwhan/walk_the_blocksProPPR Sourcecode:https://github.com/TeamCohen/ProPPRARELSourcecode:https://github.com/littlekobe/AREL