Page 1
Max-margin learningNando de Freitas
Page 2
Outline of the lectureMax margin learning is an extremely powerful idea for learningfeatures with auxiliary tasks, and then use these features to solve taskswith few data. The goal of this lecture is for you to learn
Transfer, multi-task, and multi-instance learning Harnessing auxiliary tasks to learn features
Matchings Preferences Corruption
Formulations of multi-task learningApplications:
Cross-lingual embeddings Relation learning Question answeringMemory networks
Page 3
Embedding discrete objects in metric spaces
CodeWordsFormulaeLogical expressionsSymbolsDNA sequences
Page 5
[Kotzias, Denil & NdF, 2014]
Idea: learn embeddings (features) in onetask and transfer these to solve new tasks
Page 6
Deep Multi-Instance Learning
Page 7
Deep Multi-Instance Learning
Page 8
Auxiliary tasks to learn features that can betransferred to learn tasks with few labels
[From machine learning to machine reasoning, Leon Bottou]
1. Matchings
Page 9
Auxiliary tasks to learn features that can betransferred to learn tasks with few labels
[NLP almost from scratch, Ronan Collobert and colleagues]
2. Corruption
Page 10
Auxiliary tasks to learn features that can betransferred to learn tasks with few labels
1. Preferences
Page 11
Max-margin formulations
Page 12
Max-margin formulations
[A tutorial on energy based learning, Yann LeCun et al]
Page 13
Max-margin formulations
Hinge Loss unconstrained formulation:
Page 14
Max-margin formulations
Page 16
Example: Bi-lingual word embeddings
[Karl Hermann and Phil Blunsom, 2014]
Page 17
Siamese networks (Yann LeCun)
Semi-supervised deep learning (Jason Weston et al)
Page 18
Applications: Paraphrase detection
[Phil Blunsom et al]
Page 19
Transfer: Question answering
[Grefenstette, Blunsom, NdF, Hermann, 2014]
Page 20
Relation learning
Page 21
Relation learning
[Yoshua Bengio et al]
Page 22
Relation learning
[Yoshua Bengio et al]
Page 23
Relation learning
[Yoshua Bengio et al]
Page 24
Short term & long term memory
[From machine learning to machine reasoning, Leon Bottou]
Page 25
Memory networks
[Jason Weston, Sumit Chopra & Antoine Bordes, 2015]
Page 26
Memory networks
[Jason Weston, Sumit Chopra & Antoine Bordes, 2015]
Page 27
Memory networks
[Jason Weston, Sumit Chopra & Antoine Bordes, 2015]
Page 28
Memory networks
[Jason Weston, Sumit Chopra & Antoine Bordes, 2015]
Page 29
Memory networks
[Jason Weston, Sumit Chopra & Antoine Bordes, 2015]
Page 30
Next lecture
In the next lecture, we will look models for sequences of data.