Hyperbolic Neur al Ne tworks - 2020 Conference06-15...Poincaré Embeddings for Learning Hierarchical...
26
Hyperbolic Neural Networks Hyperbolic Neural Networks
Transcript of Hyperbolic Neur al Ne tworks - 2020 Conference06-15...Poincaré Embeddings for Learning Hierarchical...
Hyperbolic Neural NetworksHyperbolic Neural Networks
Use hyperbolic space instead of Euclidean spacefor embedding data with a latent hierarchical structure
imag
e so
urce
: htt
p://
insp
irehe
p.ne
t/re
cord
/135
5197
/plo
ts
The volume of a ball growsexponentially with its
radius!
Use hyperbolic space instead of Euclidean spacefor embedding data with a latent hierarchical structure
imag
e so
urce
: htt
p://
insp
irehe
p.ne
t/re
cord
/135
5197
/plo
ts
The volume of a ball growsexponentially with its
radius!
Use hyperbolic space instead of Euclidean spacefor embedding data with a latent hierarchical structure
Imag
e so
urce
: htt
p://
prio
r.si
gchi
.org
Similarly as for a tree: the number of nodesgrows exponentially with the tree depth!
Image source: http://prior.sigchi.org
Hot topic in ML since
Poincaré Embeddings for LearningHierarchical Representations, Nickel & Kiela, (NIPS 2017)
Use hyperbolic space instead of Euclidean spacefor embedding data with a latent hierarchical structure
Poincaré BallPoincaré Ball
Poincaré BallPoincaré Ball
Poincaré BallPoincaré Ball
Our contributionsOur contributions
Image sources: stackexchange.com , wikipedia.org
exp (v)x
Our contributionsOur contributions
Our contributionsOur contributions
Our contributionsOur contributions
Our contributionsOur contributions
Riemannian OptimizationRiemannian Optimization
Both Euclidean and hyperbolic parameters
Riemannian SGD:
Riemannian gradient:
x ← exp (−η∇ L), x ∈xc
xR D c
n
∇ L =xR (1/λ ) ∇ L, conformal factor λ =x
c 2x x
c
1 − c∥x∥2
2
exp (v)x
Image source: stackexchange.com
ExperimentsExperiments
All word and sentence embeddings have dimension 5.
ExperimentsExperiments
ExperimentsExperiments
ExperimentsExperiments
THANK YOU!THANK YOU!
hyperbolicdeeplearning.com
Please visit our website:
Octavian Ganea is currently looking for postdoctoral positions!
Matrix-vector multiplication
We define:
Nice properties:
When the curvature c goes to zero, it recovers the usualmatrix multiplication!
lim M (x) =c→0⊗ c Mx
Matrix-vector multiplication
