A Geometric Perspective on Machine LearningXocuments, signals, financial time series, sequences = d...
Transcript of A Geometric Perspective on Machine LearningXocuments, signals, financial time series, sequences = d...
![Page 1: A Geometric Perspective on Machine LearningXocuments, signals, financial time series, sequences = d d,x(x ′) makes sense locally What is good global distance? What is global geometry/topology](https://reader030.fdocuments.in/reader030/viewer/2022041012/5ebdae07f710ee51347de94d/html5/thumbnails/1.jpg)
A Geometric Perspective onMachine Learning
Partha Niyogi
The University of ChicagoThanks: M. Belkin, A. Caponnetto, X. He, I. Matveeva, H. Narayanan, V. Sindhwani, S. Smale,
S. Weinberger
A Geometric Perspective onMachine Learning – p.1
![Page 2: A Geometric Perspective on Machine LearningXocuments, signals, financial time series, sequences = d d,x(x ′) makes sense locally What is good global distance? What is global geometry/topology](https://reader030.fdocuments.in/reader030/viewer/2022041012/5ebdae07f710ee51347de94d/html5/thumbnails/2.jpg)
High Dimensional Data
When can we avoid the curse of dimensionality?
Smoothnessrate ≈ (1/n)
sd
splines,kernel methods, L2 regularization...
Sparsitywavelets, L1 regularization, LASSO, compressed sensing..
Geometrygraphs, simplicial complexes, laplacians, diffusions
A Geometric Perspective onMachine Learning – p.2
![Page 3: A Geometric Perspective on Machine LearningXocuments, signals, financial time series, sequences = d d,x(x ′) makes sense locally What is good global distance? What is global geometry/topology](https://reader030.fdocuments.in/reader030/viewer/2022041012/5ebdae07f710ee51347de94d/html5/thumbnails/3.jpg)
Geometry and Data: The Central Dogma
Distribution of natural data is non-uniform andconcentrates around low-dimensional structures.
The shape (geometry) of the distribution can beexploited for efficient learning.
A Geometric Perspective onMachine Learning – p.3
![Page 4: A Geometric Perspective on Machine LearningXocuments, signals, financial time series, sequences = d d,x(x ′) makes sense locally What is good global distance? What is global geometry/topology](https://reader030.fdocuments.in/reader030/viewer/2022041012/5ebdae07f710ee51347de94d/html5/thumbnails/4.jpg)
Manifold Learning
Learning when data ∼ M ⊂ RN
Clustering: M → {1, . . . , k}connected components, min cut
Classification: M → {−1, +1}P on M×{−1, +1}
Dimensionality Reduction: f : M → Rn n << N
M unknown: what can you learn about M from data?e.g. dimensionality, connected componentsholes, handles, homologycurvature, geodesics
A Geometric Perspective onMachine Learning – p.4
![Page 5: A Geometric Perspective on Machine LearningXocuments, signals, financial time series, sequences = d d,x(x ′) makes sense locally What is good global distance? What is global geometry/topology](https://reader030.fdocuments.in/reader030/viewer/2022041012/5ebdae07f710ee51347de94d/html5/thumbnails/5.jpg)
An Acoustic Example
u(t) s(t)
l
A Geometric Perspective onMachine Learning – p.5
![Page 6: A Geometric Perspective on Machine LearningXocuments, signals, financial time series, sequences = d d,x(x ′) makes sense locally What is good global distance? What is global geometry/topology](https://reader030.fdocuments.in/reader030/viewer/2022041012/5ebdae07f710ee51347de94d/html5/thumbnails/6.jpg)
An Acoustic Example
u(t) s(t)
l
One Dimensional Air Flow
(i) ∂V∂x
= − Aρc2
∂P∂t
(ii) ∂P∂x
= − ρA
∂V∂t
V (x, t) = volume velocityP (x, t) = pressure
A Geometric Perspective onMachine Learning – p.5
![Page 7: A Geometric Perspective on Machine LearningXocuments, signals, financial time series, sequences = d d,x(x ′) makes sense locally What is good global distance? What is global geometry/topology](https://reader030.fdocuments.in/reader030/viewer/2022041012/5ebdae07f710ee51347de94d/html5/thumbnails/7.jpg)
Solutions
0.650.7
0.750.8
0.850.9
0.951
0.65
0.7
0.75
0.8
0.85
0.9
0.95
10.65
0.7
0.75
0.8
0.85
0.9
0.95
1
beta−1beta−3
beta
−7
u(t) =P∞
n=1 αn sin(nω0t) ∈ l2
s(t) =P∞
n=1 βn sin(nω0t) ∈ l2
A Geometric Perspective onMachine Learning – p.6
![Page 8: A Geometric Perspective on Machine LearningXocuments, signals, financial time series, sequences = d d,x(x ′) makes sense locally What is good global distance? What is global geometry/topology](https://reader030.fdocuments.in/reader030/viewer/2022041012/5ebdae07f710ee51347de94d/html5/thumbnails/8.jpg)
Formal Justification
Speechspeech ∈ l2 generated by vocal tract
Jansen and Niyogi (2005)
Visiongroup actions on object leading to different images
Donoho and Grimes (2004)
Roboticsconfiguration spaces in joint movements
Graphics
Manifold + Noise may be generic model in high dimensions.
A Geometric Perspective onMachine Learning – p.7
![Page 9: A Geometric Perspective on Machine LearningXocuments, signals, financial time series, sequences = d d,x(x ′) makes sense locally What is good global distance? What is global geometry/topology](https://reader030.fdocuments.in/reader030/viewer/2022041012/5ebdae07f710ee51347de94d/html5/thumbnails/9.jpg)
Take Home Message
Geometrically motivated approach to learningnonlinear, nonparametric, high dimensions
Emphasize the role of the Laplacian and Heat KernelSemi-supervised regression and classification
Clustering and Homology
Randomized Algorithms and Numerical Analysis
A Geometric Perspective onMachine Learning – p.8
![Page 10: A Geometric Perspective on Machine LearningXocuments, signals, financial time series, sequences = d d,x(x ′) makes sense locally What is good global distance? What is global geometry/topology](https://reader030.fdocuments.in/reader030/viewer/2022041012/5ebdae07f710ee51347de94d/html5/thumbnails/10.jpg)
Pattern Recognition
P on X × Y X = RN ; Y = {0, 1}, R
(xi, yi) labeled examples
find f : X → Y Ill Posed
A Geometric Perspective onMachine Learning – p.9
![Page 11: A Geometric Perspective on Machine LearningXocuments, signals, financial time series, sequences = d d,x(x ′) makes sense locally What is good global distance? What is global geometry/topology](https://reader030.fdocuments.in/reader030/viewer/2022041012/5ebdae07f710ee51347de94d/html5/thumbnails/11.jpg)
Simplicity
A Geometric Perspective onMachine Learning – p.10
![Page 12: A Geometric Perspective on Machine LearningXocuments, signals, financial time series, sequences = d d,x(x ′) makes sense locally What is good global distance? What is global geometry/topology](https://reader030.fdocuments.in/reader030/viewer/2022041012/5ebdae07f710ee51347de94d/html5/thumbnails/12.jpg)
Simplicity
A Geometric Perspective onMachine Learning – p.10
![Page 13: A Geometric Perspective on Machine LearningXocuments, signals, financial time series, sequences = d d,x(x ′) makes sense locally What is good global distance? What is global geometry/topology](https://reader030.fdocuments.in/reader030/viewer/2022041012/5ebdae07f710ee51347de94d/html5/thumbnails/13.jpg)
Regularization Principle
f = arg minf∈HK
1
n
n∑
i=1
(yi − f(xi))2 + γ‖f‖2
K
Splines
Ridge Regression
SVM
K : X × X → R is a p.d. kernele.g. e
−‖x−y‖2
σ2 , (1 + x · y)d, etc.
HK is a corresponding RKHSe.g., certain Sobolev spaces, polynomial families, etc.
A Geometric Perspective onMachine Learning – p.11
![Page 14: A Geometric Perspective on Machine LearningXocuments, signals, financial time series, sequences = d d,x(x ′) makes sense locally What is good global distance? What is global geometry/topology](https://reader030.fdocuments.in/reader030/viewer/2022041012/5ebdae07f710ee51347de94d/html5/thumbnails/14.jpg)
Simplicity is Relative
A Geometric Perspective onMachine Learning – p.12
![Page 15: A Geometric Perspective on Machine LearningXocuments, signals, financial time series, sequences = d d,x(x ′) makes sense locally What is good global distance? What is global geometry/topology](https://reader030.fdocuments.in/reader030/viewer/2022041012/5ebdae07f710ee51347de94d/html5/thumbnails/15.jpg)
Simplicity is Relative
A Geometric Perspective onMachine Learning – p.12
![Page 16: A Geometric Perspective on Machine LearningXocuments, signals, financial time series, sequences = d d,x(x ′) makes sense locally What is good global distance? What is global geometry/topology](https://reader030.fdocuments.in/reader030/viewer/2022041012/5ebdae07f710ee51347de94d/html5/thumbnails/16.jpg)
Intuitions
supp PX has manifold structure
geodesic distance v/s ambient distance
geometric structure of data should be incorporated
f versus fM
A Geometric Perspective onMachine Learning – p.13
![Page 17: A Geometric Perspective on Machine LearningXocuments, signals, financial time series, sequences = d d,x(x ′) makes sense locally What is good global distance? What is global geometry/topology](https://reader030.fdocuments.in/reader030/viewer/2022041012/5ebdae07f710ee51347de94d/html5/thumbnails/17.jpg)
Manifold Regularization
minf∈HK
1
n
n∑
i=1
(yi − f(xi))2 + γA‖f‖2
K + γI‖f‖2I
‖f‖2I =
LaplacianR
〈gradMf, gradMf〉 =R
f∆Mf
Iterated LaplacianR
f∆Mif
Heat kernel e−∆Mt
Differential OperatorR
f(Df)
Representer Theorem: f =Pn
i=1 αiK(x, xi) +R
M α(y)K(x, y)
Belkin, Niyogi, Sindhwani (2004)
A Geometric Perspective onMachine Learning – p.14
![Page 18: A Geometric Perspective on Machine LearningXocuments, signals, financial time series, sequences = d d,x(x ′) makes sense locally What is good global distance? What is global geometry/topology](https://reader030.fdocuments.in/reader030/viewer/2022041012/5ebdae07f710ee51347de94d/html5/thumbnails/18.jpg)
Approximating ‖f‖2I
M is unknown but x1 . . . xM ∈ M
‖f‖2I =
∫
M
〈∇Mf,∇Mf〉 ≈∑
i∼j
Wij(f(xi) − f(xj))2
A Geometric Perspective onMachine Learning – p.15
![Page 19: A Geometric Perspective on Machine LearningXocuments, signals, financial time series, sequences = d d,x(x ′) makes sense locally What is good global distance? What is global geometry/topology](https://reader030.fdocuments.in/reader030/viewer/2022041012/5ebdae07f710ee51347de94d/html5/thumbnails/19.jpg)
Manifolds and Graphs
M ≈ G = (V,E)
eij ∈ E if ‖xi − xj‖ < ǫ
Wij = e−‖xi−xj‖2
t
∆M ≈ L = D − W
∫
〈gradf, gradf〉 ≈ ∑
i,j Wij(f(xi) − f(xj))2
∫
f(∆f) ≈ fT Lf
A Geometric Perspective onMachine Learning – p.16
![Page 20: A Geometric Perspective on Machine LearningXocuments, signals, financial time series, sequences = d d,x(x ′) makes sense locally What is good global distance? What is global geometry/topology](https://reader030.fdocuments.in/reader030/viewer/2022041012/5ebdae07f710ee51347de94d/html5/thumbnails/20.jpg)
Manifold Regularization
1
n
n∑
i=1
V (f(xi), yi) + γA‖f‖2K + γI
∑
i∼j
Wij(f(xi) − f(xj))2
Representer Theorem: fopt =∑n+m
i=1 αiK(x, xi)
V (f(x), y) = (f(x) − y)2: Least squares
V (f(x), y) = (1 − yf(x))+: Hinge loss (Support Vector Machines)
A Geometric Perspective onMachine Learning – p.17
![Page 21: A Geometric Perspective on Machine LearningXocuments, signals, financial time series, sequences = d d,x(x ′) makes sense locally What is good global distance? What is global geometry/topology](https://reader030.fdocuments.in/reader030/viewer/2022041012/5ebdae07f710ee51347de94d/html5/thumbnails/21.jpg)
Ambient and Intrinsic Regularization
−1 0 1 2
−1
0
1
2
γA = 0.03125 γ
I = 0
SVM
−1 0 1 2
−1
0
1
2
Laplacian SVM
γA = 0.03125 γ
I = 0.01
−1 0 1 2
−1
0
1
2
Laplacian SVM
γA = 0.03125 γ
I = 1
A Geometric Perspective onMachine Learning – p.18
![Page 22: A Geometric Perspective on Machine LearningXocuments, signals, financial time series, sequences = d d,x(x ′) makes sense locally What is good global distance? What is global geometry/topology](https://reader030.fdocuments.in/reader030/viewer/2022041012/5ebdae07f710ee51347de94d/html5/thumbnails/22.jpg)
Experimental comparisonsDataset → g50c Coil20 Uspst mac-win WebKB WebKB WebKB
Algorithm ↓ (link) (page) (page+link)
SVM (full labels) 3.82 0.0 3.35 2.32 6.3 6.5 1.0
RLS (full labels) 3.82 0.0 2.49 2.21 5.6 6.0 2.2
SVM (l labels) 8.32 24.64 23.18 18.87 25.6 22.2 15.6
RLS (l labels) 8.28 25.39 22.90 18.81 28.0 28.4 21.7
Graph-Reg 17.30 6.20 21.30 11.71 22.0 10.7 6.6
TSVM 6.87 26.26 26.46 7.44 14.5 8.6 7.8
Graph-density 8.32 6.43 16.92 10.48 - - -
∇TSVM 5.80 17.56 17.61 5.71 - - -
LDS 5.62 4.86 15.79 5.13 - - -
LapSVM 5.44 3.66 12.67 10.41 18.1 10.5 6.4
LapRLS 5.18 3.36 12.69 10.01 19.2 11.0 6.9
LapSVMjoint - - - - 5.7 6.7 6.4
LapRLSjoint - - - - 5.6 8.0 5.8
A Geometric Perspective onMachine Learning – p.19
![Page 23: A Geometric Perspective on Machine LearningXocuments, signals, financial time series, sequences = d d,x(x ′) makes sense locally What is good global distance? What is global geometry/topology](https://reader030.fdocuments.in/reader030/viewer/2022041012/5ebdae07f710ee51347de94d/html5/thumbnails/23.jpg)
Real World
A Geometric Perspective onMachine Learning – p.20
![Page 24: A Geometric Perspective on Machine LearningXocuments, signals, financial time series, sequences = d d,x(x ′) makes sense locally What is good global distance? What is global geometry/topology](https://reader030.fdocuments.in/reader030/viewer/2022041012/5ebdae07f710ee51347de94d/html5/thumbnails/24.jpg)
Graph and Manifold Laplacian
Fix f : X → R.Fix x ∈ M
[Lnf ](x) = 1ntn(4πtn)d/2
∑
j(f(x) − f(xj))e−
‖x−xj‖2
4tn
Put tn = n−1/(d+2+α), where α > 0
with prob. 1, limn→∞
(Lnf)|x = ∆Mf |x
Belkin (2003), Belkin and Niyogi (2004,2005)
also Lafon (2004), Coifman et al,Hein, Gine and Koltchinski
A Geometric Perspective onMachine Learning – p.21
![Page 25: A Geometric Perspective on Machine LearningXocuments, signals, financial time series, sequences = d d,x(x ′) makes sense locally What is good global distance? What is global geometry/topology](https://reader030.fdocuments.in/reader030/viewer/2022041012/5ebdae07f710ee51347de94d/html5/thumbnails/25.jpg)
Random Graphs and Matrices
Given x1, . . . , xn ∈ M ⊂ RN
Wij =1
t(4πt)d/2e−
‖xi−xj‖|2
t
Eig[D − W ] = Eig[Ltnn ] → Eig[∆M] O(
1
n1/(d+3))
Belkin Niyogi 06,08
Allows us to reconstruct spaces of functions on the manifold.
(Patodi, Dodziuk: triangulated manifolds)
A Geometric Perspective onMachine Learning – p.22
![Page 26: A Geometric Perspective on Machine LearningXocuments, signals, financial time series, sequences = d d,x(x ′) makes sense locally What is good global distance? What is global geometry/topology](https://reader030.fdocuments.in/reader030/viewer/2022041012/5ebdae07f710ee51347de94d/html5/thumbnails/26.jpg)
Manifold + Noise
Flexible, non-parametric, geometric probability model.
−8 −6 −4 −2 0 2 4 6 8−3
−2
−1
0
1
2
3
4
5
6
7
−4 −3 −2 −1 0 1 2 3 4 5 6−2
0
2
4
6
8
10
A Geometric Perspective onMachine Learning – p.23
![Page 27: A Geometric Perspective on Machine LearningXocuments, signals, financial time series, sequences = d d,x(x ′) makes sense locally What is good global distance? What is global geometry/topology](https://reader030.fdocuments.in/reader030/viewer/2022041012/5ebdae07f710ee51347de94d/html5/thumbnails/27.jpg)
Remarks on Noise
1. Arbitrary probability distribution on the manifold:convergence to weighted Laplacian.
2. Noise off the manifold:
µ = µM + µRN
3. Noise off the manifold:
z = x + η (∼ N(0, σ2I))
We havelimt→0
limσ→0
Lt,σf(x) = ∆f(x)
A Geometric Perspective onMachine Learning – p.24
![Page 28: A Geometric Perspective on Machine LearningXocuments, signals, financial time series, sequences = d d,x(x ′) makes sense locally What is good global distance? What is global geometry/topology](https://reader030.fdocuments.in/reader030/viewer/2022041012/5ebdae07f710ee51347de94d/html5/thumbnails/28.jpg)
Local and Global Analysis
X = documents, signals, financial time series, sequences
d(x, x′) makes sense locally
What is good global distance? What is globalgeometry/topology of X?
What is good space of functions on X that is adapted togeometry of X?
A Geometric Perspective onMachine Learning – p.25
![Page 29: A Geometric Perspective on Machine LearningXocuments, signals, financial time series, sequences = d d,x(x ′) makes sense locally What is good global distance? What is global geometry/topology](https://reader030.fdocuments.in/reader030/viewer/2022041012/5ebdae07f710ee51347de94d/html5/thumbnails/29.jpg)
Similarity Metrics
Simt(x, x′) = Kt(x, x′) = αte−d2(x,x′)
t
Ltf(x) =
∫
XKt(x, y)(f(x)−f(y))dρ(y) ≈ 1
n
∑
y∈X
Kt(x, y)(f(x)−f(y))
Choose t small → λi, φi
Choose T large → HT (x, x′) =∑
e−λiT φi(x)φi(x′)
f =∑
i
αiφi;∑
i
α2i g(λi)
A Geometric Perspective onMachine Learning – p.26
![Page 30: A Geometric Perspective on Machine LearningXocuments, signals, financial time series, sequences = d d,x(x ′) makes sense locally What is good global distance? What is global geometry/topology](https://reader030.fdocuments.in/reader030/viewer/2022041012/5ebdae07f710ee51347de94d/html5/thumbnails/30.jpg)
Intrinsic Spectrograms
����
����
p
p −−−> φ (p)
extrinsic
intrinsic
A Geometric Perspective onMachine Learning – p.27
![Page 31: A Geometric Perspective on Machine LearningXocuments, signals, financial time series, sequences = d d,x(x ′) makes sense locally What is good global distance? What is global geometry/topology](https://reader030.fdocuments.in/reader030/viewer/2022041012/5ebdae07f710ee51347de94d/html5/thumbnails/31.jpg)
Speech and Intrinsic Eigenfunctions
0 1 2 3 4−0.1
−0.05
0
0.05
0.1
Time (s)
f1: Background Noise
0 1 2 3 4−0.1
−0.05
0
0.05
0.1
Time (s)
f3: Vowels
0 1 2 3 4−0.15
−0.1
−0.05
0
0.05
0.1
Time (s)
f4: Fricatives
0 1 2 3 4−0.15
−0.1
−0.05
0
0.05
0.1
Time (s)
f10
: Stops, Nasals, and Affricates
A Geometric Perspective onMachine Learning – p.28
![Page 32: A Geometric Perspective on Machine LearningXocuments, signals, financial time series, sequences = d d,x(x ′) makes sense locally What is good global distance? What is global geometry/topology](https://reader030.fdocuments.in/reader030/viewer/2022041012/5ebdae07f710ee51347de94d/html5/thumbnails/32.jpg)
LaplacianFaces: Appearance Manifolds
X. He et al.
A Geometric Perspective onMachine Learning – p.29
![Page 33: A Geometric Perspective on Machine LearningXocuments, signals, financial time series, sequences = d d,x(x ′) makes sense locally What is good global distance? What is global geometry/topology](https://reader030.fdocuments.in/reader030/viewer/2022041012/5ebdae07f710ee51347de94d/html5/thumbnails/33.jpg)
Visualizing Digits
A Geometric Perspective onMachine Learning – p.30
![Page 34: A Geometric Perspective on Machine LearningXocuments, signals, financial time series, sequences = d d,x(x ′) makes sense locally What is good global distance? What is global geometry/topology](https://reader030.fdocuments.in/reader030/viewer/2022041012/5ebdae07f710ee51347de94d/html5/thumbnails/34.jpg)
Vision Example
f : R2 → [0, 1]
F = {f |f(x, y) = v(x − t, y − r)}
A Geometric Perspective onMachine Learning – p.31
![Page 35: A Geometric Perspective on Machine LearningXocuments, signals, financial time series, sequences = d d,x(x ′) makes sense locally What is good global distance? What is global geometry/topology](https://reader030.fdocuments.in/reader030/viewer/2022041012/5ebdae07f710ee51347de94d/html5/thumbnails/35.jpg)
PCA versus Laplacian Eigenmaps
0 20 40
0
10
20
30
40
nz = 75 −5 0 5
x 10−3
−8
−6
−4
−2
0
2
4
6
8x 10
−3
−2 0 2−4
−2
0
2
4
A Geometric Perspective onMachine Learning – p.32
![Page 36: A Geometric Perspective on Machine LearningXocuments, signals, financial time series, sequences = d d,x(x ′) makes sense locally What is good global distance? What is global geometry/topology](https://reader030.fdocuments.in/reader030/viewer/2022041012/5ebdae07f710ee51347de94d/html5/thumbnails/36.jpg)
Computer Vision: Laplacian Eigenmaps
Machine vision: inferring joint angles.Corazza, Andriacchi, Stanford Biomotion Lab, 05, Partiview, Surendran
Isometrically invariant representation.
A Geometric Perspective onMachine Learning – p.33
![Page 37: A Geometric Perspective on Machine LearningXocuments, signals, financial time series, sequences = d d,x(x ′) makes sense locally What is good global distance? What is global geometry/topology](https://reader030.fdocuments.in/reader030/viewer/2022041012/5ebdae07f710ee51347de94d/html5/thumbnails/37.jpg)
Connections and Implications
Clustering and Topologysparse cuts, combinatorial Laplacians, complexes
(Niyogi, Smale, Weinberger, 2006,2008; Narayanan, Belkin, Niyogi, 2006)
Numerical Analysisheat flow based algorithms, sampling, PDEs
(Belkin, Narayanan, Niyogi, 2006; Narayanan and Niyogi, 2008)
Random Matrices and Graphsresults on spectra
Belkin and Niyogi, 2008
Speech, Text, VisionIntrinsic versus Extrinsic
He et al. 2005, Jansen and Niyogi, 2006
A Geometric Perspective onMachine Learning – p.34
![Page 38: A Geometric Perspective on Machine LearningXocuments, signals, financial time series, sequences = d d,x(x ′) makes sense locally What is good global distance? What is global geometry/topology](https://reader030.fdocuments.in/reader030/viewer/2022041012/5ebdae07f710ee51347de94d/html5/thumbnails/38.jpg)
Learning Homology
x1, . . . , xn ∈ M ⊂ RN
Can you learn qualitative features of M?
Can you tell a torus from a sphere?
Can you tell how many connected components?
Can you tell the dimension of M?
(e.g. Carlsson, Zamorodian, Edelsbrunner, Guibas, Oudot, Lieutier, Chazal, Dey, Amenta,Choi,
Cohen-Steiner, de Silva etc.)
A Geometric Perspective onMachine Learning – p.35
![Page 39: A Geometric Perspective on Machine LearningXocuments, signals, financial time series, sequences = d d,x(x ′) makes sense locally What is good global distance? What is global geometry/topology](https://reader030.fdocuments.in/reader030/viewer/2022041012/5ebdae07f710ee51347de94d/html5/thumbnails/39.jpg)
Well Conditioned Submanifolds
τ Μ
Tubular Neighborhood
Condition No. 1τ
Min. distance to medial axis
A Geometric Perspective onMachine Learning – p.36
![Page 40: A Geometric Perspective on Machine LearningXocuments, signals, financial time series, sequences = d d,x(x ′) makes sense locally What is good global distance? What is global geometry/topology](https://reader030.fdocuments.in/reader030/viewer/2022041012/5ebdae07f710ee51347de94d/html5/thumbnails/40.jpg)
Euclidean and Geodesic distance
M ⊂ RN condition ∼ τ
p, q ∈ M where ||p − q||RN = d.
For all d ≤ τ2 ,
dM(p, q) ≤ τ − τ
√
1 − 2d
τ
In fact, Second Fundamental Form Bounded by 1τ
A Geometric Perspective onMachine Learning – p.37
![Page 41: A Geometric Perspective on Machine LearningXocuments, signals, financial time series, sequences = d d,x(x ′) makes sense locally What is good global distance? What is global geometry/topology](https://reader030.fdocuments.in/reader030/viewer/2022041012/5ebdae07f710ee51347de94d/html5/thumbnails/41.jpg)
Homology
x1, . . . , xn ∈ M ⊂ RN
U = ∪ni=1Bǫ(xi)
If ǫ well chosen, then U deformation retracts to M.
Homology of U is constructed using the nerve of U
and agrees with the homology of M.
A Geometric Perspective onMachine Learning – p.38
![Page 42: A Geometric Perspective on Machine LearningXocuments, signals, financial time series, sequences = d d,x(x ′) makes sense locally What is good global distance? What is global geometry/topology](https://reader030.fdocuments.in/reader030/viewer/2022041012/5ebdae07f710ee51347de94d/html5/thumbnails/42.jpg)
Theorem
M ⊂ RN with cond. no. τ
x̄ = {x1, . . . , xn} ∼ uniformly sampled i.i.d.0 < ǫ < τ
2 β = vol(M)(sin−1(ǫ/2τ))dvol(Bǫ/8)
Let U = ∪x∈x̄Bǫ(x)
n > β(log(β) + log(1
δ))
with prob. > 1 − δ,homology of U equals the homology of M
(Niyogi, Smale, Weinberger, 2004)
A Geometric Perspective onMachine Learning – p.39
![Page 43: A Geometric Perspective on Machine LearningXocuments, signals, financial time series, sequences = d d,x(x ′) makes sense locally What is good global distance? What is global geometry/topology](https://reader030.fdocuments.in/reader030/viewer/2022041012/5ebdae07f710ee51347de94d/html5/thumbnails/43.jpg)
A Data-derived complex
x1, . . . , xn ∈ RN
Pick ǫ > 0 and balls Bǫ(xi)
Put j-face for every (i0, . . . , ij) such that
∩jm=0Bǫ(xim) 6= φ
A Geometric Perspective onMachine Learning – p.40
![Page 44: A Geometric Perspective on Machine LearningXocuments, signals, financial time series, sequences = d d,x(x ′) makes sense locally What is good global distance? What is global geometry/topology](https://reader030.fdocuments.in/reader030/viewer/2022041012/5ebdae07f710ee51347de94d/html5/thumbnails/44.jpg)
Chains and the Combinatorial Laplacian
v
wu
e f
g
σ
j chain is a formal sum∑
σ ασσ
Cj is the vector space of j-chains
∂j : Cj → Cj−1
∂∗j : Cj−1 → Cj
∆j = ∂∗j ∂j + ∂j+1∂
∗j+1
A Geometric Perspective onMachine Learning – p.41
![Page 45: A Geometric Perspective on Machine LearningXocuments, signals, financial time series, sequences = d d,x(x ′) makes sense locally What is good global distance? What is global geometry/topology](https://reader030.fdocuments.in/reader030/viewer/2022041012/5ebdae07f710ee51347de94d/html5/thumbnails/45.jpg)
Noise
P on RN
such that
P (x, y) = P (x)P (y|x) where x ∈ M, y ∈ Nx
a ≤ P (x)
P (y|x) = σ2IN−d
A Geometric Perspective onMachine Learning – p.42
![Page 46: A Geometric Perspective on Machine LearningXocuments, signals, financial time series, sequences = d d,x(x ′) makes sense locally What is good global distance? What is global geometry/topology](https://reader030.fdocuments.in/reader030/viewer/2022041012/5ebdae07f710ee51347de94d/html5/thumbnails/46.jpg)
Small Noise
√N − dσ ≤ cτ
[Theorem]There exists an algorithm that recovers homology that ispolynomial in D.
Niyogi, Smale, Weinberger; 2008
A Geometric Perspective onMachine Learning – p.43
![Page 47: A Geometric Perspective on Machine LearningXocuments, signals, financial time series, sequences = d d,x(x ′) makes sense locally What is good global distance? What is global geometry/topology](https://reader030.fdocuments.in/reader030/viewer/2022041012/5ebdae07f710ee51347de94d/html5/thumbnails/47.jpg)
Future Directions
Machine Learning
Scaling UpMulti-scaleGeometry of Natural DataGeometry of Structured Data
Algorithmic Nash embedding
Random Hodge Theory
Partial Differential Equations
Graphics
Algorithms
A Geometric Perspective onMachine Learning – p.44