Aurojit Panda, Colin Scott, Ali Ghodsi, Teemu Koponen, Scott ...
1 Learning from Shadows Dimensionality Reduction and its Application in Artificial Intelligence,...
-
Upload
sheryl-patrick -
Category
Documents
-
view
212 -
download
0
Transcript of 1 Learning from Shadows Dimensionality Reduction and its Application in Artificial Intelligence,...
![Page 1: 1 Learning from Shadows Dimensionality Reduction and its Application in Artificial Intelligence, Signal Processing and Robotics Ali Ghodsi Department of.](https://reader035.fdocuments.in/reader035/viewer/2022070418/56649efa5503460f94c0c737/html5/thumbnails/1.jpg)
1
Learning from ShadowsLearning from Shadows
Dimensionality Reduction and its Dimensionality Reduction and its Application in Artificial Intelligence, Application in Artificial Intelligence,
Signal Processing and RoboticsSignal Processing and Robotics
Ali GhodsiDepartment of Statistics and Actuarial
Science
University of WaterlooOctober 2006
![Page 2: 1 Learning from Shadows Dimensionality Reduction and its Application in Artificial Intelligence, Signal Processing and Robotics Ali Ghodsi Department of.](https://reader035.fdocuments.in/reader035/viewer/2022070418/56649efa5503460f94c0c737/html5/thumbnails/2.jpg)
2
![Page 3: 1 Learning from Shadows Dimensionality Reduction and its Application in Artificial Intelligence, Signal Processing and Robotics Ali Ghodsi Department of.](https://reader035.fdocuments.in/reader035/viewer/2022070418/56649efa5503460f94c0c737/html5/thumbnails/3.jpg)
3
Dimensionality ReductionDimensionality Reduction
![Page 4: 1 Learning from Shadows Dimensionality Reduction and its Application in Artificial Intelligence, Signal Processing and Robotics Ali Ghodsi Department of.](https://reader035.fdocuments.in/reader035/viewer/2022070418/56649efa5503460f94c0c737/html5/thumbnails/4.jpg)
4
Dimensionality ReductionDimensionality Reduction
![Page 5: 1 Learning from Shadows Dimensionality Reduction and its Application in Artificial Intelligence, Signal Processing and Robotics Ali Ghodsi Department of.](https://reader035.fdocuments.in/reader035/viewer/2022070418/56649efa5503460f94c0c737/html5/thumbnails/5.jpg)
5
Manifold and Hidden VariablesManifold and Hidden Variables
![Page 6: 1 Learning from Shadows Dimensionality Reduction and its Application in Artificial Intelligence, Signal Processing and Robotics Ali Ghodsi Department of.](https://reader035.fdocuments.in/reader035/viewer/2022070418/56649efa5503460f94c0c737/html5/thumbnails/6.jpg)
6
Data RepresentationData Representation
![Page 7: 1 Learning from Shadows Dimensionality Reduction and its Application in Artificial Intelligence, Signal Processing and Robotics Ali Ghodsi Department of.](https://reader035.fdocuments.in/reader035/viewer/2022070418/56649efa5503460f94c0c737/html5/thumbnails/7.jpg)
7
Data RepresentationData Representation
![Page 8: 1 Learning from Shadows Dimensionality Reduction and its Application in Artificial Intelligence, Signal Processing and Robotics Ali Ghodsi Department of.](https://reader035.fdocuments.in/reader035/viewer/2022070418/56649efa5503460f94c0c737/html5/thumbnails/8.jpg)
8
11 11 11 11 11
11 00 11 00 11
11 11 11 11 11
11 0.50.5 0.50.5 0.50.5 11
11 11 11 11 11
Data RepresentationData Representation
![Page 9: 1 Learning from Shadows Dimensionality Reduction and its Application in Artificial Intelligence, Signal Processing and Robotics Ali Ghodsi Department of.](https://reader035.fdocuments.in/reader035/viewer/2022070418/56649efa5503460f94c0c737/html5/thumbnails/9.jpg)
9
![Page 10: 1 Learning from Shadows Dimensionality Reduction and its Application in Artificial Intelligence, Signal Processing and Robotics Ali Ghodsi Department of.](https://reader035.fdocuments.in/reader035/viewer/2022070418/56649efa5503460f94c0c737/html5/thumbnails/10.jpg)
10
644 by 103
644 by 2
2 by 103
23 by 28 23 by 28
-2.19
-0.02
-3.19
1.02
2 by 12 by 1
![Page 11: 1 Learning from Shadows Dimensionality Reduction and its Application in Artificial Intelligence, Signal Processing and Robotics Ali Ghodsi Department of.](https://reader035.fdocuments.in/reader035/viewer/2022070418/56649efa5503460f94c0c737/html5/thumbnails/11.jpg)
11
![Page 12: 1 Learning from Shadows Dimensionality Reduction and its Application in Artificial Intelligence, Signal Processing and Robotics Ali Ghodsi Department of.](https://reader035.fdocuments.in/reader035/viewer/2022070418/56649efa5503460f94c0c737/html5/thumbnails/12.jpg)
12
![Page 13: 1 Learning from Shadows Dimensionality Reduction and its Application in Artificial Intelligence, Signal Processing and Robotics Ali Ghodsi Department of.](https://reader035.fdocuments.in/reader035/viewer/2022070418/56649efa5503460f94c0c737/html5/thumbnails/13.jpg)
13
![Page 14: 1 Learning from Shadows Dimensionality Reduction and its Application in Artificial Intelligence, Signal Processing and Robotics Ali Ghodsi Department of.](https://reader035.fdocuments.in/reader035/viewer/2022070418/56649efa5503460f94c0c737/html5/thumbnails/14.jpg)
14
![Page 15: 1 Learning from Shadows Dimensionality Reduction and its Application in Artificial Intelligence, Signal Processing and Robotics Ali Ghodsi Department of.](https://reader035.fdocuments.in/reader035/viewer/2022070418/56649efa5503460f94c0c737/html5/thumbnails/15.jpg)
15
![Page 16: 1 Learning from Shadows Dimensionality Reduction and its Application in Artificial Intelligence, Signal Processing and Robotics Ali Ghodsi Department of.](https://reader035.fdocuments.in/reader035/viewer/2022070418/56649efa5503460f94c0c737/html5/thumbnails/16.jpg)
16Hastie, Tibshirani, Friedman 2001
![Page 17: 1 Learning from Shadows Dimensionality Reduction and its Application in Artificial Intelligence, Signal Processing and Robotics Ali Ghodsi Department of.](https://reader035.fdocuments.in/reader035/viewer/2022070418/56649efa5503460f94c0c737/html5/thumbnails/17.jpg)
17
The Big PictureThe Big Picture
![Page 18: 1 Learning from Shadows Dimensionality Reduction and its Application in Artificial Intelligence, Signal Processing and Robotics Ali Ghodsi Department of.](https://reader035.fdocuments.in/reader035/viewer/2022070418/56649efa5503460f94c0c737/html5/thumbnails/18.jpg)
18
Uses of Dimensionality Uses of Dimensionality ReductionReduction
(Manifold Learning)(Manifold Learning)
![Page 19: 1 Learning from Shadows Dimensionality Reduction and its Application in Artificial Intelligence, Signal Processing and Robotics Ali Ghodsi Department of.](https://reader035.fdocuments.in/reader035/viewer/2022070418/56649efa5503460f94c0c737/html5/thumbnails/19.jpg)
19
DenoisingDenoising
Mika et. al. 1999
Zhu and Ghodsi 2005
![Page 20: 1 Learning from Shadows Dimensionality Reduction and its Application in Artificial Intelligence, Signal Processing and Robotics Ali Ghodsi Department of.](https://reader035.fdocuments.in/reader035/viewer/2022070418/56649efa5503460f94c0c737/html5/thumbnails/20.jpg)
20
Tenenbaum, V de Silva, Langford 2001
![Page 21: 1 Learning from Shadows Dimensionality Reduction and its Application in Artificial Intelligence, Signal Processing and Robotics Ali Ghodsi Department of.](https://reader035.fdocuments.in/reader035/viewer/2022070418/56649efa5503460f94c0c737/html5/thumbnails/21.jpg)
21
Roweis and. Saul 2000
![Page 22: 1 Learning from Shadows Dimensionality Reduction and its Application in Artificial Intelligence, Signal Processing and Robotics Ali Ghodsi Department of.](https://reader035.fdocuments.in/reader035/viewer/2022070418/56649efa5503460f94c0c737/html5/thumbnails/22.jpg)
22
Arranging words: Each word was initially represented by a high-dimensional vector that counted the number of times it appeared in different encyclopedia articles. Words with similar contexts are collocated
Roweis and Saul 2000
![Page 23: 1 Learning from Shadows Dimensionality Reduction and its Application in Artificial Intelligence, Signal Processing and Robotics Ali Ghodsi Department of.](https://reader035.fdocuments.in/reader035/viewer/2022070418/56649efa5503460f94c0c737/html5/thumbnails/23.jpg)
23
Hinton and Roweis 2002
![Page 24: 1 Learning from Shadows Dimensionality Reduction and its Application in Artificial Intelligence, Signal Processing and Robotics Ali Ghodsi Department of.](https://reader035.fdocuments.in/reader035/viewer/2022070418/56649efa5503460f94c0c737/html5/thumbnails/24.jpg)
24
Embedding of Sparse Music Embedding of Sparse Music Similarity GraphSimilarity Graph
Platt, 2004
![Page 25: 1 Learning from Shadows Dimensionality Reduction and its Application in Artificial Intelligence, Signal Processing and Robotics Ali Ghodsi Department of.](https://reader035.fdocuments.in/reader035/viewer/2022070418/56649efa5503460f94c0c737/html5/thumbnails/25.jpg)
25
Pattern Recognition Pattern Recognition
Ghodsi, Huang, Schuurmans 2004
![Page 26: 1 Learning from Shadows Dimensionality Reduction and its Application in Artificial Intelligence, Signal Processing and Robotics Ali Ghodsi Department of.](https://reader035.fdocuments.in/reader035/viewer/2022070418/56649efa5503460f94c0c737/html5/thumbnails/26.jpg)
26
Pattern RecognitionPattern Recognition
![Page 27: 1 Learning from Shadows Dimensionality Reduction and its Application in Artificial Intelligence, Signal Processing and Robotics Ali Ghodsi Department of.](https://reader035.fdocuments.in/reader035/viewer/2022070418/56649efa5503460f94c0c737/html5/thumbnails/27.jpg)
27
ClusteringClustering
![Page 28: 1 Learning from Shadows Dimensionality Reduction and its Application in Artificial Intelligence, Signal Processing and Robotics Ali Ghodsi Department of.](https://reader035.fdocuments.in/reader035/viewer/2022070418/56649efa5503460f94c0c737/html5/thumbnails/28.jpg)
28
Glasses vs. No GlassesGlasses vs. No Glasses
![Page 29: 1 Learning from Shadows Dimensionality Reduction and its Application in Artificial Intelligence, Signal Processing and Robotics Ali Ghodsi Department of.](https://reader035.fdocuments.in/reader035/viewer/2022070418/56649efa5503460f94c0c737/html5/thumbnails/29.jpg)
29
Beard vs. No BeardBeard vs. No Beard
![Page 30: 1 Learning from Shadows Dimensionality Reduction and its Application in Artificial Intelligence, Signal Processing and Robotics Ali Ghodsi Department of.](https://reader035.fdocuments.in/reader035/viewer/2022070418/56649efa5503460f94c0c737/html5/thumbnails/30.jpg)
30
Beard DistinctionBeard Distinction
Ghodsi , Wilkinson, Southey 2006
![Page 31: 1 Learning from Shadows Dimensionality Reduction and its Application in Artificial Intelligence, Signal Processing and Robotics Ali Ghodsi Department of.](https://reader035.fdocuments.in/reader035/viewer/2022070418/56649efa5503460f94c0c737/html5/thumbnails/31.jpg)
31
Glasses DistinctionGlasses Distinction
![Page 32: 1 Learning from Shadows Dimensionality Reduction and its Application in Artificial Intelligence, Signal Processing and Robotics Ali Ghodsi Department of.](https://reader035.fdocuments.in/reader035/viewer/2022070418/56649efa5503460f94c0c737/html5/thumbnails/32.jpg)
32
Multiple-Attribute MetricMultiple-Attribute Metric
![Page 33: 1 Learning from Shadows Dimensionality Reduction and its Application in Artificial Intelligence, Signal Processing and Robotics Ali Ghodsi Department of.](https://reader035.fdocuments.in/reader035/viewer/2022070418/56649efa5503460f94c0c737/html5/thumbnails/33.jpg)
33
Reinforcement LearningReinforcement Learning
Mahadevan and Maggioini, 2005
![Page 34: 1 Learning from Shadows Dimensionality Reduction and its Application in Artificial Intelligence, Signal Processing and Robotics Ali Ghodsi Department of.](https://reader035.fdocuments.in/reader035/viewer/2022070418/56649efa5503460f94c0c737/html5/thumbnails/34.jpg)
34
Semi-supervised LearningSemi-supervised Learning
Use graph-based discretization of manifold to infer missing labels.
Build classifiers from bottom eigenvectors of graph Laplacian.
Belkin & Niyogi, 2004; Zien et al, Eds., 2005
![Page 35: 1 Learning from Shadows Dimensionality Reduction and its Application in Artificial Intelligence, Signal Processing and Robotics Ali Ghodsi Department of.](https://reader035.fdocuments.in/reader035/viewer/2022070418/56649efa5503460f94c0c737/html5/thumbnails/35.jpg)
35
Learning CorrespondencesLearning Correspondences
How can we learn manifold structure that is shared across multiple data sets?
Ham et al, 2003, 2005
![Page 36: 1 Learning from Shadows Dimensionality Reduction and its Application in Artificial Intelligence, Signal Processing and Robotics Ali Ghodsi Department of.](https://reader035.fdocuments.in/reader035/viewer/2022070418/56649efa5503460f94c0c737/html5/thumbnails/36.jpg)
36
Mapping and Robot LocalizationMapping and Robot Localization
Bowling, Ghodsi, Wilkinson 2005
Ham, Lin, D.D. 2005
![Page 37: 1 Learning from Shadows Dimensionality Reduction and its Application in Artificial Intelligence, Signal Processing and Robotics Ali Ghodsi Department of.](https://reader035.fdocuments.in/reader035/viewer/2022070418/56649efa5503460f94c0c737/html5/thumbnails/37.jpg)
37
Action Respecting Action Respecting EmbeddingEmbedding
Joint Work with
Michael Bowlingand
Dana Wilkinson
![Page 38: 1 Learning from Shadows Dimensionality Reduction and its Application in Artificial Intelligence, Signal Processing and Robotics Ali Ghodsi Department of.](https://reader035.fdocuments.in/reader035/viewer/2022070418/56649efa5503460f94c0c737/html5/thumbnails/38.jpg)
38
Modelling Temporal Data and Modelling Temporal Data and ActionsActions
![Page 39: 1 Learning from Shadows Dimensionality Reduction and its Application in Artificial Intelligence, Signal Processing and Robotics Ali Ghodsi Department of.](https://reader035.fdocuments.in/reader035/viewer/2022070418/56649efa5503460f94c0c737/html5/thumbnails/39.jpg)
39
OutlineOutline
• Background– PCA– Kernel PCA
• Action Respecting Embedding (ARE)– Prediction and Planning– Probabilistic Actions
• Future Work
![Page 40: 1 Learning from Shadows Dimensionality Reduction and its Application in Artificial Intelligence, Signal Processing and Robotics Ali Ghodsi Department of.](https://reader035.fdocuments.in/reader035/viewer/2022070418/56649efa5503460f94c0c737/html5/thumbnails/40.jpg)
40
Principal Component Analysis Principal Component Analysis (PCA)(PCA)
![Page 41: 1 Learning from Shadows Dimensionality Reduction and its Application in Artificial Intelligence, Signal Processing and Robotics Ali Ghodsi Department of.](https://reader035.fdocuments.in/reader035/viewer/2022070418/56649efa5503460f94c0c737/html5/thumbnails/41.jpg)
41
Principal Component Analysis Principal Component Analysis (PCA)(PCA)
![Page 42: 1 Learning from Shadows Dimensionality Reduction and its Application in Artificial Intelligence, Signal Processing and Robotics Ali Ghodsi Department of.](https://reader035.fdocuments.in/reader035/viewer/2022070418/56649efa5503460f94c0c737/html5/thumbnails/42.jpg)
42
Kernel MethodsKernel Methods
![Page 43: 1 Learning from Shadows Dimensionality Reduction and its Application in Artificial Intelligence, Signal Processing and Robotics Ali Ghodsi Department of.](https://reader035.fdocuments.in/reader035/viewer/2022070418/56649efa5503460f94c0c737/html5/thumbnails/43.jpg)
43
Kernel TrickKernel Trick
![Page 44: 1 Learning from Shadows Dimensionality Reduction and its Application in Artificial Intelligence, Signal Processing and Robotics Ali Ghodsi Department of.](https://reader035.fdocuments.in/reader035/viewer/2022070418/56649efa5503460f94c0c737/html5/thumbnails/44.jpg)
44
Observed, Feature and Embedded Observed, Feature and Embedded SpacesSpaces
![Page 45: 1 Learning from Shadows Dimensionality Reduction and its Application in Artificial Intelligence, Signal Processing and Robotics Ali Ghodsi Department of.](https://reader035.fdocuments.in/reader035/viewer/2022070418/56649efa5503460f94c0c737/html5/thumbnails/45.jpg)
45
Kernel PCA Kernel PCA
![Page 46: 1 Learning from Shadows Dimensionality Reduction and its Application in Artificial Intelligence, Signal Processing and Robotics Ali Ghodsi Department of.](https://reader035.fdocuments.in/reader035/viewer/2022070418/56649efa5503460f94c0c737/html5/thumbnails/46.jpg)
46
ProblemProblem
![Page 47: 1 Learning from Shadows Dimensionality Reduction and its Application in Artificial Intelligence, Signal Processing and Robotics Ali Ghodsi Department of.](https://reader035.fdocuments.in/reader035/viewer/2022070418/56649efa5503460f94c0c737/html5/thumbnails/47.jpg)
47
IdeaIdea
![Page 48: 1 Learning from Shadows Dimensionality Reduction and its Application in Artificial Intelligence, Signal Processing and Robotics Ali Ghodsi Department of.](https://reader035.fdocuments.in/reader035/viewer/2022070418/56649efa5503460f94c0c737/html5/thumbnails/48.jpg)
48
Action Respecting Embedding Action Respecting Embedding (ARE)(ARE)
![Page 49: 1 Learning from Shadows Dimensionality Reduction and its Application in Artificial Intelligence, Signal Processing and Robotics Ali Ghodsi Department of.](https://reader035.fdocuments.in/reader035/viewer/2022070418/56649efa5503460f94c0c737/html5/thumbnails/49.jpg)
49
Action Respecting ConstraintAction Respecting Constraint
![Page 50: 1 Learning from Shadows Dimensionality Reduction and its Application in Artificial Intelligence, Signal Processing and Robotics Ali Ghodsi Department of.](https://reader035.fdocuments.in/reader035/viewer/2022070418/56649efa5503460f94c0c737/html5/thumbnails/50.jpg)
50
Preserve distances between each point and its k nearest neighbors.
Local Distances ConstraintLocal Distances Constraint
![Page 51: 1 Learning from Shadows Dimensionality Reduction and its Application in Artificial Intelligence, Signal Processing and Robotics Ali Ghodsi Department of.](https://reader035.fdocuments.in/reader035/viewer/2022070418/56649efa5503460f94c0c737/html5/thumbnails/51.jpg)
51
Preserve local distances
Local Distances ConstraintLocal Distances Constraint
![Page 52: 1 Learning from Shadows Dimensionality Reduction and its Application in Artificial Intelligence, Signal Processing and Robotics Ali Ghodsi Department of.](https://reader035.fdocuments.in/reader035/viewer/2022070418/56649efa5503460f94c0c737/html5/thumbnails/52.jpg)
52
Semidefinite ProgrammingSemidefinite Programming
![Page 53: 1 Learning from Shadows Dimensionality Reduction and its Application in Artificial Intelligence, Signal Processing and Robotics Ali Ghodsi Department of.](https://reader035.fdocuments.in/reader035/viewer/2022070418/56649efa5503460f94c0c737/html5/thumbnails/53.jpg)
53
ExperimentExperiment
![Page 54: 1 Learning from Shadows Dimensionality Reduction and its Application in Artificial Intelligence, Signal Processing and Robotics Ali Ghodsi Department of.](https://reader035.fdocuments.in/reader035/viewer/2022070418/56649efa5503460f94c0c737/html5/thumbnails/54.jpg)
54
Experiment 1Experiment 1
![Page 55: 1 Learning from Shadows Dimensionality Reduction and its Application in Artificial Intelligence, Signal Processing and Robotics Ali Ghodsi Department of.](https://reader035.fdocuments.in/reader035/viewer/2022070418/56649efa5503460f94c0c737/html5/thumbnails/55.jpg)
55
Experiment 2Experiment 2
![Page 56: 1 Learning from Shadows Dimensionality Reduction and its Application in Artificial Intelligence, Signal Processing and Robotics Ali Ghodsi Department of.](https://reader035.fdocuments.in/reader035/viewer/2022070418/56649efa5503460f94c0c737/html5/thumbnails/56.jpg)
56
Experiment 3Experiment 3
![Page 57: 1 Learning from Shadows Dimensionality Reduction and its Application in Artificial Intelligence, Signal Processing and Robotics Ali Ghodsi Department of.](https://reader035.fdocuments.in/reader035/viewer/2022070418/56649efa5503460f94c0c737/html5/thumbnails/57.jpg)
57
Experiment 4Experiment 4
![Page 58: 1 Learning from Shadows Dimensionality Reduction and its Application in Artificial Intelligence, Signal Processing and Robotics Ali Ghodsi Department of.](https://reader035.fdocuments.in/reader035/viewer/2022070418/56649efa5503460f94c0c737/html5/thumbnails/58.jpg)
58
Experiment 5Experiment 5
![Page 59: 1 Learning from Shadows Dimensionality Reduction and its Application in Artificial Intelligence, Signal Processing and Robotics Ali Ghodsi Department of.](https://reader035.fdocuments.in/reader035/viewer/2022070418/56649efa5503460f94c0c737/html5/thumbnails/59.jpg)
59
PlanningPlanning
![Page 60: 1 Learning from Shadows Dimensionality Reduction and its Application in Artificial Intelligence, Signal Processing and Robotics Ali Ghodsi Department of.](https://reader035.fdocuments.in/reader035/viewer/2022070418/56649efa5503460f94c0c737/html5/thumbnails/60.jpg)
60
PlanningPlanning
![Page 61: 1 Learning from Shadows Dimensionality Reduction and its Application in Artificial Intelligence, Signal Processing and Robotics Ali Ghodsi Department of.](https://reader035.fdocuments.in/reader035/viewer/2022070418/56649efa5503460f94c0c737/html5/thumbnails/61.jpg)
61
PlanningPlanning
![Page 62: 1 Learning from Shadows Dimensionality Reduction and its Application in Artificial Intelligence, Signal Processing and Robotics Ali Ghodsi Department of.](https://reader035.fdocuments.in/reader035/viewer/2022070418/56649efa5503460f94c0c737/html5/thumbnails/62.jpg)
62
ExperimentExperiment
![Page 63: 1 Learning from Shadows Dimensionality Reduction and its Application in Artificial Intelligence, Signal Processing and Robotics Ali Ghodsi Department of.](https://reader035.fdocuments.in/reader035/viewer/2022070418/56649efa5503460f94c0c737/html5/thumbnails/63.jpg)
63
Probabilistic Actions
![Page 64: 1 Learning from Shadows Dimensionality Reduction and its Application in Artificial Intelligence, Signal Processing and Robotics Ali Ghodsi Department of.](https://reader035.fdocuments.in/reader035/viewer/2022070418/56649efa5503460f94c0c737/html5/thumbnails/64.jpg)
64
Future workFuture work
![Page 65: 1 Learning from Shadows Dimensionality Reduction and its Application in Artificial Intelligence, Signal Processing and Robotics Ali Ghodsi Department of.](https://reader035.fdocuments.in/reader035/viewer/2022070418/56649efa5503460f94c0c737/html5/thumbnails/65.jpg)
65
Related PapersRelated Papers