Fast N-Body Learning Nando de Freitas University of British Columbia.
-
date post
19-Dec-2015 -
Category
Documents
-
view
214 -
download
0
Transcript of Fast N-Body Learning Nando de Freitas University of British Columbia.
Historical Perspective Historical Perspective
• Non-iterative or “direct” methods for eigenvalue problems and linear systems of equations require O(N3) operations.
• Let's look at the history of what has been regarded as large N:
1950: N=20 1965: N=200 1980: N=2000 1995: N=20000
• So over the course of 45 years N has increased by a factor of103. However, the speed of computers has increased by a factor of109. From this the O(N3) bottleneck is evident.
If only we could reduce the cost to O(N) – sigh!
Obvious applications of N-body Learning Obvious applications of N-body Learning
• Exact and approximate message propagation.
• Markov chain Monte Carlo
• Gaussian processes, Wishart processes and Laplace processes.
• Spectral learning: eigenmaps, SNE, NCUTS, ranking on manifolds, … (even if using Nystrom)
• Reinforcement learning.
• The E step.
• Kernel-(fill in your favourite name).
• Rao-Blackwellised Monte Carlo.
• Nearest neighbour methods.
• Some types of boosting.
• Computer graphics.
• EM, fluid dynamics, gravitation, quantum systems.
• … and much more !
Obvious applications of N-body Learning Obvious applications of N-body Learning
Illustrative Example Illustrative Example
Energy function using the graph Laplacian:
Easy, but … a big linear system:
Naïve iterative solution:
Illustrative Example Illustrative Example
We have solved a Gaussian process (where the covariance is the inverse graph Laplacian in O(N).
In this workshop In this workshop
•You’ll encounter tutorials on fast methods from the people who’ve been developing them.
• You’re likely to see encounter people arguing over error bounds, implementation strategies, applications and many more things.
• You’ll see statistics, learning, data structures and numerical computation come together.
• You’ll dream of the powder up on the hill.