Survey and Review
1
Copyright © by SIAM. Unauthorized reproduction of this article is prohibited. These are exciting times in the world of data analysis. An ever increasing amount of data, from commercial transactions to webpage content, from images to sensor data, is being collected, and there is a real need to extract useful information from the sea of data. Many of these types of data can be represented as really large matrices, and one way to gain insight into these data sets is to apply matrix decompositions, such as the SVD, to them. The progress in computer hardware cannot be expected to keep pace with the needs of data analysis. The way to meet these needs will no doubt involve a combina- tion of good hardware and good computational methods. The fact that the matrices are huge suggests that we use methods that do not require dealing with the entire matrix at once. These methods incorporate the knowledge of the outcome of the action of the matrix on a vector. Many iterative methods, based on just this principle, have been developed to deal with large matrices. Though we are conditioned to think of matrix computations as a deterministic process, there have been ideas which exploit randomized strategy. An example is the stochastic trace estimator introduced by Hutchinson in 1990 in the context of ill-posed problems. Much progress in randomized algorithms has taken place since then. The present review article, “Finding Structure with Randomness: Probabilistic Al- gorithms for Constructing Approximate Matrix Decompositions” by N. Halko, P. G. Martinsson, and J. A. Tropp, introduces a framework for stochastic approaches to matrix computations. This well-written paper starts with a survey of randomized al- gorithms in linear algebra and proceeds to develop a game-changing, original approach. The approach leads to several efficient algorithms with surprisingly good properties. The analysis provided for the method is a tour-de-force in which deep ideas from probability theory are used to establish the expected behavior of the algorithms. The paper is self-contained and very accessible. At one level, readers can quickly glean the main ideas at play. At the next level, readers can go a little further and start using the described approaches in actual computation. At the final level, readers will discover, after digging deep into the theory, just why the methods work, and perhaps get ideas on how to approach other computational problems. This editor believes that the ideas presented in this work are central to computational linear algebra, and it is not too early to introduce some of them into the classroom. Fadil Santosa Section Editor [email protected] 215 Downloaded 11/19/14 to 128.235.8.170. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php
Transcript of Survey and Review
Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.
These are exciting times in the world of data analysis. An ever increasing amountof data, from commercial transactions to webpage content, from images to sensordata, is being collected, and there is a real need to extract useful information from thesea of data. Many of these types of data can be represented as really large matrices,and one way to gain insight into these data sets is to apply matrix decompositions, suchas the SVD, to them.
The progress in computer hardware cannot be expected to keep pace with theneeds of data analysis. The way to meet these needs will no doubt involve a combina-tion of good hardware and good computational methods. The fact that the matricesare huge suggests that we use methods that do not require dealing with the entirematrix at once. These methods incorporate the knowledge of the outcome of theaction of the matrix on a vector. Many iterative methods, based on just this principle,have been developed to deal with large matrices.
Though we are conditioned to think of matrix computations as a deterministicprocess, there have been ideas which exploit randomized strategy. An example is thestochastic trace estimator introduced by Hutchinson in 1990 in the context of ill-posedproblems. Much progress in randomized algorithms has taken place since then.
The present review article, “Finding Structure with Randomness: Probabilistic Al-gorithms for Constructing Approximate Matrix Decompositions” by N. Halko, P. G.Martinsson, and J. A. Tropp, introduces a framework for stochastic approaches tomatrix computations. This well-written paper starts with a survey of randomized al-gorithms in linear algebra and proceeds to develop a game-changing, original approach.The approach leads to several efficient algorithms with surprisingly good properties.The analysis provided for the method is a tour-de-force in which deep ideas fromprobability theory are used to establish the expected behavior of the algorithms.
The paper is self-contained and very accessible. At one level, readers can quicklyglean the main ideas at play. At the next level, readers can go a little further and startusing the described approaches in actual computation. At the final level, readers willdiscover, after digging deep into the theory, just why the methods work, and perhapsget ideas on how to approach other computational problems. This editor believes thatthe ideas presented in this work are central to computational linear algebra, and it isnot too early to introduce some of them into the classroom.
Fadil SantosaSection Editor
215
Dow
nloa
ded
11/1
9/14
to 1
28.2
35.8
.170
. Red
istr
ibut
ion
subj
ect t
o SI
AM
lice
nse
or c
opyr
ight
; see
http
://w
ww
.sia
m.o
rg/jo
urna
ls/o
jsa.
php
