Compressed Sensing… M.E. Davieshomepages.inf.ed.ac.uk › ckiw › rpml › SparseRepandCS.pdf ·...
Transcript of Compressed Sensing… M.E. Davieshomepages.inf.ed.ac.uk › ckiw › rpml › SparseRepandCS.pdf ·...
![Page 1: Compressed Sensing… M.E. Davieshomepages.inf.ed.ac.uk › ckiw › rpml › SparseRepandCS.pdf · Potential applications… Compressed Sensing provides a new way of thinking about](https://reader035.fdocuments.in/reader035/viewer/2022070804/5f03619b7e708231d408ef0c/html5/thumbnails/1.jpg)
Page 1 of 14
Compressed Sensing… M.E. Davies
Sparse representations and compressed sensing
Mike DaviesInstitute for Digital Communications (IDCOM) &
Joint Research Institute for Signal and Image Processing University of Edinburgh
with thanks to
Thomas Blumensath, Gabriel Rilling
![Page 2: Compressed Sensing… M.E. Davieshomepages.inf.ed.ac.uk › ckiw › rpml › SparseRepandCS.pdf · Potential applications… Compressed Sensing provides a new way of thinking about](https://reader035.fdocuments.in/reader035/viewer/2022070804/5f03619b7e708231d408ef0c/html5/thumbnails/2.jpg)
Page 2 of 14
Compressed Sensing… M.E. Davies
Sparsity: formal definition
A vector x is K-sparse, if only K of its elements are non-zero.
In the real world “exact” sparseness is uncommon, however, many signals are “approximately” K-sparse. That is, there is a K-sparse vector xK, such that the error k x − x Kk2 is small.
![Page 3: Compressed Sensing… M.E. Davieshomepages.inf.ed.ac.uk › ckiw › rpml › SparseRepandCS.pdf · Potential applications… Compressed Sensing provides a new way of thinking about](https://reader035.fdocuments.in/reader035/viewer/2022070804/5f03619b7e708231d408ef0c/html5/thumbnails/3.jpg)
Page 3 of 14
Compressed Sensing… M.E. Davies
Why Sparsity?Why does sparsity make for a good transform?
50 100 150 200 250
50
100
150
200
250
“TOM” image Wavelet Domain
Good representations are efficient - Sparse!
![Page 4: Compressed Sensing… M.E. Davieshomepages.inf.ed.ac.uk › ckiw › rpml › SparseRepandCS.pdf · Potential applications… Compressed Sensing provides a new way of thinking about](https://reader035.fdocuments.in/reader035/viewer/2022070804/5f03619b7e708231d408ef0c/html5/thumbnails/4.jpg)
Page 4 of 14
Compressed Sensing… M.E. Davies
Efficient transform domain representations imply that our signals of interest live in a very small set.
Signals of interest
Sparse signal modelL2 ball (not sparse)
![Page 5: Compressed Sensing… M.E. Davieshomepages.inf.ed.ac.uk › ckiw › rpml › SparseRepandCS.pdf · Potential applications… Compressed Sensing provides a new way of thinking about](https://reader035.fdocuments.in/reader035/viewer/2022070804/5f03619b7e708231d408ef0c/html5/thumbnails/5.jpg)
Page 5 of 14
Compressed Sensing… M.E. Davies
Sparse signal models can be used to help solve various ill-posed linear inverse problems:
Sparsity and ill-posed inverse problems
Missing Data RecoveryImage De-blurring
![Page 6: Compressed Sensing… M.E. Davieshomepages.inf.ed.ac.uk › ckiw › rpml › SparseRepandCS.pdf · Potential applications… Compressed Sensing provides a new way of thinking about](https://reader035.fdocuments.in/reader035/viewer/2022070804/5f03619b7e708231d408ef0c/html5/thumbnails/6.jpg)
Page 6 of 14
Compressed Sensing… M.E. Davies
Sparse Representations and Generalized Sampling:compressed sensing
![Page 7: Compressed Sensing… M.E. Davieshomepages.inf.ed.ac.uk › ckiw › rpml › SparseRepandCS.pdf · Potential applications… Compressed Sensing provides a new way of thinking about](https://reader035.fdocuments.in/reader035/viewer/2022070804/5f03619b7e708231d408ef0c/html5/thumbnails/7.jpg)
Page 7 of 14
Compressed Sensing… M.E. Davies
Compressed sensingTraditionally when compressing a signal we take lots of samples move to a transform domain and then throw most of the coefficients away!
Why can’t we just sample signals at the “Information Rate”?
E. Candès, J. Romberg, and T. Tao, “Robust Uncertainty principles: Exact signal reconstruction from highly incomplete frequency information,”IEEE Trans. Information Theory, 2006
D. Donoho, “Compressed sensing,” IEEE Trans. Information Theory, 2006
This is the philosophy of Compressed Sensing…
![Page 8: Compressed Sensing… M.E. Davieshomepages.inf.ed.ac.uk › ckiw › rpml › SparseRepandCS.pdf · Potential applications… Compressed Sensing provides a new way of thinking about](https://reader035.fdocuments.in/reader035/viewer/2022070804/5f03619b7e708231d408ef0c/html5/thumbnails/8.jpg)
Page 8 of 14
Compressed Sensing… M.E. Davies
Signal space ~ RN
Set of signals of interest
Observation space ~ RM
M<<N
Linear projection (observation)
Nonlinear Approximation (reconstruction)
Compressed sensing
Compressed Sensing uses nonlinear reconstruction to invert the linear projection operator
![Page 9: Compressed Sensing… M.E. Davieshomepages.inf.ed.ac.uk › ckiw › rpml › SparseRepandCS.pdf · Potential applications… Compressed Sensing provides a new way of thinking about](https://reader035.fdocuments.in/reader035/viewer/2022070804/5f03619b7e708231d408ef0c/html5/thumbnails/9.jpg)
Page 9 of 14
Compressed Sensing… M.E. Davies
Compressed sensingCompressed Sensing Challenges:
• Question 1: In which domain is the signal sparse (if at all)?Many natural signals are sparse in some time-frequency or space scale representation
• Question 2: How should we take good measurements?Current theory suggests that a random element in the sampling process is important
• Question 3: How many measurements do we need?Compressed sensing theory provides strong bounds on this as a function of the sparsity
• Question 4: How can we reconstruct the original signal from the measurements?
Compressed sensing concentrates of algorithmic solutions with provable performance and provably good complexity
![Page 10: Compressed Sensing… M.E. Davieshomepages.inf.ed.ac.uk › ckiw › rpml › SparseRepandCS.pdf · Potential applications… Compressed Sensing provides a new way of thinking about](https://reader035.fdocuments.in/reader035/viewer/2022070804/5f03619b7e708231d408ef0c/html5/thumbnails/10.jpg)
Page 10 of 14
Compressed Sensing… M.E. Davies
Compressed sensing principle
sparse “Tom”
4
Invert transform
X =
2
Observed data
roughly equivalent
Wavelet image
Nonlinear Reconstruction
3
50 100 150 200 250
50
100
150
200
250
original “Tom”
Sparsifyingtransform
Wavelet image
50 100 150 200 250
50
100
150
200
250
1
Note that 1 is a redundant representation of 2
![Page 11: Compressed Sensing… M.E. Davieshomepages.inf.ed.ac.uk › ckiw › rpml › SparseRepandCS.pdf · Potential applications… Compressed Sensing provides a new way of thinking about](https://reader035.fdocuments.in/reader035/viewer/2022070804/5f03619b7e708231d408ef0c/html5/thumbnails/11.jpg)
Page 11 of 14
Compressed Sensing… M.E. Davies
MRI acquisition
Logan-Shepp phantom We sample in this domain
Spatial Fourier Transform
Haar Wavelet Transform
Logan-Shepp phantom Sparse in this domain
Haar Wavelet Transform
Logan-Shepp phantom
Sub-sampled Fourier Transform
≈ 7 x down sampled (no longer invertible)
…but we wish to sample here
Compressed Sensing ideas can be applied to reduced sampling in Magnetic Resonance Imaging:
• MRI samples lines of spatial frequency
• Each line takes time & heats up the patient!
The Logan-Shepp phantom image illustrates this:
![Page 12: Compressed Sensing… M.E. Davieshomepages.inf.ed.ac.uk › ckiw › rpml › SparseRepandCS.pdf · Potential applications… Compressed Sensing provides a new way of thinking about](https://reader035.fdocuments.in/reader035/viewer/2022070804/5f03619b7e708231d408ef0c/html5/thumbnails/12.jpg)
Page 12 of 14
Compressed Sensing… M.E. Davies
However what we really want to tackle problems like this…
originalLinear reconstruction (5x under-sampled)
Nonlinear reconstruction (5x under-sampled)
Rapid dynamic MRI acquisition in practice
(data courtesy of Ian Marshall & Terry Tao, SFC Brain Imaging Centre)
![Page 13: Compressed Sensing… M.E. Davieshomepages.inf.ed.ac.uk › ckiw › rpml › SparseRepandCS.pdf · Potential applications… Compressed Sensing provides a new way of thinking about](https://reader035.fdocuments.in/reader035/viewer/2022070804/5f03619b7e708231d408ef0c/html5/thumbnails/13.jpg)
Page 13 of 14
Compressed Sensing… M.E. Davies
Synthetic Aperture RadarCompressed Sensing ideas can also be applied to reduced sampling in Synthetic Aperture Radar: Samples lines from spatial Fourier transform. Sub-sampling lines allows adaptive antennas to multi-task (interrupted SAR)
![Page 14: Compressed Sensing… M.E. Davieshomepages.inf.ed.ac.uk › ckiw › rpml › SparseRepandCS.pdf · Potential applications… Compressed Sensing provides a new way of thinking about](https://reader035.fdocuments.in/reader035/viewer/2022070804/5f03619b7e708231d408ef0c/html5/thumbnails/14.jpg)
Page 14 of 14
Compressed Sensing… M.E. Davies
Potential applications…
Compressed Sensing provides a new way of thinking about signal acquisition. Potential applications areas include:
• Medical imaging
• Distributed sensing
• Seismic imaging
• Remote sensing (e.g. Synthetic Aperture Radar)
• Acoustic array processing (source separation)
• High rate analogue-to-digital conversion (DARPA A2I research program)
• The single pixel camera (novel Terahertz imaging)