Hybrid Dense/Sparse Matrices in Compressed Sensing Reconstruction Ilya Poltorak Dror Baron Deanna...
-
date post
22-Dec-2015 -
Category
Documents
-
view
217 -
download
1
Transcript of Hybrid Dense/Sparse Matrices in Compressed Sensing Reconstruction Ilya Poltorak Dror Baron Deanna...
![Page 1: Hybrid Dense/Sparse Matrices in Compressed Sensing Reconstruction Ilya Poltorak Dror Baron Deanna Needell The work has been supported by the Israel Science.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649d7e5503460f94a61a3b/html5/thumbnails/1.jpg)
Hybrid Dense/Sparse Matrices in Compressed Sensing Reconstruction
Ilya Poltorak
Dror Baron
Deanna Needell
The work has been supported by the Israel Science Foundation and National Science Foundation.
![Page 2: Hybrid Dense/Sparse Matrices in Compressed Sensing Reconstruction Ilya Poltorak Dror Baron Deanna Needell The work has been supported by the Israel Science.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649d7e5503460f94a61a3b/html5/thumbnails/2.jpg)
CS Measurement• Replace samples by more general encoder
based on a few linear projections (inner products)
measurements sparsesignal
# non-zeros
![Page 3: Hybrid Dense/Sparse Matrices in Compressed Sensing Reconstruction Ilya Poltorak Dror Baron Deanna Needell The work has been supported by the Israel Science.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649d7e5503460f94a61a3b/html5/thumbnails/3.jpg)
Caveats
• Input x strictly sparse w/ real values• Noiseless measurements
– noise can be addressed (later)
• Assumptions relevant to content distribution (later)
![Page 4: Hybrid Dense/Sparse Matrices in Compressed Sensing Reconstruction Ilya Poltorak Dror Baron Deanna Needell The work has been supported by the Israel Science.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649d7e5503460f94a61a3b/html5/thumbnails/4.jpg)
Why is Decoding Expensive?
measurementssparsesignal
nonzeroentries
Culprit: dense, unstructured
![Page 5: Hybrid Dense/Sparse Matrices in Compressed Sensing Reconstruction Ilya Poltorak Dror Baron Deanna Needell The work has been supported by the Israel Science.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649d7e5503460f94a61a3b/html5/thumbnails/5.jpg)
Sparse Measurement Matrices (dense later!)
measurementssparsesignal
nonzeroentries
• LDPC measurement matrix (sparse)• Only {-1,0,+1} in • Each row of contains L randomly placed nonzeros • Fast matrix-vector multiplication
fast encoding & decoding
![Page 6: Hybrid Dense/Sparse Matrices in Compressed Sensing Reconstruction Ilya Poltorak Dror Baron Deanna Needell The work has been supported by the Israel Science.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649d7e5503460f94a61a3b/html5/thumbnails/6.jpg)
Example
0
1
1
4
0 1 1 0 0 0
0 0 0 1 1 0
1 1 0 0 1 0
0 0 0 0 1 1
?
?
?
?
?
?
![Page 7: Hybrid Dense/Sparse Matrices in Compressed Sensing Reconstruction Ilya Poltorak Dror Baron Deanna Needell The work has been supported by the Israel Science.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649d7e5503460f94a61a3b/html5/thumbnails/7.jpg)
Example
0
1
1
4
0 1 1 0 0 0
0 0 0 1 1 0
1 1 0 0 1 0
0 0 0 0 1 1
?
?
?
?
?
?
• What does zero measurement imply?• Hint: x strictly sparse
![Page 8: Hybrid Dense/Sparse Matrices in Compressed Sensing Reconstruction Ilya Poltorak Dror Baron Deanna Needell The work has been supported by the Israel Science.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649d7e5503460f94a61a3b/html5/thumbnails/8.jpg)
Example
0
1
1
4
0 1 1 0 0 0
0 0 0 1 1 0
1 1 0 0 1 0
0 0 0 0 1 1
?
0
0
?
?
?
• Graph reduction!
![Page 9: Hybrid Dense/Sparse Matrices in Compressed Sensing Reconstruction Ilya Poltorak Dror Baron Deanna Needell The work has been supported by the Israel Science.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649d7e5503460f94a61a3b/html5/thumbnails/9.jpg)
Example
0
1
1
4
0 1 1 0 0 0
0 0 0 1 1 0
1 1 0 0 1 0
0 0 0 0 1 1
?
0
0
?
?
?
• What do matching measurements imply?• Hint: non-zeros in x are real numbers
![Page 10: Hybrid Dense/Sparse Matrices in Compressed Sensing Reconstruction Ilya Poltorak Dror Baron Deanna Needell The work has been supported by the Israel Science.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649d7e5503460f94a61a3b/html5/thumbnails/10.jpg)
Example
0
1
1
4
0 1 1 0 0 0
0 0 0 1 1 0
1 1 0 0 1 0
0 0 0 0 1 1
0
0
0
0
1
?
• What is the last entry of x?
![Page 11: Hybrid Dense/Sparse Matrices in Compressed Sensing Reconstruction Ilya Poltorak Dror Baron Deanna Needell The work has been supported by the Israel Science.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649d7e5503460f94a61a3b/html5/thumbnails/11.jpg)
Noiseless Algorithm [Luby & Mitzenmacher 2005]
[Sarvotham, Baron, & Baraniuk 2006][Zhang & Pfister 2008]
Phase1: zero measurements
Phase2: matching measurements
Phase3: singleton measurements
Initialize
Done? Arrange outputyesno
typically iterate 2-3 times
![Page 12: Hybrid Dense/Sparse Matrices in Compressed Sensing Reconstruction Ilya Poltorak Dror Baron Deanna Needell The work has been supported by the Israel Science.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649d7e5503460f94a61a3b/html5/thumbnails/12.jpg)
Numbers (4 seconds)
• N=40,000• 5% non-zeros• M=0.22N• L=20 ones per row
Only 2-3 iterations
Iteration, Phase
Zeros Non-zeros Total
1,1 30615 0 30615
1,2 35224 977 36201
1,3 35224 1500 36724
2,1 36800 1500 38300
2,2 37180 1833 39013
2,3 37180 2063 39243
3,1 37268 2063 39331
3,2 37289 2074 39363
3,3 37289 2083 39372
4,1 37289 2083 39372
4,2 37291 2084 39375
4,3 37291 2084 39375
iteration #1
![Page 13: Hybrid Dense/Sparse Matrices in Compressed Sensing Reconstruction Ilya Poltorak Dror Baron Deanna Needell The work has been supported by the Israel Science.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649d7e5503460f94a61a3b/html5/thumbnails/13.jpg)
Challenge
• With measurements parts of signal still not reconstructed
• How do we recover the rest of the signal?
![Page 14: Hybrid Dense/Sparse Matrices in Compressed Sensing Reconstruction Ilya Poltorak Dror Baron Deanna Needell The work has been supported by the Israel Science.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649d7e5503460f94a61a3b/html5/thumbnails/14.jpg)
Solution: Hybrid Dense/Sparse Matrix
• With measurements parts of signal still not reconstructed
• Add extra dense measurements • Residual of signal w/ residual dense columns
residual columns
![Page 15: Hybrid Dense/Sparse Matrices in Compressed Sensing Reconstruction Ilya Poltorak Dror Baron Deanna Needell The work has been supported by the Israel Science.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649d7e5503460f94a61a3b/html5/thumbnails/15.jpg)
Sudocodes with Two-Part Decoding [Sarvotham, Baron, & Baraniuk 2006]
• Sudocodes (related to sudoku)• Graph reduction solves most of CS
problem
• Residual solved via matrix inversion
sudo decoder
Residual via matrix inversion
residual columns
![Page 16: Hybrid Dense/Sparse Matrices in Compressed Sensing Reconstruction Ilya Poltorak Dror Baron Deanna Needell The work has been supported by the Israel Science.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649d7e5503460f94a61a3b/html5/thumbnails/16.jpg)
Contribution 1: Two-Part Reconstruction• Many CS algorithms for sparse matrices
[Gilbert et al., Berinde & Indyk, Sarvotham et al.]
• Many CS algorithms for dense matrices[Cormode & Muthukrishnan, Candes et al., Donoho et al., Gilbert et al., Milenkovic et al., Berinde & Indyk, Zhang & Pfister, Hale et al.,…]
• Solve each part with appropriate algorithm
residual columns
sparse solver
residual via dense solver
![Page 17: Hybrid Dense/Sparse Matrices in Compressed Sensing Reconstruction Ilya Poltorak Dror Baron Deanna Needell The work has been supported by the Israel Science.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649d7e5503460f94a61a3b/html5/thumbnails/17.jpg)
Runtimes (K=0.05N, M=0.22N)
![Page 18: Hybrid Dense/Sparse Matrices in Compressed Sensing Reconstruction Ilya Poltorak Dror Baron Deanna Needell The work has been supported by the Israel Science.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649d7e5503460f94a61a3b/html5/thumbnails/18.jpg)
Theoretical Results [Sarvotham, Baron, & Baraniuk 2006]
• Fast encoder and decoder– sub-linear decoding complexity– caveat: constructing data structure
• Distributed content distribution– sparsified data– measurements stored on different servers– any M measurements suffice
• Strictly sparse signals, noiseless measurements
![Page 19: Hybrid Dense/Sparse Matrices in Compressed Sensing Reconstruction Ilya Poltorak Dror Baron Deanna Needell The work has been supported by the Israel Science.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649d7e5503460f94a61a3b/html5/thumbnails/19.jpg)
Contribution 2: Noisy Measurements
• Results can be extended to noisy measurements
• Part 1 (zero measurements): measurement |ym|<
• Part 2 (matching): |yi-yj|<
• Part 3 (singleton): unchanged
![Page 20: Hybrid Dense/Sparse Matrices in Compressed Sensing Reconstruction Ilya Poltorak Dror Baron Deanna Needell The work has been supported by the Israel Science.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649d7e5503460f94a61a3b/html5/thumbnails/20.jpg)
Problems with Noisy Measurements
• Multiple iterations alias noise into next iteration!Use one iteration
• Requires small threshold (large SNR)
• Contribution 3: Provable reconstruction• deterministic & random variants
![Page 21: Hybrid Dense/Sparse Matrices in Compressed Sensing Reconstruction Ilya Poltorak Dror Baron Deanna Needell The work has been supported by the Israel Science.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649d7e5503460f94a61a3b/html5/thumbnails/21.jpg)
Summary
• Hybrid Dense/Sparse Matrix– Two-part reconstruction
• Simple (cute?) algorithm
• Fast
• Applicable to content distribution
• Expandable to measurement noise
![Page 22: Hybrid Dense/Sparse Matrices in Compressed Sensing Reconstruction Ilya Poltorak Dror Baron Deanna Needell The work has been supported by the Israel Science.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649d7e5503460f94a61a3b/html5/thumbnails/22.jpg)
THE END