Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh...
-
Upload
prudence-shelton -
Category
Documents
-
view
221 -
download
1
Transcript of Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh...
![Page 1: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/1.jpg)
CompressiveSensingA New Approach to Image Acquisition and Processing
Richard BaraniukMona Sheikh
Rice University
![Page 2: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/2.jpg)
The Digital Universe
• Size: 281 billion gigabytes generated in 2007digital bits > stars in the universegrowing by a factor of 10 every 5 years > Avogadro’s number (6.02x1023) in 15 years
• Growth fueled by multimedia data audio, images, video, surveillance cameras, sensor nets, …(2B fotos on Flickr, 4.2M security cameras in UK, …)
• In 2007 digital data generated > total storageby 2011, ½ of digital universe will have no home
[Source: IDC Whitepaper “The Diverse and Exploding Digital Universe” March 2008]
![Page 3: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/3.jpg)
Act I What’s wrong with today’s multimedia sensor systems?why go to all the work to acquire massiveamounts of multimedia data only to throw much/most of it away?
Act II One way out: dimensionality reduction (compressive sensing)enables the design of radically new sensors and systems
Act III Compressive sensing in actionnew cameras, imagers, ADCs, …
Postlude Open-access education
![Page 4: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/4.jpg)
Sense by Sampling
sample
![Page 5: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/5.jpg)
Sense by Sampling
sample too much data!
![Page 6: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/6.jpg)
Sense then Compress
compress
decompress
sample
JPEGJPEG2000
…
![Page 7: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/7.jpg)
Sparsity
pixels largewaveletcoefficients
(blue = 0)
![Page 8: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/8.jpg)
Sparsity
pixels largewaveletcoefficients
(blue = 0)
widebandsignalsamples
largeGabor (TF)coefficients
time
frequency
![Page 9: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/9.jpg)
What’s Wrong with this Picture?
• Why go to all the work to acquire N samples only to discard all but K pieces of data?
compress
decompress
sample
![Page 10: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/10.jpg)
What’s Wrong with this Picture?linear processinglinear signal model(bandlimited subspace)
compress
decompress
sample
nonlinear processingnonlinear signal model(union of subspaces)
![Page 11: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/11.jpg)
Compressive Sensing
• Directly acquire “compressed” data via dimensionality reduction
• Replace samples by more general “measurements”
compressive sensing
recover
![Page 12: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/12.jpg)
Compressive SensingTheory
A Geometrical Perspective
![Page 13: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/13.jpg)
• Signal is -sparse in basis/dictionary– WLOG assume sparse in space domain
Sampling
sparsesignal
nonzeroentries
![Page 14: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/14.jpg)
• Signal is -sparse in basis/dictionary– WLOG assume sparse in space domain
• Sampling
sparsesignal
nonzeroentries
measurements
Sampling
![Page 15: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/15.jpg)
Compressive Sensing
• When data is sparse/compressible, can directly acquire a condensed representation with no/little information loss through linear dimensionality reduction
measurements sparsesignal
nonzeroentries
![Page 16: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/16.jpg)
How Can It Work?
• Projection not full rank…
… and so loses information in general
• Ex: Infinitely many ’s map to the same(null space)
![Page 17: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/17.jpg)
How Can It Work?
• Projection not full rank…
… and so loses information in general
• But we are only interested in sparse vectors
columns
![Page 18: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/18.jpg)
How Can It Work?
• Projection not full rank…
… and so loses information in general
• But we are only interested in sparse vectors
• is effectively MxK
columns
![Page 19: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/19.jpg)
How Can It Work?
• Projection not full rank…
… and so loses information in general
• But we are only interested in sparse vectors
• Design so that each of its MxK submatrices are full rank (ideally close to orthobasis)– Restricted Isometry Property (RIP)
columns
![Page 20: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/20.jpg)
Geometrical Interpretation of RIP
sparsesignal
nonzeroentries
• An information preserving projection preserves the geometry of the set of sparse signals
![Page 21: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/21.jpg)
Geometrical Interpretation of RIP
sparsesignal
nonzeroentries
• An information preserving projection preserves the geometry of the set of sparse signals
• Model: union of K-dimensional subspaces aligned w/ coordinate axes (highly nonlinear!)
![Page 22: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/22.jpg)
RIP = Stable Embedding• An information preserving projection preserves
the geometry of the set of sparse signals
• RIP ensures that
K-dim subspaces
![Page 23: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/23.jpg)
RIP = Stable Embedding• An information preserving projection preserves
the geometry of the set of sparse signals
• RIP ensures that
![Page 24: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/24.jpg)
How Can It Work?
• Projection not full rank…
… and so loses information in general
• Design so that each of its MxK submatrices are full rank (RIP)
• Unfortunately, a combinatorial, NP-complete design problem
columns
![Page 25: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/25.jpg)
Insight from the 70’s [Kashin, Gluskin]
• Draw at random– iid Gaussian– iid Bernoulli
…
• Then has the RIP with high probability provided
columns
![Page 26: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/26.jpg)
Insight from the 70’s [Kashin, Gluskin]
• Draw at random– iid Gaussian– iid Bernoulli
…
• Then has the RIP with high probability provided
columns
![Page 27: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/27.jpg)
Randomized Sensing
• Measurements = random linear combinations of the entries of
• No information loss for sparse vectors whp
measurements sparsesignal
nonzeroentries
![Page 28: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/28.jpg)
CS Signal Recovery
• Goal: Recover signal from measurements
• Problem: Randomprojection not full rank(ill-posed inverse problem)
• Solution: Exploit the sparse/compressiblegeometry of acquired signal
![Page 29: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/29.jpg)
CS Signal Recovery• Random projection
not full rank
• Recovery problem:givenfind
• Null space
• Search in null space for the “best” according to some criterion– ex: least squares (N-M)-dim hyperplane
at random angle
![Page 30: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/30.jpg)
• Recovery: given(ill-posed inverse problem) find (sparse)
• Optimization:
• Closed-form solution:
Signal Recovery
![Page 31: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/31.jpg)
• Recovery: given(ill-posed inverse problem) find (sparse)
• Optimization:
• Closed-form solution:
• Wrong answer!
Signal Recovery
![Page 32: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/32.jpg)
• Recovery: given(ill-posed inverse problem) find (sparse)
• Optimization:
• Closed-form solution:
• Wrong answer!
Signal Recovery
![Page 33: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/33.jpg)
• Recovery: given(ill-posed inverse problem) find (sparse)
• Optimization:
Signal Recovery
“find sparsest vectorin translated nullspace”
![Page 34: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/34.jpg)
• Recovery: given(ill-posed inverse problem) find (sparse)
• Optimization:
• Correct!
• But NP-Complete alg
Signal Recovery
“find sparsest vectorin translated nullspace”
![Page 35: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/35.jpg)
• Recovery: given(ill-posed inverse problem) find (sparse)
• Optimization:
• Convexify the optimization
Signal Recovery
Candes Romberg Tao Donoho
![Page 36: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/36.jpg)
• Recovery: given(ill-posed inverse problem) find (sparse)
• Optimization:
• Convexify the optimization
• Correct!
• Polynomial time alg(linear programming)
Signal Recovery
![Page 37: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/37.jpg)
Compressive Sensing
• Signal recovery via optimization [Candes, Romberg, Tao; Donoho]
random measurements
sparsesignal
nonzeroentries
![Page 38: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/38.jpg)
Compressive Sensing
• Signal recovery via iterative greedy algorithm
– (orthogonal) matching pursuit [Gilbert, Tropp]
– iterated thresholding [Nowak, Figueiredo; Kingsbury, Reeves; Daubechies, Defrise, De Mol; Blumensath, Davies; …]
– CoSaMP [Needell and Tropp]
random measurements
sparsesignal
nonzeroentries
![Page 39: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/39.jpg)
Iterated Thresholding
update signal estimate
prune signal estimate(best K-term approx)
update residual
![Page 40: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/40.jpg)
Summary: CS
• Encoding: = random linear combinations of the entries of
• Decoding: Recover from via optimization
measurements sparsesignal
nonzeroentries
![Page 41: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/41.jpg)
CS Hallmarks
• Stable– acquisition/recovery process is numerically stable
• Asymmetrical (most processing at decoder) – conventional: smart encoder, dumb decoder– CS: dumb encoder, smart decoder
• Democratic– each measurement carries the same amount of information– robust to measurement loss and quantization– “digital fountain” property
• Random measurements encrypted
• Universal – same random projections / hardware can be used for
any sparse signal class (generic)
![Page 42: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/42.jpg)
Compressive SensingIn Action
![Page 43: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/43.jpg)
Gerhard Richter 4096 Farben / 4096 Colours
1974254 cm X 254 cmLaquer on CanvasCatalogue Raisonné: 359
Museum Collection:Staatliche Kunstsammlungen Dresden (on loan)
Sales history: 11 May 2004Christie's New York Post-War and Contemporary Art (Evening Sale), Lot 34US$3,703,500
![Page 44: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/44.jpg)
“Single-Pixel” CS Camera
randompattern onDMD array
DMD DMD
single photon detector
imagereconstructionorprocessing
w/ Kevin Kelly
scene
![Page 45: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/45.jpg)
“Single-Pixel” CS Camera
randompattern onDMD array
DMD DMD
single photon detector
imagereconstructionorprocessing
scene
• Flip mirror array M times to acquire M measurements• Sparsity-based (linear programming) recovery
…
![Page 46: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/46.jpg)
First Image Acquisition
target 65536 pixels
1300 measurements (2%)
11000 measurements (16%)
![Page 47: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/47.jpg)
![Page 48: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/48.jpg)
Utility?
DMD DMD
single photon detector
Fairchild100Mpixel
CCD
![Page 49: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/49.jpg)
CS Low-Light Imaging with PMT
true color low-light imaging
256 x 256 image with 10:1 compression
[Nature Photonics, April 2007]
![Page 50: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/50.jpg)
CS Infrared Camera
20% 5%
![Page 51: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/51.jpg)
CS Hyperspectral Imager
spectrometer
hyperspectral data cube450-850nm
1M space x wavelength voxels200k random sums
![Page 52: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/52.jpg)
CS MRI
• Lustig, Pauly, Donoho et al at Stanford
• Design MRI sampling patterns (in frequency/k-space) to be close to random
• Speed up acquisition by reducing number of samples required for a given image reconstruction quality
![Page 53: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/53.jpg)
Multi-Slice Brain Imaging [M. Lustig]
![Page 54: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/54.jpg)
3DFT Angiography [M. Lustig]
![Page 55: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/55.jpg)
CS In Action• CS makes sense when measurements
are expensive
• Ultrawideband A/D converters[DARPA “Analog to Information” program]
• Camera networks– sensing/compression/fusion
• Radar, sonar, array processing– exploit spatial sparsity of targets
• DNA microarrays– smaller, more agile arrays for
bio-sensing [Milenkovic, Orlitsky, B]
![Page 56: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/56.jpg)
Summary
• Compressive sensing– randomized dimensionality reduction– integrates sensing, compression, processing– exploits signal sparsity information– enables new sensing modalities, architectures, systems– relies on large-scale optimization
• Why CS works: preserves information in signals with concise geometric
structure sparse signals | compressible signals | manifolds
![Page 57: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/57.jpg)
Open Research Issues
• Links with information theory– new encoding matrix design via codes (LDPC, fountains)– new decoding algorithms (BP, etc.)– quantization and rate distortion theory
• Links with machine learning– Johnson-Lindenstrauss, manifold embedding, RIP
• Processing/inference on random projections– filtering, tracking, interference cancellation, …
• Multi-signal CS– array processing, localization, sensor networks, …
• CS hardware– ADCs, receivers, cameras, imagers, radars, …
![Page 58: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/58.jpg)
Beyond Sparsity
• Sparse signal model captures simplistic primary structure
wavelets:natural images
Gabor atoms:chirps/tones
pixels:background subtracted
images
![Page 59: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/59.jpg)
Structured Sparsity
• Sparse signal model captures simplistic primary structure
• Modern compression/processing algorithms capture richer secondary coefficient structure
wavelets:natural images
Gabor atoms:chirps/tones
pixels:background subtracted
images
![Page 60: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/60.jpg)
Wavelet Tree-Sparse Recovery
target signal CoSaMP, (RMSE=1.12)
Tree-sparse CoSaMP (RMSE=0.037)
N=1024M=80
L1-minimization(RMSE=0.751)
![Page 61: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/61.jpg)
dsp.rice.edu/cs
![Page 62: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/62.jpg)
![Page 63: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/63.jpg)
vibrantinteractivecommunityconnectedinnovativeup-to-dateefficienteffective
![Page 64: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/64.jpg)
create
share
usefreely
re-useopenly
vibrantinteractivecommunityconnectedinnovativeup-to-dateefficienteffective
![Page 65: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/65.jpg)
create
share
usefreely
re-useopenly
vibrantinteractivecommunityconnectedinnovativeup-to-dateefficienteffective
![Page 66: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/66.jpg)
create share
use re-usefreely openly
why?
today’s textbooks/courses lock up educational ideas – closed formats– closed copyrights
![Page 67: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/67.jpg)
create share
use re-usefreely openly
open education
![Page 68: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/68.jpg)
OEenablers
![Page 69: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/69.jpg)
enabler 1: technology
Web/XMLmodularizationcommon framework for sharing
Internetvirtually free distributionvirtually infinite, permanent
storage
![Page 70: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/70.jpg)
Connexions – primordial state
![Page 71: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/71.jpg)
textbook / course
![Page 72: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/72.jpg)
personalize
![Page 73: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/73.jpg)
reuse
![Page 74: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/74.jpg)
enabler 2: new IP
intellectual propertyand copyright
make content safe to share
common legal vocabulary
inspiration: open-source software (ex: Linux)
![Page 75: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/75.jpg)
today’s textbook pipeline
authoring
editing
quality control
publishing
distribution
![Page 76: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/76.jpg)
open education ecosystem
authoring
editing
quality control
publishing
distribution
feedbackpeersuserslearning
![Page 77: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/77.jpg)
Connexions (cnx.org)
non-profit OE platform/communitybased at Rice U since 1999($6m+ in foundation funding)
950+ open textbooks/courses15000+ reusable Lego modules1.5 million+ unique users/month
free on-linelow-cost in print
![Page 78: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/78.jpg)
![Page 79: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/79.jpg)
Universality
• Random measurements can be used for signals sparse in any basis
![Page 80: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/80.jpg)
Universality
• Random measurements can be used for signals sparse in any basis
![Page 81: Compressive Sensing A New Approach to Image Acquisition and Processing Richard Baraniuk Mona Sheikh Rice University.](https://reader038.fdocuments.in/reader038/viewer/2022110100/56649ddf5503460f94ad8f5c/html5/thumbnails/81.jpg)
Universality
• Random measurements can be used for signals sparse in any basis
sparsecoefficient
vector
nonzeroentries