Richard Baraniuk Rice University compressive nonsensing.
-
Upload
gwendoline-baldwin -
Category
Documents
-
view
224 -
download
3
Transcript of Richard Baraniuk Rice University compressive nonsensing.
Case in Point: MR Imaging
• Measurements very expensive
• $1-3 million per machine
• 30 minutes per scan
Case in Point: DARPA ARGUS-IS
• 1.8 Gpixel image sensor– video rate output:
444 Gbits/s– comm data rate:
274 Mbits/s
factor of 1600xway out of reach ofexisting compressiontechnology
• Reconnaissancewithout conscience– too much data to transmit to a ground station– too much data to make effective real-time decisions
Sparsity
pixels largewaveletcoefficients
(blue = 0)
widebandsignalsamples
largeGabor (TF)coefficients
time
frequ
en
cy
Sparsity
pixels largewaveletcoefficients
(blue = 0)
sparsesignal
nonzeroentries
nonlinear signal model
Dimensionality Reduction
• When data is sparse/compressible, can directly acquire a compressed representation with no/little information loss through linear dimensionality reduction
measurements sparsesignal
nonzeroentries
Stable Embedding• An information preserving projection preserves
the geometry of the set of sparse signals
• SE ensures that
K-dim subspaces
Stable Embedding• An information preserving projection preserves
the geometry of the set of sparse signals
• SE ensures that
Random Embedding is Stable
• Measurements = random linear combinations of the entries of
• No information loss for sparse vectors whp
measurements sparsesignal
nonzeroentries
Signal Recovery
• Goal: Recover signal from measurements
• Problem: Randomprojection not full rank(ill-posed inverse problem)
• Solution: Exploit the sparse/compressiblegeometry of acquired signal
• Recovery via (convex) sparsitypenalty or greedy algorithms[Donoho; Candes, Romberg, Tao, 2004]
Signal Recovery
• Goal: Recover signal from measurements
• Problem: Randomprojection not full rank(ill-posed inverse problem)
• Solution: Exploit the sparse/compressiblegeometry of acquired signal
• Recovery via (convex) sparsitypenalty or greedy algorithms[Donoho; Candes, Romberg, Tao, 2004]
“Single-Pixel” CS Camera
randompattern onDMD array
DMD DMD
single photon detector
imagereconstructionorprocessing
w/ Kevin Kelly
scene
“Single-Pixel” CS Camera
randompattern onDMD array
DMD DMD
single photon detector
imagereconstructionorprocessing
scene
• Flip mirror array M times to acquire M measurements• Sparsity-based recovery
…
Random Demodulator
• CS enables sampling near signal’s (low) “information rate” rather than its (high) Nyquist rate
A2Isampling rate
number oftones /window
Nyquistbandwidth
• Problem: In contrast to Moore’s Law, ADC performance doubles only every 6-8 years
Example: Frequency Hopper
20x sub-Nyquist sampling
spectrogram sparsogram
Nyquist rate sampling
• Sparse in time-frequency
challenge 1data too expensive
means fewer expensive measurements needed for the same resolution scan
6640 citations
dsp.rice.edu/cs archive >1500 papers
nuit-blanche.blogspot.com > 1 posting/sec
2004—2014
9797 citations
From: M. V. Subject: Interesting application for compressed sensingDate: June 10, 2011 at 11:37:31 PM EDTTo: [email protected], [email protected]
Drs. Candes and Romberg,You may have already been approached about this, but I feel I should say something in case you haven't. I'm writing to you because I recently read an article in Wired Magazine about compressed sensing
I'm excited about the applications CS could have in many fields, but today I was reminded of a specific application where CS could conceivably settle an area of dispute between mainstream historians and Roswell UFO theorists. As outlined in the linked video below, Dr. Rudiak has analyzed photos from 1947 in which a General Ramey appears holding a typewritten letter from which Rudiak believes he has been able to discern a number of words which he believes substantiate the extraterrestrial hypothesis for the Roswell Incident). For your perusal, I've located a "hi-res" copy of the cropped image of the letter in Ramey's hand.
I hope to hear back from you. Is this an application where compressed sensing could be useful? Any chance you would consider trying it?
Thank you for your time,M. V.
P.S. - Out of personal curiosity, are there currently any commercial entities involved in developing CS-based software for use by the general public?
--
Back to Reality
• “There's no such thing as a free lunch”
• “Something for Nothing” theorems
• Dimensionality reductionis no exception
• Result: CompressiveNonsensing
Measurement Noise
• Stable recoverywith additive measurement noise
• Noise is added to
• Stability: noise only mildly amplified in recovered signal
Signal Noise
• Often seek recoverywith additive signal noise
• Noise is added to
• Noise folding: signal noise amplified in by 3dB for every doubling of
• Same effect seen in classical “bandpass subsampling”
[Davenport, Laska, Treichler, B 2011]
“Tail Folding”
• Can model compressible(approx sparse) signals as
“signal” + “tail”
• Tail “folds” into signal asincreases
sorted index
“signal”
“tail”
[Davies, Guo, 2011; Davenport, Laska, Treichler, B 2011]
All Is Not Lost – Dynamic Range
• In wideband ADC apps
• As amount of subsampling grows, can employan ADC with a lower sampling rate and hence higher-resolution quantizer
Dynamic Range
• CS can significantly boost the ENOB of an ADC system for sparse signals
conventional ADC
CS ADC w/ sparsity
log sampling frequency
state
d n
um
ber
of
bit
s
Dynamic Range
• As amount of subsampling grows, can employan ADC with a lower sampling rate and hence higher-resolution quantizer
• Thus dynamic range of CS ADC can significantly exceed Nyquist ADC
• With current ADC trends, dynamic range gain is theoretically 7.9dB for each doubling in
Adaptivity
• Say we know the locations of the non-zero entriesin
• Then we boostthe SNR by
• Motivates adaptivesensing strategiesthat bypass the noise-folding tradeoff[Haupt, Castro, Nowak, B 2009; Candes, Davenport 2011]
columns
’
CS and Quantization
• Vast majority of work in CS assumes the measurements are real-valued
• In practice, measurements must be quantized (nonlinear)
• Should measure CS performance in terms of number of measurement bits
rather thannumber of (real-valued) measurements
• Limited progress– large number of bits per measurement– 1 bit per measurement
N=2000, K=20, M = (total bits)/(bits per meas)
CS and Quantization
1 bit6 bits
8 bits
10 bits12 bits/meas
4 bits
2 bits
Weak Models
• Sparsity models in CS emphasize discrete bases and frames– DFT, wavelets, …
• But in real data acquisition problems, the world is continuous, not discrete
The Grid Problem• Consider “frequency sparse” signal
– suggests the DFT sparsity basis
• Easy CS problem: K=1frequency
• Hard CS problem: K=1frequency
slow decay due to sincinterpolation of off-grid sinusoids(asymptotically, signal is not even in L1)
Going Off the Grid• Spectral CS [Duarte, B, 2010]
– discrete formulation
• CS Off the Grid [Tang, Bhaskar, Shah, Recht, 2012]– continuous formulation
best case
worst caseaverage case
Spectral CS
20dB
Misguided Focus on Recovery
• Recall the data deluge problem in sensing– ex: large-scale imaging, HSI, video,
ultrawideband ADC, – data ambient dimension N too large
• When N ~ billions, signal recoverybecomes problematic, if notimpossible
• Solution: Perform signalexploitation directly on the compressive measurements
Compressive Signal Processing
• Many applications involve signal inference and not reconstruction
detection < classification < estimation < reconstruction
• Good news: CS supports efficient learning, inference, processing directly on compressive measurements
• Random projections ~ sufficient statisticsfor signals with concise geometrical structure
Classification• Simple object classification problem
– AWGN: nearest neighbor classifier
• Common issue:– L unknown articulation parameters
• Common solution: matched filter– find nearest neighbor under all articulations
CS-based Classification• Target images form a low-dimensional
manifold as the target articulates– random projections preserve information
in these manifolds if
• CS-based classifier: smashed filter– find nearest neighbor under all articulations
under random projection [Davenport, B, et al 2006]
Smashed Filter
• Random shift and rotation (L=3 dim. manifold)• White Gaussian noise added to measurements• Goals: identify most likely shift/rotation parameters
identify most likely class
number of measurements Mnumber of measurements Mavg
. sh
ift
esti
mate
err
or
cla
ssifi
cati
on
rate
(%
)more noise
more noise
Frequency Tracking
• Compressive Phase Locked Loop (PLL) – key idea: phase detector in PLL computes inner product
between signal and oscillator output– RIP ensures we can compute this inner product between
corresponding low-rate CS measurements
CS-PLL w/ 20xundersampling
Performance Guarantees
• CS performance guarantees– RIP, incoherence, phase transition
• To date, rigorous results only for random matrices– practically not useful– often pessimistic
• Need rigorous guarantees for non-random, structured sampling matrices with fast algorithms– analogous to the progress in coding theory from Shannon’s
original random codes to modern codes
12-Step ProgramTo End Compressive Nonsensing
1. Don’t give in to the hype surrounding CS
2. Resist the urge to blindly apply L1 minimization
3. Face up to robustness issues
4. Deal with measurement quantization
5. Develop more realistic signal models
6. Develop practical sensing matrices beyond random
7. Develop more efficient recovery algorithms
8. Develop rigorous performance guarantees for practical CS systems
9. Exploit signals directly in the compressive domain
10. Don’t give in to the hype surrounding CS
11. Resist the urge to blindly apply L1 minimization
12. Don’t give in to the hype surrounding CS