Compressive Spectral Image Sensing, Processing, and Optimization
-
Upload
distinguished-lecturer-series-leon-the-mathematician -
Category
Technology
-
view
189 -
download
3
description
Transcript of Compressive Spectral Image Sensing, Processing, and Optimization
![Page 1: Compressive Spectral Image Sensing, Processing, and Optimization](https://reader034.fdocuments.in/reader034/viewer/2022051514/54859cd9b4af9f7d0d8b4e78/html5/thumbnails/1.jpg)
Compressive Spectral ImageSensing and Optimization
Gonzalo R. ArceCharles Black Evans Professor
University of DelawareNewark, Delaware, USA 19716
Distinguished Lecturer SeriesAristotle University of Thessaloniki
March 14, 2014
![Page 2: Compressive Spectral Image Sensing, Processing, and Optimization](https://reader034.fdocuments.in/reader034/viewer/2022051514/54859cd9b4af9f7d0d8b4e78/html5/thumbnails/2.jpg)
Optical imaging and spectroscopy discovers the characteristics of scenes andmaterials by capturing EM radiation in the 0.01 to 10000 nm spectrum window.
Sensitive not only to spatial and spectral information of a scene, but also topolarization, tomographic, angular, and even chemical composition.
Multidimensional imaging provides dimension preserving mappings
y = Hf,
where H is the “forward model“, characterizing the focal plane data.
Optical coding shapes H, under some criteria, to match the computational toolsof inverse problems to significantly improve the overall imaging performance.
MURA Hadamard SPECT
Gonzalo R. Arce () Compressive Spectral Image Sensing Oct., 2013 2 / 50
![Page 3: Compressive Spectral Image Sensing, Processing, and Optimization](https://reader034.fdocuments.in/reader034/viewer/2022051514/54859cd9b4af9f7d0d8b4e78/html5/thumbnails/3.jpg)
THE SPECTRAL IMAGING PROBLEM?
Push broom spectral imaging: Traditional approach, expensive, lowsensing speed, senses N × N × L voxels
Optical Filters; Again senses N × N × L voxels; limited by number ofcolors
Gonzalo R. Arce () Compressive Spectral Image Sensing Oct., 2013 3 / 50
![Page 4: Compressive Spectral Image Sensing, Processing, and Optimization](https://reader034.fdocuments.in/reader034/viewer/2022051514/54859cd9b4af9f7d0d8b4e78/html5/thumbnails/4.jpg)
Compressive Spectral Imaging (CASSI), New revolutionary method, CSmakes a significant difference, senses only N2 N × N × L
Coded apertures arethe only variableelement.
Coded Apertures arethe key elements inCASSI.
Gonzalo R. Arce () Compressive Spectral Image Sensing Oct., 2013 4 / 50
![Page 5: Compressive Spectral Image Sensing, Processing, and Optimization](https://reader034.fdocuments.in/reader034/viewer/2022051514/54859cd9b4af9f7d0d8b4e78/html5/thumbnails/5.jpg)
WHY IS THIS IMPORTANT?Remote sensing and surveillance
Visible, NIR, SWIR
Devices are challenging in NIR and SWIR: cost, size,resolution, cooling
Gonzalo R. Arce () Compressive Spectral Image Sensing Oct., 2013 5 / 50
![Page 6: Compressive Spectral Image Sensing, Processing, and Optimization](https://reader034.fdocuments.in/reader034/viewer/2022051514/54859cd9b4af9f7d0d8b4e78/html5/thumbnails/6.jpg)
WHY IS THIS IMPORTANT?Medical Imaging: Vascular tissue imaging, angiography,contrast agent
paint restoration
Compressive Spectral ImagingReduce sensing complexity
Gonzalo R. Arce () Compressive Spectral Image Sensing Oct., 2013 6 / 50
![Page 7: Compressive Spectral Image Sensing, Processing, and Optimization](https://reader034.fdocuments.in/reader034/viewer/2022051514/54859cd9b4af9f7d0d8b4e78/html5/thumbnails/7.jpg)
Introduction
Compressive sensing introduced by Donoho†, Candès‡, Tao,Romberg...
Measurements are given by y = Φxy
xΦ
M x 1
Measurements M x N
Sampling Operator
N x 1
Sparse Signal
A sparse solution x is recovered from y by solving the inverseproblem
x = minx‖x‖1 s.t. y = Φx .
†Donoho. IEEE Trans. on Information Theory. December 2006.‡Candès, Romberg and Tao. IEEE Trans. on Information Theory. April 2006.
Gonzalo R. Arce () Compressive Spectral Image Sensing Oct., 2013 7 / 50
![Page 8: Compressive Spectral Image Sensing, Processing, and Optimization](https://reader034.fdocuments.in/reader034/viewer/2022051514/54859cd9b4af9f7d0d8b4e78/html5/thumbnails/8.jpg)
Introduction
Measurements are given by y = Φx
y ΨΦ α
x
A sparse solution α is recovered from y by solving the inverseproblem
α = minα‖α‖1 s.t. y = ΦΨα.
Gonzalo R. Arce () Compressive Spectral Image Sensing Oct., 2013 8 / 50
![Page 9: Compressive Spectral Image Sensing, Processing, and Optimization](https://reader034.fdocuments.in/reader034/viewer/2022051514/54859cd9b4af9f7d0d8b4e78/html5/thumbnails/9.jpg)
Introduction
Datacube
f = Ψθ
Compressive Measurements
g = HΨθ + w
Underdetermined system of equations
f = Ψminθ‖g− HΨθ‖2 + τ‖θ‖1
Gonzalo R. Arce () Compressive Spectral Image Sensing Oct., 2013 9 / 50
![Page 10: Compressive Spectral Image Sensing, Processing, and Optimization](https://reader034.fdocuments.in/reader034/viewer/2022051514/54859cd9b4af9f7d0d8b4e78/html5/thumbnails/10.jpg)
Coded-Aperture Spectral Imaging (CASSI)
Gonzalo R. Arce () Compressive Spectral Image Sensing Oct., 2013 10 / 50
![Page 11: Compressive Spectral Image Sensing, Processing, and Optimization](https://reader034.fdocuments.in/reader034/viewer/2022051514/54859cd9b4af9f7d0d8b4e78/html5/thumbnails/11.jpg)
Coded-Aperture Spectral Imaging (CASSI)
Gonzalo R. Arce () Compressive Spectral Image Sensing Oct., 2013 11 / 50
![Page 12: Compressive Spectral Image Sensing, Processing, and Optimization](https://reader034.fdocuments.in/reader034/viewer/2022051514/54859cd9b4af9f7d0d8b4e78/html5/thumbnails/12.jpg)
Undetermined system of equations: N ×M × L Unknowns andN(M + L− 1) Equations.
Gonzalo R. Arce () Compressive Spectral Image Sensing Oct., 2013 12 / 50
![Page 13: Compressive Spectral Image Sensing, Processing, and Optimization](https://reader034.fdocuments.in/reader034/viewer/2022051514/54859cd9b4af9f7d0d8b4e78/html5/thumbnails/13.jpg)
Computational Model
A single shot compressive measurement across the FPA:
Gnm =L−1∑i=0
Fi(n+m)mTi(n+m) + win
F is the N ×M × L datacubeT is the binary code aperturew is the sensing noise
In vector form, the FPA measurement can be written as
g = Hf + w
H accounts for the aperture code and the dispersive element.
Gonzalo R. Arce () Compressive Spectral Image Sensing Oct., 2013 13 / 50
![Page 14: Compressive Spectral Image Sensing, Processing, and Optimization](https://reader034.fdocuments.in/reader034/viewer/2022051514/54859cd9b4af9f7d0d8b4e78/html5/thumbnails/14.jpg)
CASSI Multishot Matrix Model
g0
g1
...gk−1
=
H0
H1...
Hk−1
f, (1)
g = Hf, (2)
where H ∈ 0, 1N(M+L−1)K×NML.
Multi-shot coding done by using multiple coded apertures or aDigital-Micromirror-Device (DMD)
Gonzalo R. Arce () Compressive Spectral Image Sensing Oct., 2013 14 / 50
![Page 15: Compressive Spectral Image Sensing, Processing, and Optimization](https://reader034.fdocuments.in/reader034/viewer/2022051514/54859cd9b4af9f7d0d8b4e78/html5/thumbnails/15.jpg)
Matrix CASSI representation g = Hf
Data cube:N × N × L
Spectral bands: L
Spatial resolution:N × N
Sensor sizeN × (N + L− 1)
V=N(N+L-1)
Gonzalo R. Arce () Compressive Spectral Image Sensing Oct., 2013 15 / 50
![Page 16: Compressive Spectral Image Sensing, Processing, and Optimization](https://reader034.fdocuments.in/reader034/viewer/2022051514/54859cd9b4af9f7d0d8b4e78/html5/thumbnails/16.jpg)
Coded Aperture Optimization: Sensing andReconstruction
Restricted Isometry Property in CASSIGiven
g = HΨ︸︷︷︸A
θ with |θθθ| = S
The RIP of the CASSI matrix A is defined as the smallest constant δssuch that
(1− δs) ||θθθ||22 ≤ ||Aθθθ||22 ≤ (1 + δs) ||θθθ||22, (3)
whereδs = max
T⊂[n],|T |≤S
√λmax
(A|T ||T | − I
)(4)
A|T ||T | = AT|T |A|T |, A|T | is a m × |T | matrix whose columns are equal to
|T | columns of the CASSI matrix A, and λmax (.) denotes the largesteigenvalue.
Gonzalo R. Arce () Compressive Spectral Image Sensing Oct., 2013 16 / 50
![Page 17: Compressive Spectral Image Sensing, Processing, and Optimization](https://reader034.fdocuments.in/reader034/viewer/2022051514/54859cd9b4af9f7d0d8b4e78/html5/thumbnails/17.jpg)
Let the entries of ΨΨΨ be Ψj,k and let the columns of ΨΨΨ be [ψψψ0, . . . ,ψψψn−1].The entries of A|T | can be written as(
A|T |)
jk = (HψψψΩk )j = hTj ψψψΩk
=L−1∑r=0
(ti)
j−rNΨj+r(N′),Ωk
for j = 0, . . . ,m − 1, k = 0, . . . , |T | − 1, where i = bj/Vc, N ′ = N2 − N,and Ωk ∈ 0, . . . ,n − 1. The entries of A|T ||T | can be expressed as
(A|T ||T |
)jk =
K−1∑i=0
V−1∑`=0
L−1∑r=0
L−1∑u=0
(ti)`−rN
(ti)`−uN
Ψ`+rN′,Ωj Ψ`+uN′,Ωk (5)
for j , k = 0, . . . , |T | − 1.
Note that the coded aperture products(ti)
`−rN
(ti)
`−uN determine theeigenvalues of A|T ||T |, and consequently they determine the constantδs in the RIP.
Gonzalo R. Arce () Compressive Spectral Image Sensing Oct., 2013 17 / 50
![Page 18: Compressive Spectral Image Sensing, Processing, and Optimization](https://reader034.fdocuments.in/reader034/viewer/2022051514/54859cd9b4af9f7d0d8b4e78/html5/thumbnails/18.jpg)
Boolean Coded Apertures
An optimal ensemble of four 64× 64 boolean coded apertures.
Each spatialcoordinate in theensemble containsonly one 1-valueentry.
Gonzalo R. Arce () Compressive Spectral Image Sensing Oct., 2013 18 / 50
![Page 19: Compressive Spectral Image Sensing, Processing, and Optimization](https://reader034.fdocuments.in/reader034/viewer/2022051514/54859cd9b4af9f7d0d8b4e78/html5/thumbnails/19.jpg)
Bernoulli Coded Apertures
An ensemble of four 64× 64 Bernoulli coded apertures.
Gonzalo R. Arce () Compressive Spectral Image Sensing Oct., 2013 19 / 50
![Page 20: Compressive Spectral Image Sensing, Processing, and Optimization](https://reader034.fdocuments.in/reader034/viewer/2022051514/54859cd9b4af9f7d0d8b4e78/html5/thumbnails/20.jpg)
Performance of coded apertures
Gonzalo R. Arce () Compressive Spectral Image Sensing Oct., 2013 20 / 50
![Page 21: Compressive Spectral Image Sensing, Processing, and Optimization](https://reader034.fdocuments.in/reader034/viewer/2022051514/54859cd9b4af9f7d0d8b4e78/html5/thumbnails/21.jpg)
Original datacube Boolean (40.4dB) Unsigned grayscale (31.2dB)
Binary (27.7dB) Hadamard (27.7dB) Signed grayscale (22.7dB)
Gonzalo R. Arce () Compressive Spectral Image Sensing Oct., 2013 21 / 50
![Page 22: Compressive Spectral Image Sensing, Processing, and Optimization](https://reader034.fdocuments.in/reader034/viewer/2022051514/54859cd9b4af9f7d0d8b4e78/html5/thumbnails/22.jpg)
Original datacube Boolean Unsigned grayscale
Binary Hadamard Signed grayscale
Gonzalo R. Arce () Compressive Spectral Image Sensing Oct., 2013 22 / 50
![Page 23: Compressive Spectral Image Sensing, Processing, and Optimization](https://reader034.fdocuments.in/reader034/viewer/2022051514/54859cd9b4af9f7d0d8b4e78/html5/thumbnails/23.jpg)
Coded Aperture Optimization: Spectral Selectivity
UAV sensor requirements depend on flight duration, range,altitude, etc.Need: Hyperspectral imaging that dynamically adapts to optimalspectral bands.
Gonzalo R. Arce () Compressive Spectral Image Sensing Oct., 2013 23 / 50
![Page 24: Compressive Spectral Image Sensing, Processing, and Optimization](https://reader034.fdocuments.in/reader034/viewer/2022051514/54859cd9b4af9f7d0d8b4e78/html5/thumbnails/24.jpg)
Gonzalo R. Arce () Compressive Spectral Image Sensing Oct., 2013 24 / 50
![Page 25: Compressive Spectral Image Sensing, Processing, and Optimization](https://reader034.fdocuments.in/reader034/viewer/2022051514/54859cd9b4af9f7d0d8b4e78/html5/thumbnails/25.jpg)
((a)) Original ((b)) 12 Random Codes ((c)) 9 Optimal Codes
The resulting spectral data cubes are shown as they would be viewed by a StingrayF-033C CCD Color Camera.
Gonzalo R. Arce () Compressive Spectral Image Sensing Oct., 2013 25 / 50
![Page 26: Compressive Spectral Image Sensing, Processing, and Optimization](https://reader034.fdocuments.in/reader034/viewer/2022051514/54859cd9b4af9f7d0d8b4e78/html5/thumbnails/26.jpg)
((a)) Random Code ((b)) Original ((c)) Optimal Code
((d)) Random Codes ((e)) Optimized Codes
Gonzalo R. Arce () Compressive Spectral Image Sensing Oct., 2013 26 / 50
![Page 27: Compressive Spectral Image Sensing, Processing, and Optimization](https://reader034.fdocuments.in/reader034/viewer/2022051514/54859cd9b4af9f7d0d8b4e78/html5/thumbnails/27.jpg)
((a)) Random ((b)) Optimized
((c)) ((d))
Differences between the original and the reconstructed 3rd spectral channel (479nm)
Gonzalo R. Arce () Compressive Spectral Image Sensing Oct., 2013 27 / 50
![Page 28: Compressive Spectral Image Sensing, Processing, and Optimization](https://reader034.fdocuments.in/reader034/viewer/2022051514/54859cd9b4af9f7d0d8b4e78/html5/thumbnails/28.jpg)
((a)) Original ((b)) 12 Random Codes ((c)) 12 Optimized Codes
The resulting spectral data cubes are shown as they would be viewed by a StingrayF-033C CCD Color Camera.
Gonzalo R. Arce () Compressive Spectral Image Sensing Oct., 2013 28 / 50
![Page 29: Compressive Spectral Image Sensing, Processing, and Optimization](https://reader034.fdocuments.in/reader034/viewer/2022051514/54859cd9b4af9f7d0d8b4e78/html5/thumbnails/29.jpg)
Coded Aperture Optimization: Image Classification
Goal: classification of a spectral scene usingCompressive measurementsOptimal code apertureSparsity signal model
Gonzalo R. Arce () Compressive Spectral Image Sensing Oct., 2013 29 / 50
![Page 30: Compressive Spectral Image Sensing, Processing, and Optimization](https://reader034.fdocuments.in/reader034/viewer/2022051514/54859cd9b4af9f7d0d8b4e78/html5/thumbnails/30.jpg)
Given the FPA measurements g, every test pixel fi belongs to one ofthe P known classes
H(1),H(2), ...,H(P)
0 10 20 30 40 50 60 700
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Spectrum Band
Glutamine
0 10 20 30 40 50 60 700
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Spectrum Band
Histidine
0 10 20 30 40 50 60 700
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Spectrum Band
Isoleucine
If a pixel fi ∈ H(k), its spectral profile lies in a low-dimensionalsubspace spanned by the training samples: s(k)
j j=1,...,Np
fi ≈ [s(k)1 ,s(k)
2 , ...,s(k)Nk
][α(k)1 , α
(k)2 , ..., α
(k)Np
]T = S(k)α(k)
where, α(k) is a sparse vector.Gonzalo R. Arce () Compressive Spectral Image Sensing Oct., 2013 30 / 50
![Page 31: Compressive Spectral Image Sensing, Processing, and Optimization](https://reader034.fdocuments.in/reader034/viewer/2022051514/54859cd9b4af9f7d0d8b4e78/html5/thumbnails/31.jpg)
Sparsity Model
Combining all class sub-dictionaries
fi ≈ [S(1), ...,S(k), ...,S(P)][α(1), ...,α(k), ...,α(P)]T = Sα
Ideally, if fi ∈ H(k), thenα(j) = 0; ∀j = 1, ...,P; j 6= k .
Gonzalo R. Arce () Compressive Spectral Image Sensing Oct., 2013 31 / 50
![Page 32: Compressive Spectral Image Sensing, Processing, and Optimization](https://reader034.fdocuments.in/reader034/viewer/2022051514/54859cd9b4af9f7d0d8b4e78/html5/thumbnails/32.jpg)
Superposition of 3 shots for the different codes
Optimal codes
5 10 15 20 25 30
5
10
15
20
25
30
0
0.5
1
1.5
2
2.5
3
Hadamard Codes
5 10 15 20 25 30
5
10
15
20
25
30
0
0.5
1
1.5
2
2.5
3
Bernoulli codes
5 10 15 20 25 30
5
10
15
20
25
30
0
0.5
1
1.5
2
2.5
3
Ck =∑K−1
k=0 (tkm,n)2 at each (m,n) spatial location.
Optimal Hadamard BernoulliGonzalo R. Arce () Compressive Spectral Image Sensing Oct., 2013 32 / 50
![Page 33: Compressive Spectral Image Sensing, Processing, and Optimization](https://reader034.fdocuments.in/reader034/viewer/2022051514/54859cd9b4af9f7d0d8b4e78/html5/thumbnails/33.jpg)
Image Classification Results
200 spectral bands. 50 shots.Ground-truth Single spectral band SVM-full datacube (73.5%)
Bernoulli-25%datacube(64.3%)
Hadamard-25%datacube(63.4%)
Optimal-25%datacube (73.7%)
Gonzalo R. Arce () Compressive Spectral Image Sensing Oct., 2013 33 / 50
![Page 34: Compressive Spectral Image Sensing, Processing, and Optimization](https://reader034.fdocuments.in/reader034/viewer/2022051514/54859cd9b4af9f7d0d8b4e78/html5/thumbnails/34.jpg)
Image Classification Results
200 spectral bands. 50 shots.Ground-truth FPA measurement SVM-full datacube (73.5%)
Bernoulli-25%datacube(64.3%)
Hadamard-25%datacube(63.4%)
Optimal-25%datacube (73.7%)
Gonzalo R. Arce () Compressive Spectral Image Sensing Oct., 2013 34 / 50
![Page 35: Compressive Spectral Image Sensing, Processing, and Optimization](https://reader034.fdocuments.in/reader034/viewer/2022051514/54859cd9b4af9f7d0d8b4e78/html5/thumbnails/35.jpg)
Colored Coded Aperture Spectral ImagingPatterned coating combines micro-lithography with optical coatingtechnology.
Precision patterned coatingand patternsSub-pixel alignment accuracyUltraviolet, visible, NIR, SWIRMulti-filter arrays on monolithicsubstrates
Gonzalo R. Arce () Compressive Spectral Image Sensing Oct., 2013 35 / 50
![Page 36: Compressive Spectral Image Sensing, Processing, and Optimization](https://reader034.fdocuments.in/reader034/viewer/2022051514/54859cd9b4af9f7d0d8b4e78/html5/thumbnails/36.jpg)
NEW FAMILY OF CODED APERTURES
Boolean Spectrally Selective
Super-resolution Colored
Gonzalo R. Arce () Compressive Spectral Image Sensing Oct., 2013 36 / 50
![Page 37: Compressive Spectral Image Sensing, Processing, and Optimization](https://reader034.fdocuments.in/reader034/viewer/2022051514/54859cd9b4af9f7d0d8b4e78/html5/thumbnails/37.jpg)
Colored coded aperture model
Colored coded aperture is a color filter arrayEach entry is a wavelength selective color filter3D Mask model has the same dimensions than the objective discrete data cube
Gonzalo R. Arce () Compressive Spectral Image Sensing Oct., 2013 37 / 50
![Page 38: Compressive Spectral Image Sensing, Processing, and Optimization](https://reader034.fdocuments.in/reader034/viewer/2022051514/54859cd9b4af9f7d0d8b4e78/html5/thumbnails/38.jpg)
Linear dispersion and focal plane array integrationLinear shifting operation
Focal plane array (FPA) projections
The number of pixels of the FPAdetector is N(N + L− 1) N2L(size of the spectral data cube)
Gonzalo R. Arce () Compressive Spectral Image Sensing Oct., 2013 38 / 50
![Page 39: Compressive Spectral Image Sensing, Processing, and Optimization](https://reader034.fdocuments.in/reader034/viewer/2022051514/54859cd9b4af9f7d0d8b4e78/html5/thumbnails/39.jpg)
Random Boolean Code
Gonzalo R. Arce () Compressive Spectral Image Sensing Oct., 2013 39 / 50
![Page 40: Compressive Spectral Image Sensing, Processing, and Optimization](https://reader034.fdocuments.in/reader034/viewer/2022051514/54859cd9b4af9f7d0d8b4e78/html5/thumbnails/40.jpg)
4 Colors Random Code
Gonzalo R. Arce () Compressive Spectral Image Sensing Oct., 2013 40 / 50
![Page 41: Compressive Spectral Image Sensing, Processing, and Optimization](https://reader034.fdocuments.in/reader034/viewer/2022051514/54859cd9b4af9f7d0d8b4e78/html5/thumbnails/41.jpg)
Restricted Isometry Property of Colored CASSIA = HΨΨΨ, fff = ΨΨΨθθθ, ΨΨΨ = W⊗ΨΨΨ2D
Definition
(1− δs) ||θθθ||22 ≤ ||Aθθθ||22 ≤ (1 + δs) ||θθθ||22,
δs = maxT ⊂[N2L],|T |≤S
||AT|T |A|T | − I||22,
A|T |, |T | columns of A indexed by the set T
δs = maxT ⊂[N2L],|T |≤S
λmax(A|T ||T | − I
)(A|T |
)ir
= hiψψψΩr
=
L−1∑k=0
(t`ik
)mi−kN
Ψmi +k(N′),Ωr
(hi )j =
(
t`ikj
)i−`i V−kj N
, if i − `i V = j − kj N′
0, otherwise,
A = HΨΨΨ2D
A = H(W⊗ΨΨΨ2D)
Gonzalo R. Arce () Compressive Spectral Image Sensing Oct., 2013 41 / 50
![Page 42: Compressive Spectral Image Sensing, Processing, and Optimization](https://reader034.fdocuments.in/reader034/viewer/2022051514/54859cd9b4af9f7d0d8b4e78/html5/thumbnails/42.jpg)
A|T ||T | = AT|T |A|T |
(A|T ||T |
)r ,u =
K−1∑`=0
V−1∑i=0
L−1∑k1=0
L−1∑k2=0
(t`k1
)i−k1N
(t`k2
)i−k2N
Ψi+k1N′,ΩrΨi+k2N′,Ωu
Gonzalo R. Arce () Compressive Spectral Image Sensing Oct., 2013 42 / 50
![Page 43: Compressive Spectral Image Sensing, Processing, and Optimization](https://reader034.fdocuments.in/reader034/viewer/2022051514/54859cd9b4af9f7d0d8b4e78/html5/thumbnails/43.jpg)
Results: LH-Colored Coded Aperture
A geometric interpretation of the colored coded apertures for LH-Colored filters (3shots).
Gonzalo R. Arce () Compressive Spectral Image Sensing Oct., 2013 43 / 50
![Page 44: Compressive Spectral Image Sensing, Processing, and Optimization](https://reader034.fdocuments.in/reader034/viewer/2022051514/54859cd9b4af9f7d0d8b4e78/html5/thumbnails/44.jpg)
Code Design Results B-Colored Coded Apertures (Band Pass Filters)
Geometric interpretation of colored coded apertures for B-filters (3 shots)
Gonzalo R. Arce () Compressive Spectral Image Sensing Oct., 2013 44 / 50
![Page 45: Compressive Spectral Image Sensing, Processing, and Optimization](https://reader034.fdocuments.in/reader034/viewer/2022051514/54859cd9b4af9f7d0d8b4e78/html5/thumbnails/45.jpg)
461nm 470nm 479nm 488nm
497nm 506nm 515nm 524nm
533nm 542nm 551nm 560nm
569nm 578nm 587nm 596nm
16 channels461-596nm256× 256pixels
Gonzalo R. Arce () Compressive Spectral Image Sensing Oct., 2013 45 / 50
![Page 46: Compressive Spectral Image Sensing, Processing, and Optimization](https://reader034.fdocuments.in/reader034/viewer/2022051514/54859cd9b4af9f7d0d8b4e78/html5/thumbnails/46.jpg)
Reconstruction From B-Colored Coded Apertures
2 4 6 8 10 12 14
30
40
50
60
70
80
90
Number of Shots
PSNR
Block Unblock
Random B-colored
B-colored
Mean PSNR of the reconstructed data cubes.
Gonzalo R. Arce () Compressive Spectral Image Sensing Oct., 2013 46 / 50
![Page 47: Compressive Spectral Image Sensing, Processing, and Optimization](https://reader034.fdocuments.in/reader034/viewer/2022051514/54859cd9b4af9f7d0d8b4e78/html5/thumbnails/47.jpg)
Reconstructions from Real Measurements (RGB)
K = 1 K = 1
Sh=1
(i) 1 snapshot: Photomask, Coloring, Optimal
K = 4 K = 4
Sh=4
(j) 4 snapshots: Photoask, Coloring, Optimal
Gonzalo R. Arce () Compressive Spectral Image Sensing Oct., 2013 47 / 50
![Page 48: Compressive Spectral Image Sensing, Processing, and Optimization](https://reader034.fdocuments.in/reader034/viewer/2022051514/54859cd9b4af9f7d0d8b4e78/html5/thumbnails/48.jpg)
Reconstruction Results (Photomask vs ColoredCoded Apertures)
454 nm K = 1 468 nm K = 1 485 nm K = 1 506 nm K = 1 529 nm K = 1 557 nm K = 1 594 nm K = 1 639 nm K = 1
454 nm K = 1 468 nm K = 1 485 nm K = 1 506 nm K = 1 529 nm K = 1 557 nm K = 1 594 nm K = 1 639 nm K = 1
(a) One snapshot reconstruction. (First row) Photomask, (Second row) Coloring
454 nm K = 4 468 nm K = 4 485 nm K = 4 506 nm K = 4 529 nm K = 4 557 nm K = 4 594 nm K = 4 639 nm K = 4
454 nm K = 4 468 nm K = 4 485 nm K = 4 506 nm K = 4 529 nm K = 4 557 nm K = 4 594 nm K = 4 639 nm K = 4
(b) 4 snapshots reconstructions. (First row) Photomask, (Second row) Coloring
Gonzalo R. Arce () Compressive Spectral Image Sensing Oct., 2013 48 / 50
![Page 49: Compressive Spectral Image Sensing, Processing, and Optimization](https://reader034.fdocuments.in/reader034/viewer/2022051514/54859cd9b4af9f7d0d8b4e78/html5/thumbnails/49.jpg)
Spectral Reconstruction From Colored Coded Apertures
Sh=4
P1
P2
454 468 485 506 529 557 594 6390
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Nanometers
Norm
. In
tensity
Spectral Comparison: 4 Snapshots
Spectrometer
Photomask
Coloring
Optimal
454 468 485 506 529 557 594 6390
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Nanometers
Norm
. In
tensity
Spectral Comparison: 4 Snapshots
Spectrometer
Photomask
Coloring
Optimal
Pixel P1
Pixel P2
Gonzalo R. Arce () Compressive Spectral Image Sensing Oct., 2013 49 / 50
![Page 50: Compressive Spectral Image Sensing, Processing, and Optimization](https://reader034.fdocuments.in/reader034/viewer/2022051514/54859cd9b4af9f7d0d8b4e78/html5/thumbnails/50.jpg)
Conclusions
Optical Coding for Compressive Spectral ImagingGood codes for reconstruction, classification, unmixingColored codes offer multidimensional coding - open problem
Convolution Optical Coding (Projections)Light field imagingX-ray tomographySuper-resolution microscopy
Gonzalo R. Arce () Compressive Spectral Image Sensing Oct., 2013 50 / 50