Motivation

 Necessity of combining physical models and statistical techniques in unmixing

 Methods of integration of scientific models & statistical algorithms not always obvious –  Enable machine learning techniques to produce scientifically

meaningful results in unsupervised settings

 Meaningful information not always readily accessible or easily readable –  Representation = Simplification

Outline

 Background –  Mineral spectral signatures in VNIR

 Meaningful features for mineral identification  Data challenges for planetary data  Data processing pipeline  Validation: expert assessment  Comparison with state-of-the-art  Conclusions and future work

Mineral spectral signatures   Each mineral has

a distinct spectral shape (signature)

  Discriminative information mostly in absorption band positions and shapes

  Difference can be subtle

  Can create parameters for discrimination

Mineral spectral signatures   Each mineral has

a distinct spectral shape (signature)

  Discriminative information mostly in absorption band positions and shapes

  Difference can be subtle

  Can create parameters for discrimination

Spectral features for minerals

  Use splines   Select knots so that

Spectral features for minerals

  Use splines   Select knots so that

–  Reconstruction insensitive to artifacts

Spectral features for minerals

  Use splines   Select knots so that

–  Reconstruction insensitive to artifacts

–  Reconstruction with higher sensitivity in diagnostic areas

Spectral features for minerals

  Use splines   Select knots so that

–  Reconstruction insensitive to artifacts

–  Reconstruction with higher sensitivity in diagnostic areas

–  Reconstruction sharper (green) for vibrational bands and smoother (red) for electronic transition bands

  B-spline coefficients as feature vector

hyperspectral data challenges

  Cloud has curved boundaries and is non convex

  Some dimensions uninformative

  No apparent clusters, high density

  Noise creates outliers   Most unique spectra =

extreme points or “corners” or “image endmembers”

Objectives for spectral unmixing

 Separation of spectral pixels in families (image segmentation)

 Sensitivity to subtle changes in spectral absorption positions and shapes (mineral sub-families)

 Sensitivity to small (spatial) outcrops  Robustness with respect to noise  Useful visualization

Pipeline

Dimensionality reduction

Clustering

Unmixing

Pruning

43

1

2

1 2

4

3

1 2

4 3

5

1 2

34

1 2 21

5 1 1

Preprocessing

Pipeline

Dimensionality reduction

Clustering

Unmixing

Pruning

43

1

2

1 2

4

3

1 2

4 3

5

1 2

34

1 2 21

5 1 1

Preprocessing

Operation modes

  Two capabilities: –  Select areas based on parameter maps (user version) –  Divide the image in sections (pipeline version)

  Operate on each area independently

50 pixels

100 pixels

Pipeline

Dimensionality reduction

Clustering

Unmixing

Pruning

43

1

2

1 2

4

3

1 2

4 3

5

1 2

34

1 2 21

5 1 1

Preprocessing

Dimensionality reduction: issues

  Intrinsic dimensionality of data is low: benefit from dimensionality reduction.

  From movie: –  Need nonlinear transform. –  Need to preserve local geometry –  Need to highlight natural clusters

  Reduce dimensionality to 2 – 3 for visualization

Dimensionality reduction High-D Feature Space Low-D Space

x1x2

x3y3

y2y1

ynxn

x1, . . . , xn (known) as vertices of a graph

y1, . . . , yn (unknown) as vertices of a graph

Edge weights proportional to spectral dissimilarities and spatial adjacency

Edge weights fixed

Dimensionality reduction High-D Feature Space Low-D Space

x1x2

x3y3

y2y1

ynxn

p12 q12

Small distance = small distance

Dimensionality reduction High-D Feature Space Low-D Space

x1x2

x3y3

y2y1

ynxn

Med/big distance = bigger distance

q13p13

Dimensionality reduction High-D Feature Space Low-D Space

x1x2

x3y3

y2y1

ynxn

P = {pij} Q = {qij}

Correl. Neighbor Embedding (Parente 2011)

  Minimize relative entropy

  Solve by gradient descent

  Variation on t-Stochastic Neighbor Embedding (Van der Maaten et al. 2008)

High-D Space Low-D Space

dij = 1! cij

cij = !ij,spatial · cij,spect

pij =exp(!d2

ij/2!2i )!

k !=l exp(!d2kl/2!

2k )

D = argminyi,yj

!

i,j

pij(xi, xj) logpij(xi, xj)

qij(yi, yj)

!D(P ||Q)

!yi= 4

!

j

"ij(yi ! yj)(pij ! qij)

Pipeline

Dimensionality reduction

Clustering

Unmixing

Pruning

43

1

2

1 2

4

3

1 2

4 3

5

1 2

34

1 2 21

5 1 1

Preprocessing

Graph partitioning as clustering

  Cluster points in the transformed space to take advantage of separated sections

  The geometry is nonlinear: need clustering on curved structure

  Consider the set of vertices of the graph and the edge weights

  Clustering is equivalent to partitioning graph into disjoint subsets. –  can be done by spectral clustering

because CNE creates several connected components

yi

qij(yi, yj)

Image segmentation

Original image

Segmentation map Clustering = mineral

family mapping = image segmentation

Pipeline

Dimensionality reduction

Clustering

Unmixing

Pruning

43

1

2

1 2

4

3

1 2

4 3

5

1 2

34

1 2 21

5 1 1

Preprocessing

Local endmember detection

  The clusters in the original space are roughly convex   Locally to a cluster can assume linear mixing   approximate the data cloud with a conic or convex

combination of a small number of “endmembers”   Have a way to extrapolate endmembers if the data

does not support clear detections

1 2

4

3

Robust Nonneg. Matrix Factorization

  k is the number of local endmembers   is a robust estimator   D imposes smoothness and corrects MNF “problems”   Solve with alternating projected gradient   Zymnis 2009, Parente 2009, Parente 2011

!

minimize !(Y !WH) + "||DW ||2Fsubject to H " 0, 1T H = 1T

W ! Rm!k, H ! Rk!n

W,

Pipeline

Dimensionality reduction

Clustering

Unmixing

Pruning

43

1

2

1 2

4

3

1 2

4 3

5

1 2

34

1 2 21

5 1 1

Preprocessing

Spectral pruning

 Spectrum 1 is any local endmember candidate

 Spectrum 2 is either a local endmember or an estimate of the baricenter of the cloud

  If the score is higher than a threshold Spectrum 1 is pruned

Cross-correlate feature vectors

Score

Features for spectrum 1

Features for spectrum 2

Validation

 Martian image analysis lacks ground truth  Simulation of the complete hyperspectral image

formation process (Parente et al. 2010) –  Soil mixing, atmosphere, instrument response, noise

 Comparison with manual expert assessment –  the expert can extract the complete spectral variability (3E12) –  the expert can only extract partial spectral variability (94F6) –  The expert cannot extract spectral variability

 Self-consistency: comparison with state-of the art (partial)

Validation: 3E12

 Different mineral families evident from RGB

 Low noise  Good spectral

variability

Validation: 3E12

Automatically retrieved spectra over the whole scene

Manually selected spectra over the whole scene

Validation: 94F6

R=band 233, G=band 78, B=band 13 R=D2300, G=OLINDEX, B=BD2210

94F6 manual retrieval

Regions of Interest (ROIʼs) Spectra from ROIʼs

94F6

  Several more spectral families detected by the algorithm

  Letʼs zoom in!

94F6 automated

94F6 automated

 2.205 µm

94F6 automated

 2.205 µm  2.2913 µm

94F6 automated

 2.205 µm  2.2913 µm  2.3046 µm

94F6 automated

 2.205 µm  2.2913 µm  2.3046 µm  2.3244 µm

94F6 automated

 2.205 µm  2.2913 µm  2.3046 µm  2.3244 µm  2.4038 µm

94F6 automated

 2.205 µm  2.2913 µm  2.3046 µm  2.3244 µm  2.4038 µm  2.5229 µm

94F6 automated

 2.205 µm  2.2913 µm  2.3046 µm  2.3244 µm  2.4038 µm  2.5229 µm  2.5295 µm

94F6 automated

 2.205 µm  2.2913 µm  2.3046 µm  2.3244 µm  2.4038 µm  2.5229 µm  2.5295 µm

Carbonate !!

Some difficult data:199C7

199C7 automated

199C7 automated

 2.04 µm  2.29 µm, 2.30 µm,

2.31 µm  2.52 µm, 2.53 µm

Comparison with state of the art

  Current unmixing algorithms: –  require convexity – developed for earth

 environmental conditions are known  ground truth is available

– donʼt consider impulsive noise – some require linear assumptions

  Nonlinear unmixing not yet mature   Not able to discriminate subtle spectral differences

Comparison with other algorithms

The proposed algorithm is insensitive to noise and picks up more surface components

The SMACC algorithm is extremely sensitive to noise

B141 Mawrth Vallis

ENVI SMACC endmembers

Proposed approach

ABCB: Nili Fossae

Endmembers from VCA Proposed

approach

More algorithms

(a) Proposed Algorithm (b) N-FINDR (c) PPI

(d) SMACC (e) SISAL

Conclusions

  Presented a novel method for unmixing   The algorithm effectively captures the image spectral

variability, down to subtle differences, is robust to noise and outperforms current state-of-the-art algorithms

  Can be applied to any hyperspectral dataset   Produces segmentation and endmember maps   We proposed this technique to the CRISM and M3 teams

as the “official” data summarization tool for their processing pipelines.

Future work

  Include a physical unmixing layer: use radiative transfer theory

 Provide mechanism to tag “virtual” endmembers  Complete validation process with expert feedback

References

  L. van deer Maaten and G. Hinton, (2008). Visualizing data using t-SNE, Journal of Machine Learning, 9, pp. 2579-2605.

  A. Ng, M. Jordan and Y. Weiss, (2001). On spectral clustering: Analysis and an algorithm, NIPS.

  M. Parente , J.T. Clark, A. Brown and J.L. Bishop (2010). End-to-end simulation of the image generation process for CRISM spectrometer data, IEEE Transactions on Geoscience and Remote Sensing.

  M. Parente, (2011). Summarization of hyperspectral images: application to Mars, IEEE Transactions on Geoscience and Remote Sensing, (in review).

  M. Parente, J. L. Bishop and J. F. Bell III, (2009), Spectral unmixing and anomaly detection for mineral identification in Pancam images of Gusev soils, Icarus, Vol 203, N. 2, p. 421-436.

Questions?

Publications based on project    Parente M. and A. Plaza (2010), Survey of geometric and statistical unmixing algorithms for

hyperspectral images, IEEE 2nd WHISPERS (Workshop on hyperspectral image and signal processing: evolution of remote sensing) Conf. June 14-16, Reykjavyk, Iceland (invited keynote presentation for special session on “Geometric vs. statistical unmixing algorithms”).

  M. Parente Spectral unmixing using nonnegative basis learning: comparison of geometrical and statistical endmember extraction algorithms. (invited paper) Space Exploration Technologies, edited by Wolfgang Fink Proc. of SPIE Vol. 6960, 69600P, (2008). doi: 10.1117/12.777895

  M. Parente Exploratory data analysis of planetary datasets – new development, (invited talk) Jet Propulsion Laboratory, Pasadena CA, December 4 2008.

  Parente M., Clark J.T., Brown A.J., and Bishop J.L.. (2009). Simulation of the image generation process for CRISM spectrometer data. IEEE WHISPERS (Workshop on hyperspectral image and signal processing: evolution of remote sensing) Conf. Aug 26-28 Grenoble, France. (Best paper award)

  Bishop J. L., Noe Dobrea E. Z., McKeown N. K., Parente M., Ehlmann B. L., Michalski J. R., Milliken R. E., Poulet F., Swayze G. A., Mustard J. F., Murchie S. L., and Bibring J.-., P. (2008) Phyllosilicate diversity and past aqueous activity revealed at Mawrth Vallis, Mars. Science 321, DOI: 10.1126/science.1159699, pp. 830-833.

  Parente, M. and J.L. Bishop, (2010). Extracting endmember spectra from CRISM images: comparison of new Direx image transform technique with MNF, Lunar Planet Science Conf, XLI abstr. #2633.

Backup slides

MRO-CRISM: VNIR Spectra Can Characterize Small Deposits on Mars

Examples of surface features at different CRISM spatial resolutions

• Global Mode: 70 channels • Targeted Mode: 544 channels

OMEGA (300-1000 m/pixel, 13 nm/ch.)

discovers large deposits

CRISM multispectral survey (100-200 m/pix, 70 ch.) discovers small

deposits

CRISM targeted hyperspectral (15-38 m/pixel, 6.55 nm/ch)

characterizes deposits

CRISM Noise sources

  Both artifacts create spikes in the spectral domain

1.  Vertical striping due to miscalibration of pixel sensors (red arrows).

2.  Pixels with elevated bias or abnormal dark ("bad" pixels) create stripe segments (cyan)

60/40

Noise removal with CIRRUS

  CIRRUS (CRISM Iterative Recognition and Removal of Unwanted Spiking) (Parente 2008)

  CIRRUS currently in use in CRISM processing pipeline

Original Cleaned

Original

Cleaned

Comparison with PCA

Proposed approach (3D) PCA (first 3 PCs)

  Natural clusters well separated

  Between-clusters, different spectra

  Within-cluster, similar spectra

  Natural clusters not evident

  similar points can differ in norm

  1st PC illumination gradient

Comparison with other techniques

Proposed approach (3D) PCA (first 3 PCs)

  Natural clusters well separated

  Between-clusters, different spectra

  Within-cluster, similar spectra

  Natural clusters not evident

  Some endmembers evident

  Clustering particularly hard

LLE (3D)

  Natural clusters not evident

  similar points can differ in norm

  1st PC illumination gradient

Graph partitioning as clustering

Graph partitioning as clustering

Graph partitioning as clustering

Graph partitioning as clustering

Clustering for case study

Clustering performance comparison

Original image

Proposed approach

K-means in original space

Hierarchical in original space

Hierarchical in 3-D space

K-means with correlation in original space

K-Eigenvector Clustering 1.  Construct matrix of normalized weights Aʼ 2.  Decomposition: Find the eigenvectors of Aʼ

corresponding to the k largest eigenvalues. These form the the columns of the new matrix X.

3.  Form the matrix Y –  Renormalize each of Xʼs rows to have unit length –  Y | –  Treat each row of Y as a point in

3.  Cluster into k clusters via k-means 4.  Final Cluster Assignment

–  Assign point to cluster j iff row i of Y was assigned to cluster j

k can be found by maximum spread between eigenvalues

(Ng et al. 2001)

Validation   This software is undergoing extensive validation

aimed at confirming that the proposed method can be used pervasively and reliably in the summarization of the whole CRISM database.

  The validation process starts with requesting from the community image IDʼs with manually selected endmembers.

  An automated pipeline is in place that sends back via email the spectra retrieved by the algorithm to each author of manual analysis.

  Upon receiving feedback on dissimilarities and quality of the detections the pipeline will calculate validation statistics and will send them to the team for review.

  After validation the production stage will begin.

ID Solicitation

Processing

Feedback

Validation statistics