Radiometric Characterization Projector (HIP) Projector validation tool, SI traceable, for in-car...

Optical Technology Division Hyperspectral Image Projector (HIP) 19Sept2011 NewRad

Radiometric Characterization

of a Hyperspectral Image Projector (HIP)

Joseph Rice*, Stephen Maxwell, Howard Yoon, and Steve Brown

Optical Technology Division

National Institute of Standards and Technology

Gaithersburg, MD 20899 USA

This presentation includes contributions from many NIST and non-NIST

collaborators, including:

Mike Kehoe, Casey Dodge, Casey Smith, Rand Swanson, Resonon (Design/Build of

the first non-prototype HIP):

Jorge Neira, Allan Smith, David Allen, Bob Saunders Resonon (Software

development, performance characterization, design applications):

James Goodman, University of Puerto Rico

Edward Livingston & Karel Zuzak, U. Texas Southwestern Medical Center

*[email protected]


Introduction Motivation:

Scene Projector validation tool, SI traceable, for in-car police video cameras

Scene projector for Fire-fighter Sensor Evaluation

Scene Projector performance validation artifact for military hardware

Scene Projector for Quantitative Optical Medical Imaging

Scene Projector for Multi and Hyperspectral Imaging/Earth-remote Sensing

General solution:

The Hyperspectral Image Projector (HIP) – A 2D scene projector where every pixel has a programmable spectrum

A few optical technologies introduced along the way

• Micromirror (Digital Micromirror Device – DMD) arrays,

• Liquid Crystal on Silicon (LCOS) arrays,

• Supercontinuum sources


Outline

• Introduction to the concept of a Hyperspectral

Image Projector (HIP)

• Show what a realized HIP looks like

• Show some example scenes

– San Diego Naval Air Station

– Enrique Reef in Puerto Rico

– Medical scene of liver/bile duct

• Future Directions


Ac

Hyperspectral Image Projector (HIP)

• HIP projects 2-d hyperspectral images

– Complex spatial scenes with use-defined spectral

content

• A source analog to Hyperspectral Image Sensors

Hyperspectral Data Cube

Acquire a Data Set

Project the data cube to the

Instrument Under Test

& the Reference Instrument

'Reality TV' Projector where user-definable,

SI traceable spectral distributions are projected

in each pixel.


5

HIP Basic Concept

HIP replaces the color filter

wheel in a conventional DLP

projection system.

1. Enable user-defined

eigenspectra

2. Change the number of

different spectra from 3 to

an arbitrary, user-defined,

number

Red Green Blue

Time

Pixel k

Pixel j

Pixel i

Pixel l

One full rotation of the color wheel

Grey Scale: Time-Division Multiplexing

Time-Division Multiplexing: Digital Micromirror Device (DMD)


Digital Micromirror Devices (DMD’s)

• An array of (Micro-Electro-Mechanical

System) MEMS micromirror elements

• Developed by Texas Instruments

– 1024 x 768 elements, +/- 12 degree tilt

angle

– Aluminum mirrors

– 13.68 micron pitch

– < 24 microseconds mechanical

switching time

• Two nice features

– Mirrors don’t fail

– Control software has been developed

Mirror -12 degrees Mirror +12 degrees

Two Pixels from the DMD Mirror Array:

Liquid crystal technology is also being investigated for the HIP

by Boulder Nonlinear Systems, Inc. under SBIR programs


Overview of the NIST HIP

Spectrally-

Programmable Source

Reference Instrument Broadband Fiber

Light Source

Spatial Engine

(Image Projector) Unit Under Test

Computer

DMD1 DMD2

1. More than 3

Eigenspectra

2. These functions

now user-defined

Benefits of the

Spectrally

Programmable

Source


Principle of the Spectrally Programmable Source Double subtractive spectrometer

Spatial Light Modulator (SLM)

Grating

Mirror

Mirror

Mirror

Mirror

Grating

Wavelength

Inte

nsi

ty

Input

Output


How the DMD is used in the Spectral Engine:

450 500 550 600 650 700

Wavelength [nm]

Inte

nsi

ty [

a. u.]

Near IRUV Visible

1024 mirrors

768

mir

rors

Wavelength

Inte

nsi

ty

mirror array

Wavelength and Intensity Selection with a Digital Mirror Device (note mirror # reduced by a factor of 10 for clarity)

Mirrors in "On"Position

Near IRUV Visible

Digital Mirror Device as a Spectral Light Engine (note mirror # reduced by a factor of 10 for clarity)

1024 mirrors

768

mir

rors

Wavelength

Inte

nsi

ty

mirror array

Monochromator Mode

Broad-band Source Mode

0.0

0.50

1.0

1.5

2.0

2.5

400 500 600 700 800 900 1000 1100

HIP Irradiance

Sp

ectr

al Ir

radia

nce (

W m

-2 n

m-1

)

Wavelength (nm)

TOA Solar

ASTM AM1.5


Spectral Matching Examples

0.0

0.50

1.0

1.5

2.0

2.5

400 500 600 700 800 900 1000 1100

HIP Irradiance

Sp

ectr

al Ir

rad

ian

ce

(W

m-2

nm

-1)

Wavelength (nm)

TOA Solar

ASTM AM1.5

0.0

0.50

1.0

1.5

2.0

2.5

400 500 600 700 800 900 1000 1100

HIP Irradiance

Sp

ectr

al Ir

rad

ian

ce

(W

m-2

nm

-1)

Wavelength (nm)

TOA Solar

ASTM AM1.5


Spectral Matching Examples:

0

100

200

300

400

500

600

400 500 600 700 800 900 1000 1100

OLI Matching Absolute

Sp

ectr

al R

adia

nce

(W

/m2sr

um

)

Wavelength (nm)

TOA Solar

Bare Desert Soil

Vegetation

HIP VNIR Limits

Measured from HIP Prototype VNIR Spectral Engine 11/3/10

Modeled Spatial Engine: XGA DMD f/3 with 20% Transmittance


Source: Laser-pumped Photonic Crystal Fiber

• Utilizes non-linear effects in a photonic crystal optical fiber to greatly broaden the spectrum of a 1064 nm pump laser.

• Broadband light is generated in a single-mode (5 um core diameter) photonic crystal (holey) optical fiber

– No etendue issues as with lamps or blackbodies.

– Ideally suited for coupling to a spectral engine.

– High radiance, not high power

• High power and high spectral resolution:

– 3mW/nm spectral power density from 450 nm to 1700 nm

• Commercially available.

0.0

0.2

0.4

0.6

0.8

1.0

1.2

1.4

1.6

1.8

2.0

500 1000 1500 2000 2500

SC450 ASD

Powe

r (ar

bitra

ry u

nits

)

Wavelength (nm)


Supercontinuum Source Stability

Fianium 4W SC Source

(100 scans averaged - ~ 60 s)

0

0.2

0.4

0.6

0.8

1

400 900 1400 1900 2400

Wavelength [nm]

% S

tDev

Mean

Longer term: 60 minutes Short term: 60 seconds

-5

-4

-3

-2

-1

0

1

2

3

600 800 1000 1200 1400 1600

HIP stability diff

15 min diff30 min diff45 min diff60 min diff

Radia

nce D

iffe

rence (

%)

Wavelength (nm)


NIST/Resonon Prototype HIP VNIR-SWIR Spectral Engine

Inside view from back: Outside view from front:

Supercontinuum Light Source

VNIR

Spectral

Engine

SWIR

Spectral

Engine


HIP VNIR-SWIR Spectral Engine

Dichroic

Beamsplitter

Supercontinuum

Light Source

VNIR

Spectral

Engine

SWIR

Spectral

Engine

Source

Collimator

Electronics

Section

Output

} }

View from back: Dichroic

Beamcombiner


VNIR-SWIR Spatial Engine

Optical Design

Sensor


VNIR-SWIR Spatial Engine

Mechanical Design

DMD

DMD Driver Electronics PC Boards Cooling Fan

Replaceable

Collimator

Purged, Sealed

Optics Section Beam Pipe

For Direct Coupling

For Spectral Engine

Electrical

Connections

To Spectral

Engine


NIST/Resonon VNIR-SWIR HIP Prototype System

VNIR

Spectral

Engine

SWIR

Spectral

Engine

Spatial

Engine

Collimator For Projection

to UUT

Super-

continuum

Source

Power Supply


Liquid Light Guide:

•Couples HIP Spectral Engine

to HIP Spatial Engine

HIP

Spectral

Engine

(rear view)

HIP

Spatial

Engine

Translation

Stage

NIST/Resonon VNIR-SWIR HIP Prototype System


Reference Instrument PIXIS camera with a liquid crystal tunable filter


Hyperspectral Image Data Cube AVIRIS Image Cube of the San Diego Naval Air Station


Input

Image Cube

Example: AVIRIS Image Cube

of San Diego Naval Air Station

Software such as ENVI/SMACC is used to find the

Eigenspectra and their Abundances

J. Gruninger, A. J. Ratkowski, and M. L. Hoke, “The sequential maximum angle

convex cone (SMACC) endmember model,” Proc. SPIE 5425, 1-14 (2004).

EA-3

EA-4

EA-2 EA-5

EA-6

Eigenspectra Abundances (EA)

EA-1

ES-1

Eigenspectra (ES)

ES-2

ES-3

ES-5

ES-4

ES-6

ES-7

Compressive Projection is Used to Achieve Higher Brightness

Then we need only project N = 6 broadband spectra

instead of M = 30+ monochromatic spectra.


Enrique Reef, Puerto Rico James Goodman, University of Puerto Rico

David Allen, NIST


Enrique Reef Decomposition

0

0.02

0.04

0.06

0.08

0.1

0.12

0.14

0.16

0.18

0.2

450 500 550 600 650 700

Rel

ativ

e In

tesi

ty [

arb

itra

ry u

nit

s]

Wavelength [nm]

EM1

EM2

EM3

EM4

EM5

EM6

EM1

EM3

EM5 EM6

EM4

EM2

Eigenspectra Abundance Images


Original Image

Re-created Image


Gall Bladder, Liver, Skin Image Drs. Edward Livingston & Karel Zuzak

University of Texas Southwestern Medical Center

Maritoni Litorja, NIST

Liver

Gall Bladder Skin

Surgical

Dressing


Image Decomposition Into Endmember Spectra and Abundance Images

Abundance Images

Spectrum 2 Spectrum 3 Spectrum 4 Spectrum 5 Spectrum 1

Original RGB image


Image Decomposition - Visible Classification according to components spectra

Skin

(bloody)

Gall

bladder

Cystic

duct

Liver

Original RGB

image


skin

gall

bladder

duct

liver

Image Decomposition - Near IR Classification according to components spectra

Original NIR image

at 800 nm


Future Directions

• Working with NOAA exploring Liquid

Crystal on silicon (LCOS) array-based

spectral light engines

• Working with NASA to include polarization

• Extending range into UV and further into

the IR

– Currently looking at the 3 to 5 um range

– Plans to consider extending capabilities to the

8mm to 12 mm range


Hyperspectral Image Projector (HIP) Being developed as a scene projector for testing spectral/imaging sensors

• Projects a 2D image with programmable spectra at each pixel

• Unique applications in testing sensors with realistic spectra and scenes

• Using a 4 W supercontinuum source, the HIP is capable of 2 nm spectral resolution

over the VNIR and 5 nm over the SWIR, providing spectral, spatial, and radiometric

fidelity for simulating solar-illuminated Earth scenes.

• Used for testing Laboratory for Atmospheric and Space Physics (LASP) HSI

prototype in May 2011

• Currently being used for testing NASA Ocean Radiometer for Carbon Assessment

(ORCA) prototype

Summary

This work was funded in part by

the U.S. Department of Defense Test Resource Management Center (TRMC)

Test and Evaluation / Science and Technology (T&E/S&T) Multi-Spectral Test (MST) Program,

by the NIST Office of Law Enforcement Standards, and

by the NIST Optical Medical Imaging IMS project

For more information: Contact Joe Rice ([email protected])

Radiometric Characterization Projector (HIP) Projector validation tool, SI traceable, for in-car...

Documents

Transcript of Radiometric Characterization Projector (HIP) Projector validation tool, SI traceable, for in-car...