Hyperspectral Camera Design Project - EDGEedge.rit.edu/content/P06522/public/Hyperspectral...

definition • design • development D3Engineering

Hyperspectral Camera Design Project


Team Members

• Will Shaffer– Electrical Engineering

• Jeff Sidoni– Electrical Engineering

• Dan Scorse– Mechanical Engineering

• Jordan Gartenhaus– Mechanical Engineering

• Sponsored by:D3 Engineering222 Andrews StreetRochester, NY 14604


Hyperspectral Overview

• Imaging system capable of capturing several tens to several hundreds of narrow, contiguous bands of the electromagnetic spectrum.

• Covers visible spectrum, extends from NIR to long wave IR.

• Data sets are large, but allow for the detection of subtle differences overlooked by traditional and multispectral systems.


Spectral Reflectance

• Defined as the ratio of reflected energy to incident energy as a function of wavelength

• Materials have a unique spectral signature that is used for identification after a data set is acquired.

0 5 10 15-10

0

10

20

30

40

50

60

70

Wavelength λ (μm)

% R

efle

ctan

ce

AsphaltDry GrassGraniteGrassDark Soil


Data Acquisition

• Hyperspectral images are obtained using an imaging spectrometer; a device used to measure various properties of light over the spectrum.

• Typically contains an optical system, a dispersing element, possibly in the form or a prism or grating and an array of detectors.


Linear Array Spectrometer

• Uses a single row of detectors. Allows for the spectrum of a given point in a target to be acquired.

• Scanning mirror adds a dimension to capture an entire row of thetarget.

• Second dimension is typically accomplished by the movement of a plane or satellite.


2D Sensor Method

• Using a two dimensional sensor, an entire row can be captured in a single frame.

• Scanning mirror adds second dimension.• In remote sensing applications where the camera is in a plane,

data sets can be captured without the need of a scanning mirror.


Data Organization

• Hyperspectral data sets, by nature, are very large

• Usually consist of several hundred bands of the spectrum.

• Organized in a cube where two of the axes are spatial and one is spectral.


Identification of Materials

• Unique spectral signature for materials

• Many libraries available containing hundreds or thousands of signatures for natural and man-made materials

• Research and evaluation of algorithms to improve classification and identification


Applications

• Remote Sensing Applications– Military

• Terrain Mapping• Target Detection

– Agricultural• Crop Yield Prediction• Soil Quality Evaluation

– Mineral Exploration– Environmental Research

• Close Range Applications– Medical

• Early cancer detection

– Food Industry• Bacteria detection and

illness prevention

– Counterfeit Currency


Defense Applications

• Used for target identification in combat areas• Target below is a camouflaged tank, hidden to traditional and

even multispectral imaging systems.


Project Outline

• Purpose is to design and build a functional hyperspectral camera system for use by D3 Engineering’s design staff.


Project Requirements

• Prototype a Hyperspectral Camera that– Uses a single CMOS or CCD imager– Captures spectra from ~400 to ~850nm (Visible Light)– Has spectral resolution of 25-50nm– Develop back end software to acquire and process a

hyperspectral data set and extract basic information such as material identification

– Utilizes hardware already developed and tested by D3 Engineering for the image acquisition and motor control for the scanning mirror


System Block Diagram

Processing Board

Motion Board

Imager Board

Image Sensor DSPVideoData

USB

Communication

I2C Serial Communications

Motion Control

SDRAM

FLASH

5VDC

Mirror

Target


Project TimelineID Task Name Duration Start Finish

1 Hyperspectral Camera Design 124 days Mon 12/5/05 Thu 5/25/062 Requirements Generation 2 w ks Mon 12/5/05 Fri 12/16/053 Design Documentation 2 w ks Mon 12/19/05 Fri 12/30/054 Initial Prototyping w / CDK 2 w ks Mon 12/19/05 Fri 12/30/055 Working Camera 2 w ks Mon 1/2/06 Fri 1/13/066 Preliminary Design Review 1 day Fri 2/24/06 Fri 2/24/067 CDK 61 days Mon 1/2/06 Mon 3/27/068 Camera Softw are Development 9.2 w ks Mon 1/2/06 Mon 3/6/069 1Mpixel 1 w k Tue 3/7/06 Mon 3/13/06

10 Frequency registration 2 w ks Tue 3/14/06 Mon 3/27/0611 Optical Subsystem 40 days Mon 2/27/06 Fri 4/21/0612 Use Spectrograph Optics 7 w ks Mon 2/27/06 Fri 4/14/0613 Optical Design 2 w ks Mon 2/27/06 Fri 3/10/0614 Lightpath Diagram 2 w ks Mon 3/13/06 Fri 3/24/0615 Simulation 2 w ks Mon 3/27/06 Fri 4/7/0616 Testing 2 w ks Mon 4/10/06 Fri 4/21/0617 Scanning Mirror 20 days Mon 2/27/06 Fri 3/24/0618 Linear stepper motor 4 w ks Mon 2/27/06 Fri 3/24/0619 Mechanical Structure 4 w ks Mon 2/27/06 Fri 3/24/0620 Software 45 days Mon 2/27/06 Fri 4/28/0621 Raw Data Dow nlaod 1 w k Mon 2/27/06 Fri 3/3/0622 Conversion to Matlab 2 w ks Mon 3/6/06 Fri 3/17/0623 Frequency Resolution 3 w ks Mon 3/20/06 Fri 4/7/0624 Create data Cube 3 w ks Mon 4/10/06 Fri 4/28/0625 Mechanical 50 days Mon 2/27/06 Fri 5/5/0626 Mirror 2 w ks Mon 2/27/06 Fri 3/10/0627 Enclosure 4 w ks Mon 3/13/06 Fri 4/7/0628 Optical Mounts 4 w ks Mon 4/10/06 Fri 5/5/0629 Intergration 20 days Mon 4/24/06 Fri 5/19/0630 Testing 3 w ks Mon 4/24/06 Fri 5/12/0631 Verify vs Requirements 1 w k Mon 5/15/06 Fri 5/19/0632 Final Report 1 w k Fri 5/19/06 Thu 5/25/06

1/2

4 11 18 25 1 8 15 22 29 5 12 19 26 5 12 19 26 2 9 16 23 30 7 14 21 28 4 11Dec '05 Jan '06 Feb '06 Mar '06 Apr '06 May '06 Jun '06


Task Partitioning

• Four main design areas– Optics System

• Development of custom optics system• Evaluation of ImSpector Optics System

– Image System / Algorithm Development• Interfacing image capture to MATLAB• Algorithms to normalize capture data and identification

– Scanning Mirror Motor Control• Precise movement of scanning mirror

– Mechanical Components• Enclosure and mounting system for optics


Optical Design


Background

• As a part-time employee at ITT Space Systems Division basic knowledge in optics is benificial

• Took on optical design as a learning experience


Requirements

• Initially minimal

• Achieve a wavelength of 400-700nm

• Resolution 25-50nm


Initial Research

• Initial inquiries made at ITT about optical design

• Dr. Conrad Wells assisted with providing relevant books and guidance to understanding calculations

• Dr. Wells also provided initial design paths as possibilities


Initial Optical Path

TAR

GE

T

MIR

RO

R

ENC

LOSU

RE

BA

FFLE

PR

ISM

LEN

S(3

) PL

ACE

S


TAR

GE

T

MIR

RO

R

EN

CLO

SUR

E

BA

FFLE

PR

ISM

LEN

S(3

) PL

ACES

Alternate Path


Relevant Work

• Upon looking into existing companies who make hyperspectral camera’s, Specim, a company in Finland was brought to attention by Scott Reardon

• On the website, the dispersion element, prism-grating-prism (PGP) was discussed

• Searching PGP turned up Mauri Aikio’s dissertation on hyperspectral imaging and his invention of the PGP


Work Continued

• Calculations for optimization of focal length while optimizing a prism size were preformed.

• Focal length results were in the range of one meter which was unacceptable for making a compact design.

• Alternate dispersion elements looked at.


lamda vs. h

-30

-20

-10

0

10

20

30

0.4 0.45 0.5 0.55 0.6 0.65 0.7 0.75 0.8 0.85 0.9 0.95 1

1.05

lamda (nm)

h (m

m)

Series1

Lambda vs. Height


Wavelength vs. Optimized Refracted Angle

-0.03

-0.02

-0.01

0

0.01

0.02

0.03

0.4

0.45 0.5

0.55 0.6

0.65 0.7

0.75 0.8

0.85 0.9

0.95 1

1.05

Lamda (nm)

Thet

a_t2

-thet

a_no

t (de

g)

Series1


Detector size Vs Input angle

0

10

20

30

40

50

60

70

80

90

100

0 10 20 30 40 50 60 70

Incident Angle (deg)

Foca

l Pla

ne S

ize

(mm

)

Series1


detector size vs. prism angle

0

5

10

15

20

25

30

35

40

45

50

0 10 20 30 40 50 60 70 80

prism angle (deg)

dete

ctor

siz

e (m

m)

Series1Series2Series3Series4Series5Series6Series7Series8Series9Series10Series11Series12Series13Series14Series15


Work Continued

• Research on dispersion elements and spectrographs performed

• Result of research turned up prism-grating-prism (PGP) element

• Correspondences with Edmund Optics (E.O.): suggested a monochromator

• Additional searching turned up handheld spectrograph


Dispersion Elements

• Two different dispersion elements considered

• Prism

• Blazed holographic grating


Prism ApplicationsLittrow Prism


Prism Apps. Cont.


Prism Test Sets


Prism Test Sets Cont.


Prism Test Sets Cont.

• This test set utilizes the Edmund Optics handheld spectrograph.

• The light source is an Hg pencil lamp.

• The video camera is recording the spectrum.


Prism-Grating-Prism

• Design discovered by Mauri Aikio of Finland

• A mate of a prism, longpass filter, covering glass, grating, substrate glass, shortpass filter, and another prism

• Provides for a compact and straight thru design for spectrographs


PGP Element

Prism 1LPF

Coveringglass

Grating

SPF

Prism 2

Substrate glass


Handheld Spectrograph

• This unit is approximately 4” long and 7/8” in diameter.

• It utilizes a multiple prism alignment or mate similar to the PGP element in the ImSpector.

• Very easy to use and test.

• Can easily be adapted to an existing system.


Handheld Spectrograph Prism Element Concept


Monochromator

• E.O. suggested a monorchromator in place of ordering a holographic grating and slit separately.

• Using the monochromator E.O. suggested allows you to tune it to what wavelength you want to look at.


Solution Guidelines

• Prism calculations yielded a large focal length

• D3 is attempting to demonstrate data acquisition

• Calibration of optics must be done by optical technician


Optical Design Solution

• The Handheld spectroscope, monocromator, or ImSpector camera all stood out to be the best optical solutions to meet requirements

• An optical system will be created in parallel by D. Scorse utilizing a Littrow prism as the dispersion element.


Imaging System

• Embedded System comprised of an imager and a DSP to capture and process frames

• Initial design will use USB to send acquired data to MATLAB for processing– Ease development of algorithms for normalization and

classification– Eventually move final processing inside the DSP


Imaging Hardware

• Several tradeoffs while choosing hardware for this application– CMOS vs. CCD

• Pre-existing interface hardware to CMOS sensor• CCD has complex circuitry, but better dynamic range and bit

depth• Preliminary design will use CMOS, future revisions may

implement a CCD sensor– Texas Instruments DSP

• C6416 vs. DM642


Image AcquisitionProcessing Board

Imager Board

1.3 MegaPixel10 Bit CMOS

Imager

DSP

C6416

VideoData

USB

I2C Serial Communications

SDRAM

FLASH

5VDC

Imager0

Optics


Image Classification

• Initial development of algorithms done using MATLAB

• Several associated issues– Camera calibration– Reflectance conversion– Image normalization– Spectral Mixing Correction– Material Classification against reference libraries


Scanning Mirror ControlStages of Development:• Generate PWM signals to rotate and stop mirror at 1° increments as a

baseline for data acquisition• Increase resolution to achieve a smooth steady rotation while varying

angular velocity to accommodate imager• Implement close-loop Kruse control to maximize resolution and provide

rotor position feedback• Incorporate additional interrupts to handle serial communications and

external velocity control


Driver Schematic for Stepper Motor Phase A

100

10k

100

10k

10k

100

10k

100

47k

47k

1k

1k .033 .033

IRFZ44E

IRFZ44EIRFZ44E

IRFZ44E

1uF

IN2

SHDN3LO

5

HO7

COM4

VB8

VCC1

VS6

IR21040

PWM_A1+

820nF

ZMM5251B100 uF

1uF

0

0

0

0

0

Vdd Vdd

31

2-

+

MtrDA1- MtrDA1+

820nF

Vmm

IN2

SHDN3 LO

5

HO7

COM4

VB8

VCC1

VS6

IR2104

V3_3/2

ViA1

Motor Current Sense

Half - Bridge DriverHalf - Bridge Driver

0

0

PWM_A1-


PWM Generation

• The top waveform is generated by selecting a base PWM carrier frequency

•The bottom waveform is the modulated signal produced through comparison with the percentage values stored in a table

•Thus, the angular velocity of the rotor can be controlled by slowing down the transition through the look-up table


PWM Generation

• Motor windings are modeled as a series LC circuit; and thus a low-pass filter

• Each PWM signal drives one half-bridge driver – need four signals per motor

• Other signals are generated identically except for either inversion or 90° phase shift

Effective Driver Signal


Mechanics

• Mirror is required to rotate +/- 20° relative to the center of the target image

• The distance between the mirror and the target is remains variable to allow for optimized data acquisition


Slit Width Analysis

tan(20 )y d= ⋅ °

Target Width 2y=

1 tan( )y dΔ = ⋅ ΔΘ 2 1tan(2 )y d yΔ = ⋅ ΔΘ − Δ 3 2 1tan(3 )y d y yΔ = ⋅ ΔΘ − Δ − Δ

( )1

1

tan( ) 1i

i jj

y d i y i−

=

Δ = ⋅ ΔΘ − Δ ∀ >∑

( ) ( )1 2 2

1 1 1tan( ) tan( ) tan ( 1)

i i i

i j j jj j j

y d i y d i d i y y− − −

= = =

⎛ ⎞Δ = ⋅ ΔΘ − Δ = ⋅ ΔΘ − − ΔΘ − Δ − Δ⎜ ⎟

⎝ ⎠∑ ∑ ∑

( ) ( )tan tan ( 1)iy d i d iΔ = ⋅ ΔΘ − ⋅ − ΔΘ


d [m] y [m] μStep Size Δθ

# Slits/ Half Image

Avg. Δy [mm] Min Δy [mm] Max Δy [mm]

0.0625 0.225 88 4.136 3.927 4.4300.03125 0.1125 177 2.056 1.963 2.2200.015625 0.05625 355 1.025 0.982 1.111

0.0078125 0.028125 711 0.512 0.491 0.5560.00390625 0.0140625 1422 0.256 0.245 0.278

0.0625 0.225 88 8.272 7.854 8.8600.03125 0.1125 177 4.113 3.927 4.4390.015625 0.05625 355 2.051 1.963 2.222

0.0078125 0.028125 711 1.024 0.982 1.1120.00390625 0.0140625 1422 0.512 0.491 0.556

0.0625 0.225 88 12.408 11.781 13.2890.03125 0.1125 177 6.169 5.890 6.6590.015625 0.05625 355 3.076 2.945 3.333

0.0078125 0.028125 711 1.536 1.473 1.6670.00390625 0.0140625 1422 0.768 0.736 0.834

0.0625 0.225 88 16.544 15.708 17.7190.03125 0.1125 177 8.225 7.854 8.8780.015625 0.05625 355 4.101 3.927 4.444

0.0078125 0.028125 711 2.048 1.963 2.2230.00390625 0.0140625 1422 1.024 0.982 1.112

0.0625 0.225 88 20.680 19.635 22.1490.03125 0.1125 177 10.282 9.817 11.0980.015625 0.05625 355 5.126 4.909 5.555

0.0078125 0.028125 711 2.560 2.454 2.7790.00390625 0.0140625 1422 1.280 1.227 1.390

1.092

5 1.820

4 1.456

3

3.6 ° per Full Step:

0.3641

2 0.728

Comparison of Open-Loop Control Variables


Close-Loop Kruse Control

• Technique for accurately calculating the rotor angle in real-time and without the use of an encoder

• Achieved by feeding back emf from sense windings and integrating to produce voltage signals proportional to the rotor’s displacement angle

• Allows for 40,000-60,000 micro-steps per revolution


d [m] y [m] μSteps / Full

Rev. Δθ# Slits/ Half

Image Avg. Δy [mm] Min Δy [mm]Max Δy [mm]

40000 0.00900 2222 0.163802986 0.1570796 0.177874045000 0.00800 2500 0.145588094 0.1396263 0.158115250000 0.00720 2777 0.131065983 0.1256637 0.142294355000 0.00655 3055 0.119139193 0.1142397 0.129362260000 0.00600 3333 0.109201990 0.1047198 0.118584940000 0.00900 2222 0.327605971 0.3141593 0.355747945000 0.00800 2500 0.291176187 0.2792527 0.316230450000 0.00720 2777 0.262131966 0.2513274 0.284588655000 0.00655 3055 0.238278386 0.2284795 0.258724460000 0.00600 3333 0.218403981 0.2094395 0.237169840000 0.00900 2222 0.491408957 0.4712389 0.533621945000 0.00800 2500 0.436764281 0.4188790 0.474345650000 0.00720 2777 0.393197948 0.3769911 0.426882955000 0.00655 3055 0.357417579 0.3427192 0.388086660000 0.00600 3333 0.327605971 0.3141593 0.355754740000 0.00900 2222 0.655211943 0.6283185 0.711495945000 0.00800 2500 0.582352375 0.5585054 0.632460850000 0.00720 2777 0.524263931 0.5026548 0.569177255000 0.00655 3055 0.476556772 0.4569589 0.517448860000 0.00600 3333 0.436807962 0.4188790 0.474339640000 0.00900 2222 0.819014929 0.7853982 0.889369845000 0.00800 2500 0.727940469 0.6981317 0.790576150000 0.00720 2777 0.655329914 0.6283185 0.711471555000 0.00655 3055 0.595695964 0.5711987 0.646811160000 0.00600 3333 0.546009952 0.5235988 0.5929245

Kruse Control

1 0.364

2 0.728

5 1.820

3 1.092

4 1.456

Slit Width Analysis for Kruse Controlled System


Mechanical Components

Main components:

– Project enclosure

– Scanning mirror fixture

– Optical mounts


ImSpector Imager Mount

• Threaded for standard C-mount fixture• Tapped for mounting of an imager adapter plate


Hyperspectral Enclosure

• Optical breadboard based• Light-weight lift-off aluminum box top• Draw latches for easy open/close


Scanning Mirror Fixture

• Machined mounting plate• Attached to stepper motor shaft• Adjustability of mirror height and angle• Set screw allows for easy manipulation


Lenses

• Commercially available mounting hardware• Modified to suit requirements

Hyperspectral Camera Design Project - EDGEedge.rit.edu/content/P06522/public/Hyperspectral...

Documents

Transcript of Hyperspectral Camera Design Project - EDGEedge.rit.edu/content/P06522/public/Hyperspectral...