Spatial and Spectral Evaluation of Image Fusion Methods

Sascha KlonusManfred Ehlers

Institute for Geoinformatics and Remote SensingUniversity of Osnabrck

ContentIntroduction Image Fusion

Test Site

Fusion Results

Color Distortions

Evaluation Methods and Results

Ehlers Fusion

Conclusions and Future Work

Data Fusion: Why is it Necessary?Remote sensors have different spatial resolution for panchromatic and multispectral imagery

The ratios vary between 1:2 and 1:5

For multisensor fusion the ratios can exceed 1:30 (e.g. Ikonos/Landsat)

Objectives of Image FusionSharpen imagesImprove geometric correctionsProvide stereo-viewing capabilitiesEnhance certain features Complement data setsDetect changesSubstitute missing information Replace defective data

Pohl & van Genderen (1998)

Meaning of Pan-SharpeningSpatialSpectral +panchromatic &high geometric resolutionmulti-/hyperspectral image &low geometric resolutionmulti-/hyperspectral &high geometric resolution

Fusion MethodsColor TransformationsModified IHS Transformation

Statistical MethodsPrincipal Component Merge

Numerical MethodsBroveyCN Spectral SharpeningGram-Schmidt Spectral Sharpening Wavelet based Fusion

Combined MethodsEhlers Fusion

Test Site

Original DataQuickbird Multispectral image (2004-09-04)Quickbird Panchromatic image (2004-09-04)

Formosat Multispectral image (2004-01-30)Ikonos Multispectral image (2005-08-03)

Single Sensor Fusion: QuickbirdQuickbird Multispectral imageFused with BroveyFused with CN Spectral SharpeningFused with EhlersFused with WaveletFused with Gram-Schmidt Fused with PCFused with modified IHS

Multisensor Fusion: IkonosIkonos Multispectral imageFused with BroveyFused with CN Spectral SharpeningFused with EhlersFused with modified IHSFused with PCFused with Gram-Schmidt Fused with Wavelet

Multisensor Fusion: FormosatFormosat Multispectral imageFused with BroveyFused with CN Spectral SharpeningFused with EhlersFused with modified IHSFused with PCFused with Gram-Schmidt Fused with Wavelet

Fusion Problem: Color DistortionPanchromatic band has a different spectral sensitivity

Multisensoral differences (e.g. Ikonos and SPOT merge)

Multitemporal (seasonal) changes between pan and ms image dataInconsistent panchromatic information is fused into the multispectral bands

Spectral Comparison Methods (1) Visual (Structure and Colour Preservation)

Results RMSE

Results Correlation Coefficients

Spectral Comparison Methods (2)Per Pixel DeviationDegradeDegraded to ground resolution of original image (Formosat = 8m)Original multispectral image (Formosat 8m)Result: Vector containing the deviation per pixelFused image (Formosat 2m)

Band 12.56Band 22.92Band 33.49Band 43.35

Mean Per Pixel Deviation

Spatial Comparison Methods (1)Edge Detection---

Band 191.16 %Band 292.10 %Band 392.64 %Mean91.96 %

Results Edge Detection

Spatial Comparison Methods (2)Highpass FilteringCorrelation

Band 10.8012Band 20.7820Band 30.7912Mean0.7918

Highpass Correlation Results

FFT Filter Based Data Fusion (Ehlers Fusion)Panchromatic ImageMultispectral ImageBasis: IHS Transform and Filtering in the Fourier Domain

Panchromatic image and its spectrumOriginal panchromatic imagePanchromatic Spectrum

Filtersetting effectsFiltered Panchromatic Spectrum

Effects in the spatial domainFiltered panchromatic imageFused image

Filtersetting effectsFiltered Panchromatic Spectrum

Effects in the spatial domainFiltered panchromatic imageFused image

Filtersetting effectsFiltered Panchromatic Spectrum

Effects in the spatial domainFiltered panchromatic imageFused image

ResultsEhlers Fusion shows the best overall results in all images

It works also if the panchromatic Information does not match the spectral sensitivity of the merged bands (multitemporal and multisensoral fusion)

Its performance is superior to standard fusion techniques (IHS, Brovey Transform, PC Merge)

Wavelet preserves the spectral characteristics at the cost of spatial improvement

Ehlers Fusion is integrated in a commercial image processing system (Erdas Imagine 9.1)

Future WorkFusion of radar- and optical Data

Development of one method to evaluate the spatial and spectral quality of an fused image

Comparison with the algorithm of Zhang (PCI Geomatica)

Research on automation for filter design

Thanks for your Attention

Questions???

Ehlers Fusion Program

Ehlers Fusion Program

Ehlers Fusion Program

Ehlers Fusion Program

Ehlers Fusion Program

Multispectral image and its spectrumOriginal multispectral intensityMultispectral intensity spectrum

Filtersetting effectsFiltered multispectral spectrum

Filtersetting effectsFiltered multispectral spectrum

Filtersetting effectsFiltered multispectral spectrum

Sharpen imagesImprove geometric correctionsProvide stereo-viewing capabilities for stereophotogrammetryEnhance certain features not visible in either of the single data aloneComplement data sets for improved classificationDetect changes using multitemporal imagesSubstitute missing information in one image with signals from another sensor imageReplace defective data

4th September 2004, for Ikonos the 30th January 2004, for Formosat the 3rd August 2005