Detecting Changes in Polarimetric SAR Datawith Content-Based Image Retrieval

Matthieu Molinier∗†, Jorma Laaksonen†, Yrjo Rauste∗ and Tuomas Hame∗∗VTT Technical Research Centre of Finland, Digital Information Systems, Earth Observation Team,

P.O. Box 1000, FI-02044 VTT, Finland - Email: matthieu.molinier@vtt.fi†Helsinki University of Technology, Adaptive Informatics Research Centre,

P.O. Box 5400, FI-02015 HUT, Finland

Abstract—In this study, we extended the potential of a Content-Based Image Retrieval (CBIR) system based on Self-OrganizingMaps (SOMs), for the analysis of remote sensing data. A databasewas artificially created by splitting each image to be analyzedinto small images (or imagelets). Content-based image retrievalwas applied to fully polarimetric airborne SAR data, using aselection of polarimetric features. After training the system onthis imagelet database, automatic queries could detect changes.Results were encouraging on airborne SAR data and may bemore useful for spaceborne polarimetric data.

I. INTRODUCTION

Two novel polarimetric SAR satellites, TerraSAR-X andRADARSAT-2, will be launched in 2007. In addition, ALOShas been launched in 2006 and is being taken into operativeuse. The immense amount of data generated by these satellitemissions demands new approaches to manage it efficiently.There is a growing interest in the remote sensing communityfor Content-Based Image Retrieval (CBIR), which allowsmanagement of large image archives, as well as satellite imageannotation and interpretation.

Our work extends the potential of PicSOM [1], a CBIRsystem based on Self-Organizing Maps (SOMs) [2], for polari-metric SAR image analysis. The key idea of our study [3] is toartificially generate a database of small images – or imagelets– from each full satellite image to be analyzed. Imageletscan be extracted from one scene for the detection of man-made structures or other targets, or from two (or more) scenesto detect changes. In this paper we present our experimentson change detection in fully polarimetric SAR data usingPicSOM.

II. PRE-PROCESSING AND DATABASE PREPARATION

Due to unavailability of fully-polarimetric spacebornedatasets suitable for change detection, polarimetric airbornedata was considered. The data consists of 2 EMISAR single-look complex (SLC in scattering matrix format) scenes ac-quired in March and July 1995. The March scene was thenregistered to the July scene – details on the data can be foundin [4]. 3-by-3 coherent averaging was applied to form 9-lookimages of 1280×1616 pixels, using PolSARPro software [5].

A scene is typically divided into several thousands ofimagelets, so that PicSOM produces relevant indexing. By thisoperation, the number of target classes within an imagelet isreduced compared to the original full scene. The extracts were

cut into 16 × 16 pixels small images, forming a database of8080 imagelets per scene.

For the purpose of method evaluation, a ground truth wascreated by classifying each scene into 5 classes {mountain,forest, water, ice, shadow} [4]. Supervised Wishart classifica-tion was used in PolSARPro, after delineating training areasover the Pauli decomposition RGB image. Lack of opticaldata for creating the ground truth resulted in reduced numberof classes and classification reliability.

III. FEATURES

Features were extracted from the polarimetric SAR image-lets to allow their indexing by the Self-Organizing Maps. Theoriginal PicSOM features were developed for RGB opticalimages. They are standard low-level measures of texture andcolor information, not suitable for polarimetric SAR images.

Table I sums up the features included into PicSOM forpolarimetric data analysis. Four Touzi polarimetric discrimi-nators [6] were considered : R0max the maximum scatteredintensity, NDR0 the normalized difference of the scatteredintensity, pmax the maximum degree of polarization and ∆pthe dynamic range of the degree of polarization. The polarazimuthal polarimetric signature [7] presents several advan-tages over the original polarimetric signature [8], mainly thecontinuity of range in orientation angle, and a less ambiguousmapping of horizontally and vertically polarized targets.

In addition, a coordinate feature was carried along thewhole analysis process, both to keep track of the positionof any imagelet within the full image, and to complete theframework for change detection. Table II lists the featuregroups extracted from the imagelets, and their dimensionality.For all features except the polarimetric signature and xy-coordinates, the imagelets were divided into 4 quadrants,from which the basic features were extracted after averagingthe coherency matrix. For example, an imagelet of 16 × 16pixels was divided into 4 quadrants of 8 × 8 pixels, overwhich the coherency matrix was averaged before extractingthe LOGRATIOS features – thus generating a feature vectorof dimension 4× 3 = 12. The copolarized and crosspolarizedsignatures were calculated on the average coherency matrixover a whole imagelet, then aggregated into a single 1444-dimension vector.

TABLE IFEATURES CONSIDERED IN THE STUDY. 〈〉 INDICATES SPATIAL AVERAGING (MULTILOOKING), S IS THE SCATTERING MATRIX, R IS THE STOKES

VECTOR. APPLICATION DOMAINS OF FEATURES IN THE REFERENCE PUBLICATIONS ARE GIVEN.

Feature [ref.] Expression Application

Amplitude of HH-VV correlation coeff. [9], [10] ρHH−V V =

∣∣∣ 〈SHHS∗V V 〉√|SHH |2|SV V |2

∣∣∣ Winter / spring crops discrimination

Phase difference HH-VV [10], [11] φHH−V V = arg(〈SHHS∗V V 〉) Sea ice classification

Ratio HV/VV in dB [9] 10 · log

(|SHV |2|SV V |2

)Bare soil / vegetation discrimination

Co-polarized ratio in dB [12] 10 · log

(|SV V |2|SHH |2

)Vegetation types discrimination

Cross-polarized ratio in dB [12] 10 · log

(|SHV |2|SHH |2

)Vegetation types discrimination

Co-polarized HH backscattering coeff. [10], [11] σ0HH = 〈SHHS∗

HH〉 Sea ice classification

Co-polarized VV backscattering coeff. [10], [11] σ0V V = 〈SV V S∗

V V 〉 Sea ice classification

Cross-polarized HV backscattering coeff. [9], [10] σ0HV = 〈SHV S∗

HV 〉 Agriculture (crops), sea ice classification

Co-polarization ratioσ0

V V

σ0HH

[11] γ =〈SV V S∗V V 〉〈S

HHS∗

HH〉 Sea ice classification

Depolarization ratioσ0

HV

σ0HH

+σ0V V

[10], [11] δ =〈SHV S∗HV 〉

〈SHH

S∗HH

〉+〈SV V

S∗V V

〉 Sea ice classification

H-α decomposition [13] H =1

log(3)

3∑i=1

−Pi · log(Pi), α =

3∑i=1

Piαi Target scattering properties

Degree of polarization [6] Dpol =

√R2

1+R22+R2

3R0

Target characterization / separation

Polar azimuthal polarimetric signature [7] Target scattering properties

TABLE IIFEATURE GROUPS EXTRACTED FROM THE IMAGELETS.

Feature group Features Dim

HHVV HH-VV correlation and phase difference 8LOGRATIOS 3 log-ratios 12

BACKSCATTER 3 backscatter coefficients 12POLRATIOS Copol. and depol. ratios 8

HALPHA Entropy H and α 8TOUZIDISC R0max, NDR0, pmax, ∆p 16

POLSIG Copol and crosspol signatures 1444xy-coordinates Imagelet index 2

IV. SELF-ORGANIZING MAPS FOR CONTENT-BASEDIMAGE RETRIEVAL CHANGE DETECTION

A. SOM training

The Self-Organizing Map is an unsupervised learningtechnique, which forms a non-linear mapping of a high-dimensional input space (here the space of imagelets) into atypically two-dimensional grid of artificial neural networks (orunits). In PicSOM, a separate SOM is trained for each featuretype. Through this mapping, feature vectors that reside neareach other in the input space are mapped into nearby units onthe map (called best matching units, or BMUs). Consequently,imagelets that are mutually similar in respect to the givenfeature have BMUs located near each other on the SOM.

After the SOMs have been trained, the database can be

visually queried: the system presents images then the usermarks a subset of them as relevant to the present query. Thisrelevance information is fed back to the system, which is thenable to find more similar images and return them in the nextquery round. The PicSOM system provides a semi-automated,interactively supervised analysis of satellite images [3].

B. Change detection with PicSOM

The PicSOM system can also be used for change detection,following the same approach. Considering two polarimetricSAR images taken over a given location at different times,each full image is similarly divided into imagelets. All image-lets are then merged to form a database, on which the featureSOMs are trained. The coordinate feature allows to define pairsof imagelets covering the same location (within the imageregistration accuracy), while temporal information is storedin the imagelet names.

The change magnitude is defined not in the image domain,but on the SOM grids: the distance between BMUs of theimagelets in an imagelet pair characterizes the change. Image-lets having similar content between the two acquisitions areexpected to have same or neighboring BMUs, whereas verydissimilar imagelets are represented by BMUs that are faraway from each other on the SOMs. The highest between-BMUs distances computed on all imagelet pairs point outlocations in the full image where the most significant changeshave occurred.

mountain forest water ice shadow

coords March coords July

POLSIG March POLSIG July

TOUZIDISC March TOUZIDISC July

Fig. 1. Distribution of ground truth over SOMs

V. RESULTS

Training of all the feature SOMs took 1.5 hours, usingeach feature vector 100 times in training. Fig. 1 shows thedistribution of the ground truth over several SOMs. The xy-coordinates SOM looks similar to the ground truth classifi-cation [4], whereas the other SOMs have their own coor-dinate systems not related to geographical coordinates. TheTOUZIDISC map shows some fragmented areas especially inMarch – where ground truth classification was challenging dueto ice coverage. The classes were less fragmented in July –that was the case for all feature SOMs. The more localizeda ground truth class is on a SOM, the better this SOM canretrieve imagelets containing that class. The POLSIG map wasthe least fragmented of all, even in March. Other maps weremuch more fragmented than that of TOUZIDISC feature.

Fig. 2 shows in red the 300 imagelets where the mostsignificant changes occurred, on the TOUZIDISC map. It wasthe feature that best detected the change corresponding tomelting ice in the central part of the lake. Surprisingly, those

changes do not appear anymore in the combination of featuresin Fig. 3(b). It may be because the other features (POLSIGmainly) dominated over the TOUZIDISC feature.

Fig. 2. 300 imagelets with the most significant changes – TOUZI feature

Fig. 3 shows in red the 300 imagelets where the changeswith most magnitude occurred, for two other maps. TheLOGRATIOS feature (Fig. 3(a)) produced an interesting result,although the ground truth was very fragmented over thismap. Most ”true” changes between March and July (observedvisually in the Pauli decompositions images) were detected bythis feature combination, i.e. low vegetation appearing near theriver (on the left part of the image), and ice melting on thelake (bottom-left part and central part of the lake).

The combination of TOUZIDISC, POLSIG andBACKSCATTER features (Fig. 3(b)) best described thechanges occurring around the river in left part of the image(mainly appearing vegetation).

The presented change detection technique was unsuper-vised. With a more reliable ground truth it would have beenpossible to perform supervised or interactive change detection[3]. A major advantage of the imagelet-based approach tochange detection is that it is less sensitive to mis-registrationthan pixel-based approaches – for example those tested in[4] on the same dataset. Also the type of the change can bedetermined from the locations of the BMUs on the SOMs.

Apart from combining features into a same SOM, one coulduse directly the Best-Matching Units (BMUs) to combineinformation from different SOMs. If a pair of imagelet isconsidered to contain changes for a given feature (e.g. LO-GRATIOS), one can check the BMUs of those two imageletson another feature SOM (e.g. HALPHA). BMUs of eachimagelet on that SOM map would be the values of entropyH and angle α, and they could be plot on a H-α plane forfurther target and change characterization.

VI. CONCLUSIONS

Our preliminary change detection results on polarimetricSAR data were encouraging, though need more tuning.Imagelet-based structure detection does not provide direct

(a) LOGRATIOS feature (b) Combination of TOUZIDISC, POLSIG and BACKSCATTER

Fig. 3. 300 imagelets with the most significant changes

delineation of objects of interest (contrary to pixel-basedmethods), but it can highlight in a full scene the possiblelocations where structures or other content of interest mayexist. Such a system could point out locations in the fullimage where there may be potentially interesting structures, orfeatures defined visually by the user that may have appearedor disappeared. One of the many advantages of PicSOMfor polarimetric SAR data analysis is its ability to integratevarious features, and to perform a selection of relevant onesbased on user queries. Further research will investigate othercombinations of polarimetric SAR features, and will require adataset with more accurate ground truth for method evaluation.The method could also be applied to man-made structuredetection in polarimetric images, as it was on optical images inour previous study. It is also expected the PicSOM system willbe tested on a pair of ALOS PALSAR images over Finland.

ACKNOWLEDGMENT

The authors are grateful to Professor Torbjørn Eltoft andAnthony Doulgeris, from University of Tromsø (Norway), forproviding us with the EMISAR data. We also wish to thankRidha Touzi and Iain Woodhouse for clarifying some pointson their respective publications.

REFERENCES

[1] PicSOM Development Group, “PicSOM online demonstration,”http://www.cis.hut.fi/picsom, Helsinki University of Technology, 1998–2006.

[2] T. Kohonen, Self-Organizing Maps, 3rd ed., ser. Springer Series inInformation Sciences. Springer-Verlag, 2001, vol. 30.

[3] M. Molinier, J. Laaksonen, and T. Hame, “Detecting man-made struc-tures and changes in satellite imagery with a content-based informationretrieval system built on self-organizing maps,” IEEE Trans. Geoscienceand Remote Sensing, vol. 45, pp. 861–874, April 2007.

[4] M. Molinier and Y. Rauste, “Comparison and evaluation of polarimetricchange detection techniques in aerial SAR data,” in IEEE InternationalGeoscience and Remote Sensing Symposium IGARSS’07, 2007.

[5] E. Pottier, L. Ferro-Famil, S. Allain, S. Cloude, I. Hajnsek, K. Pap-athanassiou, A. Moreira, M. Williams, T. Pearson, and Y.-L. Desnos,“An overview of the PolSARpro v2.0 software. the educational toolboxfor polarimetric and interferometric polarimetric SAR data processing,”in POLinSAR07 ESA Workshop, January 2007.

[6] R. Touzi, S. Goze, T. L. Toan, A. Lopes, and E. Mougin, “Polarimetricdiscriminators for SAR images,” IEEE Transactions on Geoscience andRemote Sensing, vol. 30, no. 5, pp. 973–980, September 1992.

[7] I. Woodhouse and D. Turner, “On the visualization of polarimetricresponse,” International Journal of Remote Sensing, vol. 24, no. 6, pp.1377–1384, March 2003.

[8] J. V. Zyl, H. Zebker, and C. Elachi, “Imaging radar polarizationsignatures: theory and observation,” Radio Science, vol. 22, no. 4, pp.529–543, July-August 1987.

[9] S. Quegan, T. Le Toan, H. Skriver, J. Gomez-Dans, M. C. Gonzalez-Sampedro, and D. H. Hoekman, “Crop classification with multitemporalpolarimetric SAR data,” in Workshop on POLinSAR - Applications ofSAR Polarimetry and Polarimetric Interferometry (ESA SP-529), January2003.

[10] H. Skriver, W. Dierking, P. Gudmandsen, T. L. Toan, A. Moreira,K. Papathanassiou, and S. Quegan, “Applications of synthetic apertureradar polarimetry,” in Workshop on POLinSAR - Applications of SARPolarimetry and Polarimetric Interferometry (ESA SP-529), January2003.

[11] W. Dierking, H. Skriver, and P. Gudmandsen, “SAR polarimetry forsea ice classification,” in Workshop on POLinSAR - Applications ofSAR Polarimetry and Polarimetric Interferometry (ESA SP-529), January2003.

[12] J. R. Buckley, “Environmental change detection in prairie landscapeswith simulated RADARSAT 2 imagery,” in IEEE International Geo-science and Remote Sensing Symposium IGARSS’02, 2002, pp. 3255–3257.

[13] S. Cloude and E. Pottier, “An entropy based classification schemefor land applications of polarimetric SAR,” IEEE Transactions onGeoscience and Remote Sensing, vol. 35, no. 1, pp. 68–78, January1997.