Land cover identification for finding hazelnut fields using WV2 imagery

Kadim Taşdemir1 and Selcuk Reis2

1 Monitoring Agricultural Resources Unit Institute for Environment and SustainabilityEuropean Commission Joint Research Centre

2 Aksaray University, Turkey

Outline• Introduction

– Significance of the problem– Detection of orchards– Hazelnut study area

• Methodology– Textural (Gabor) features and spectral values– Self Organizing Maps (SOMs) and learning vector quantization (LVQ)

• Experimental results– Study area– Performance measures (accuracy, kappa, etc.)– Additional value of WV2 imagery

• Conclusions and broader applications

Significance of the problem• Land cover identification from remote sensing images has been

essential for agricultural management and monitoring.

• An economically very essential agricultural practice is permanent crops.

• Among permanent crops, nuts (hazelnuts, almonds, walnuts, pistachios, locust beans) are recognized for their role in– rural development (their orchards are often grown in lands where

cultivation is difficult) and – environmental respect (the adopted practices for nuts cultivation

enable to efficiently fight against erosion).

An accurate land cover identification delineating fields of permanent crops (hazelnuts in this study) will aid in better agricultural management.

Detection of orchards• Interactive detection of individual orchards by analysis of panchromatic or

multispectral bands – OliCount, Peedell et al. ESRI 1999 – NUTGIS report, JRC, 2006– Daliakopoulos et al. PE&RS 2009

• Textural features calculated from panchromatic band to delineate orchards, vineyards, and forest, thanks to their (ideally) discriminative spatial patterns – Wavelet features, Ranchin et al. PE&RS 2001– Autocorrelation of neighbor pixels, Warner and Steinmaus, PE&RS 2005– Variogram analysis, Trias-Sanz, IEEE TGRS 2006– Structural features, Yalniz and Aksoy, Pattern Recognition, 2010

• Combination of spectral and textural features– Aksoy et al., IEEE TGRS 2010

• Combination of spectral, temporal and (Gabor) textural features (Quickbird)– Reis and Taşdemir, ISPRS Photo. 2011

Hazelnut study area• The major hazelnut producer

in the world (about 75%) : Turkey– along the Black Sea coast

(where relief is rather strong) – hazel orchards are often

small with a high planting density, and various plantation patterns

– natural vegetation in the area can be spatially discontinuous.

A photo from the study area

A color composite image of hazelnut fields

Methodology

1. Feature selection– Textural features for spatial properties, obtained from panchromatic imagery– Spectral values– Merged features obtained by combining normalized spectral values and Gabor features

2. Classification (using SOM and LVQ)– Determine land cover types 5 classes: hazelnut, other woodlands, agriculture, soil, urban

areas – C1: Classify textural features good performance for woody vegetation (hazelnut and

woodlands)– C2: Classify merged features good overall classification performance.

3. Decision rule– Modify the classification result of C2, by updating hazelnut assignment according to C1

Spectral values

Textural features

Classifier 2

Classifier 1

Multispectral imagery

Panchromatic imagery

Decision rule

Classification result

Feature selection• Textural features: multi-scale multi-oriented Gabor features

to represent distinctive spatial properties (regular plantation of orchards) 4 scales, 6 orientation: select the highest response for each scale to obtain orientation independent features, resulting in 4 features

• Spectral values: to represent spectral reflectance at various bands

An RGB color composite of 3 scales of Gabor features

Woody vegetation (nuts, forests, woodlands) have high responses for small scales (greenish, brownish regions);

Urban areas have high responses for all/large scales (white, bluish regions) whereas

Smooth areas (soil, agriculture) have low responses (dark regions)

Classification method*: SOMs

• A common approach for land cover identification is the use of the self-organizing map (SOM)

• SOMs are shown useful especially for remote sensing applications with complex cluster structures (Villmann et al. 2003, Merenyi et al 2005, Taşdemir and Merenyi 2009)

• Learning vector quantization (LVQ):supervised vector quantization based on SOM, which can be successfully used with few labeled samples

*: For this study, the aim is to analyze the performance of WorldView2 imagery, therefore selection of the classifier is not of primary importance. Instead of SOM and LVQ usage, other common approaches such as Maximum Likelihood Classifier, Radial Basis Functions and Support Vector Machines may also be preferred.

Self-organizing maps

A topology preserving 2-d spatial mapping of acoustic frequencies on the auditory cortex

A Self-Organizing Map (SOM)

• an unsupervised artificial neural network learning paradigm

• intends to mimic this data processing (Kohonen, 1982).

Self-organizing maps

A 2-d SOM rigid lattice

Data manifold

d-dimensional

prototypes

Voronoi polyhedra of

some neighbor prototypes

1. Competition

i = arg minj ||x - wj||, j=1, … , N

2. Cooperation

The neighborhood of i is activated wrt a neighborhood function hj,i(x)(t).

3. Synaptic adaptation

wj(t+1) = wj (t)+ε0(t) h j,i(x) (t) (x -

wj(t))

Self-organizing maps

A 2-d SOM rigid lattice

Data manifold

d-dimensional

prototypes

adaptive vector quantizer

prototypes ordered by similarities on a 2-d rigid grid

nonlinear topology preserving mapping

Voronoi polyhedra of

some neighbor prototypes

Learning vector quantization• LVQ is a supervised learning, which slightly modifies the

SOM units when a misclassification occurs:

– Each SOM unit is labeled by maximum vote of the labels of training samples mapped to it.

– Labeled training samples v are iteratively mapped to the labeled SOM, and the closest SOM unit wi and the second closest SOM unit wj to a randomly selected training sample v are adapted, using the equation below, if wi and v are in different classes whereas wj and v are in the same class.

Outline• Introduction

– Significance of the problem– Detection of orchards– Hazelnut study area

• Methodology– Textural (Gabor) features and spectral values– Self Organizing Maps and learning vector quantization

• Experimental results– Study area– Performance measures (accuracy, kappa, etc.)– Additional value of WV2 imagery

• Conclusions and broader applications

Study area• WorldView 2 imagery

Panchromatic and 8 multispectral bands

• 2920*4775 pixels

• ~3.5 km2

Study areaAn example representation of the study area using natural color composite

For agricultural management and monitoring required in this study,5 classes (hazelnut, woodland, agriculture, soil and urban regions) are selected.

Classification maps - GaborClassification map produced when Gabor features (calculated from pan) were used:

PA_hazelnut= 95.0%; UA_hazelnut =80.7%; F_b=0.87; OA = 77.9%; Kappa = 0.66

PA: producer accuracy, UA: user accuracy, F_b: geometric mean of PA and UA of hazelnutOA:overall accuracy

Classification maps – 8bands&GaborClassification map produced when 8 multispectral bands and Gabor features were used:

PA_hazelnut= 93.6%; UA_hazelnut =74.7%; F_b=0.83; OA = 83.2%; Kappa = 0.74

PA: producer accuracy, UA: user accuracy, F_b: geometric mean of PA and UA of hazelnutOA:overall accuracy

Classification maps – MergedClassification map produced when a decision rule was added to 8 bands and Gabor features

PA_hazelnut= 90.6%; UA_hazelnut =85.1%; F_b=0.88; OA = 87.8%; Kappa = 0.81

PA: producer accuracy, UA: user accuracy, F_b: geometric mean of PA and UA of hazelnutOA:overall accuracy

Performance results of WV2 imageryWorldView 2 imagery

Measure Class MS 8band MS 4 band Gabor MS+Gabor Merged

PA

woodland 64,68 75,51 79,82 71,73 85,8nut 85,41 87,77 95 93,64 90,6urban 93,24 93,19 39,46 82,64 82,64agri 80,26 45,99 50,54 84,37 84,37soil 75,81 84,76 46,64 93,07 93,07

UA

woodland 84,05 91,05 81,34 89,56 87,97nut 68,62 70,19 80,68 74,66 85,09urban 77,55 84,28 88,83 93,09 93,09agri 77,76 64,26 54,27 92,35 92,35soil 92,75 94,1 60 92,62 92,62

Overall accuracy 76,38 79,47 77,94 83,24 87,8F_b 0,76 0,78 0,87 0,83 0,88

Kappa 0,64 0,69 0,66 0,74 0,81

Gabor features has good PA and UA values for woodlands and hazelnut, in expense of very poor results for other

classes

Combined usage of 8 bands and Gabor features increases performance significantly

A decision rule in addition to combined usage of 8 bands and Gabor features

produces the best classification performance

Confusion matrix for MergedTotal Woodland Hazelnut Urban Agri. Soil PA

90072 77280 12591 127 70 4 85,8082824 6572 75035 0 1217 0 90,6014488 1290 0 11973 0 1225 82,6418442 2317 561 1 15560 3 84,3716611 388 0 761 2 15460 93,07

UA 87,97 85,09 93,09 92,35 92,62 87,80

4 bands produces better classification than 8 bands (for this study)

in expense of poor performance for agriculture class (confusion with

dense hazelnut fields)

Low performance measures

Additional value of 4 new bands

The use of 8-bands produced higher performance measures

for the proposed method

When merged features considered only, the use of 8-bands: higher F_b on average; has a lower performance wrt Kappa and overall accuracyhas better discrimination for 3 classesMain confusion is between woodland and hazelnut

Comparison to QB performance

Imagery --> WorldView 2 imagery Quickbird imageryMeasure Class MS 8band MS 4 band Gabor MS 4 band Gabor

PA

woodland 64,68 75,51 79,82 65,66 74,96nut 85,41 87,77 95 81,37 92,16urban 93,24 93,19 39,46 85,96 32,98agri 80,26 45,99 50,54 37,65 11,85soil 75,81 84,76 46,64 16,86 17,55

UA

woodland 84,05 91,05 81,34 76,06 75,52nut 68,62 70,19 80,68 63,8 71,3urban 77,55 84,28 88,83 88,8 80,06agri 77,76 64,26 54,27 35,55 28,37soil 92,75 94,1 60 55,28 25,8

Overall accuracy 76,38 79,47 77,94 66,72 69,19

F_b (hazelnut) 0,76 0,78 0,87 0,72 0,8

Kappa 0,64 0,69 0,66 0,49 0,51

A better performance by using spectral bands (either 8-band or 4-

band) of WV2 imagery

A better performance by using Gabor features extracted from WV2

panchromatic imagery

The performance improvement achieved by using WV2 imagery,

mainly due to the increased spatial resolution (0.5m versus 0.6m of QB imagery),

significant to achieve an accurate land cover identification for finding hazelnut fields

Conclusions• A classification system which accurately finds hazelnut fields together

with lands in good agricultural condition.

• If the aim of the study is solely to determine hazelnut fields, the use of textural features can achieve a similar detection accuracy.

– However, including multispectral bands improves the overall accuracy and F_b values significantly.

• The use of 8-band produces a better performance than the use of 4-band, – but the improvement is not of great significance and a cost/benefit analysis should be

assessed based on the application requirements.

• The resulting precise delineation of nut fields aids their management and control support schemes regulated by the CAP of the EU.

Conclusions• The proposed system can be applied to find other permanent crops, since

they often have a regular plantation pattern that can be represented by textural features.

• The proposed system may also be used to determine lands in good

agricultural condition,– especially for Bulgaria and Romania, where it is necessary to annually

assess agricultural land eligible for the CAP subsidies

• Moreover, it may be used find eligible/ineligible features within the agricultural lands, which need to be extracted according to the CAP regulations.

Thank you for your attention!