UAV-based Crops Classification with joint features from ......An aerial image classification method...

6
UAV-based Crops Classification with joint features from Orthoimage and DSM data Bin Liu 1 , Yun Shi 1, *, Yulin Duan 1 , Wenbin Wu 1 1 (1Key Laboratory of Agricultural Remote Sensing, Ministry of Agriculture, China /Institute of Agricultural Resources and Regional Planning, Chinese Academy of Agricultural Sciences, Beijing 100081, China)- (liubin, shiyun, duanyulin, wuwenbin)@caas.cn Commission VI, WG VI/4 KEY WORDS: UAV; crop classification; SVM; DOM; DSMs; Texture. ABSTRACT: Accurate crops classification remains a challenging task due to the same crop with different spectra and different crops with same spectrum phenomenon. Recently, UAV-based remote sensing approach gains popularity not only for its high spatial and temporal resolution, but also for its ability to obtain spectra and spatial data at the same time. This paper focus on how to take full advantages of spatial and spectrum features to improve crops classification accuracy, based on an UAV platform equipped with a general digital camera. Texture and spatial features extracted from the RGB orthoimage and the digital surface model of the monitoring area are analysed and integrated within a SVM classification framework. Extensive experiences results indicate that the overall classification accuracy is drastically improved from 72.9% to 94.5% when the spatial features are combined together, which verified the feasibility and effectiveness of the proposed method. 1. INTRODUCTION Information on crop sowing type and yield is an important basis for the country to formulate scientific agricultural policies and economic plans, and is also an important part of the core indicators of agricultural statistics in countries around the world. Timely understanding and mastering crop types has important practical significance for accurately estimating and predicting crop yields, strengthening crop production management, adjusting agricultural cropping structure, and ensuring national food security. In addition, the acquisition of high-precision crop acreage has gradually become one of the most important scientific issues in agricultural land system research (Cao Weibin et al, 2004a; Huang Qing et al, 2009a; Yang Bangjie et al, 1997a). For a long time, China's agricultural crop sowing types, sown area, planting quantity and other important agricultural statistics have been obtained mainly through comprehensive statistical methods or sample surveys those have been reported step by step. Agricultural survey teams distributed throughout the country regularly collect planting areas and types of crops. Crops growth and disaster-affected situations are reported step by step or reported directly to the Ministry of Agriculture as the basis for analysing the state of agricultural structure planting and taking countermeasures. The traditional method is to manually measure the ground sample or use GE images to assist in the investigation of crop classification (Liu Jia et al, 2015a). Because the surveyors' abilities to investigate are different and cannot be objectively standardized, there are lags in the collection, processing, and reporting process. Differences in the information, and this method has many defects such as huge investigation workload, high financial and material costs, and long investigation period (Yang Bangjie et al, 1997a). In recent years, remote sensing technology has played an important role in dynamic information extraction of crop areas and crop distribution mapping (Zhang Jiankang et al, 2012a). Satellite remote sensing information has the characteristics of large coverage area, short detection period, abundant data, low cost, and provides new technical means for quickly and accurately obtaining crop types (Chen Zhongxin et al, 2016a). However, due to the long period of high-resolution satellite re- * Corresponding author entry cycle, the data for a given area at a given time cannot be guaranteed. The accuracy of crop land area monitoring using satellite remote sensing cannot meet the requirements and must be supplemented by ground sample surveys and ground sample survey data. Using the ground sample survey data to calculate the deduction coefficient of linear and fine features, to achieve the fine extraction of crop planting areas extracted by remote sensing. The emergence and development of UAV remote sensing technology has provided new ideas for the collection of crop information (Freeman et al, 2015a; Mesas-Carrascosa et al, 2014a; Rokhmana et al, 2015a). At the small and medium scale, UAV remote sensing can play a greater role and can obtain more accurate crop distribution information, which is of great significance to the development and application of crop monitoring technology. UAV remote sensing has features such as high resolution, simple operation, fast data acquisition, and low cost. It can quickly perform image collection for a certain area and combine ground actual measurement data to quickly and accurately complete the crop planting information monitoring task. It can be used as a useful complement to satellite remote sensing and aerial remote sensing, and provides accuracy verification for large-scale remote sensing (Del Pozo et al, 2014a) At present, many scholars have done a lot of research on the classification of crops based on drones, and put forward many techniques and methods. Pena used object-oriented methods based on the obtained six-band multispectral images to achieve weed mapping during early corn growth (Pena et al, 2013a). Gini used the maximum likelihood method and used the obtained multispectral images to classify different trees (Gini et al, 2016a). Sona used multi-spectral images for soil and crop mapping to realize the application of multi-spectral data in precision agriculture (Sona et al, 2016a). Rypochi Doi performed color synthesis of multi-spectral images, and found that this method can increase the distinguishability of similar pixels (Rypochi Doi et al, 2015a). Oumer used multi-spectral images and RGB images to achieve land cover and crop classification using Random Forest Algorithm (Oumer et al, 2017a). Xiuliang Jin used an optical camera mounted on a drone to acquire RGB images and used the SVM algorithm to estimate the planting density of winter wheat. The maximum likelihood method and SVM The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-3, 2018 ISPRS TC III Mid-term Symposium “Developments, Technologies and Applications in Remote Sensing”, 7–10 May, Beijing, China This contribution has been peer-reviewed. https://doi.org/10.5194/isprs-archives-XLII-3-1023-2018 | © Authors 2018. CC BY 4.0 License. 1023

Transcript of UAV-based Crops Classification with joint features from ......An aerial image classification method...

Page 1: UAV-based Crops Classification with joint features from ......An aerial image classification method based on feature selection was proposed to realize the recognition of flooded areas

UAV-based Crops Classification with joint features from Orthoimage and DSM data

Bin Liu1, Yun Shi 1, *, Yulin Duan 1, Wenbin Wu 1

1 (1Key Laboratory of Agricultural Remote Sensing, Ministry of Agriculture, China /Institute of Agricultural Resources and

Regional Planning, Chinese Academy of Agricultural Sciences, Beijing 100081, China)- (liubin, shiyun, duanyulin,

wuwenbin)@caas.cn

Commission VI, WG VI/4

KEY WORDS: UAV; crop classification; SVM; DOM; DSMs; Texture.

ABSTRACT:

Accurate crops classification remains a challenging task due to the same crop with different spectra and different crops with same

spectrum phenomenon. Recently, UAV-based remote sensing approach gains popularity not only for its high spatial and temporal

resolution, but also for its ability to obtain spectra and spatial data at the same time. This paper focus on how to take full advantages of

spatial and spectrum features to improve crops classification accuracy, based on an UAV platform equipped with a general digital

camera. Texture and spatial features extracted from the RGB orthoimage and the digital surface model of the monitoring area are

analysed and integrated within a SVM classification framework. Extensive experiences results indicate that the overall classification

accuracy is drastically improved from 72.9% to 94.5% when the spatial features are combined together, which verified the feasibility

and effectiveness of the proposed method.

1. INTRODUCTION

Information on crop sowing type and yield is an important basis

for the country to formulate scientific agricultural policies and

economic plans, and is also an important part of the core

indicators of agricultural statistics in countries around the world.

Timely understanding and mastering crop types has important

practical significance for accurately estimating and predicting

crop yields, strengthening crop production management,

adjusting agricultural cropping structure, and ensuring national

food security. In addition, the acquisition of high-precision crop

acreage has gradually become one of the most important

scientific issues in agricultural land system research (Cao Weibin

et al, 2004a; Huang Qing et al, 2009a; Yang Bangjie et al, 1997a).

For a long time, China's agricultural crop sowing types, sown

area, planting quantity and other important agricultural statistics

have been obtained mainly through comprehensive statistical

methods or sample surveys those have been reported step by step.

Agricultural survey teams distributed throughout the country

regularly collect planting areas and types of crops. Crops growth

and disaster-affected situations are reported step by step or

reported directly to the Ministry of Agriculture as the basis for

analysing the state of agricultural structure planting and taking

countermeasures. The traditional method is to manually measure

the ground sample or use GE images to assist in the investigation

of crop classification (Liu Jia et al, 2015a). Because the

surveyors' abilities to investigate are different and cannot be

objectively standardized, there are lags in the collection,

processing, and reporting process. Differences in the information,

and this method has many defects such as huge investigation

workload, high financial and material costs, and long

investigation period (Yang Bangjie et al, 1997a).

In recent years, remote sensing technology has played an

important role in dynamic information extraction of crop areas

and crop distribution mapping (Zhang Jiankang et al, 2012a).

Satellite remote sensing information has the characteristics of

large coverage area, short detection period, abundant data, low

cost, and provides new technical means for quickly and

accurately obtaining crop types (Chen Zhongxin et al, 2016a).

However, due to the long period of high-resolution satellite re-

* Corresponding author

entry cycle, the data for a given area at a given time cannot be

guaranteed. The accuracy of crop land area monitoring using

satellite remote sensing cannot meet the requirements and must

be supplemented by ground sample surveys and ground sample

survey data. Using the ground sample survey data to calculate the

deduction coefficient of linear and fine features, to achieve the

fine extraction of crop planting areas extracted by remote sensing.

The emergence and development of UAV remote sensing

technology has provided new ideas for the collection of crop

information (Freeman et al, 2015a; Mesas-Carrascosa et al,

2014a; Rokhmana et al, 2015a). At the small and medium scale,

UAV remote sensing can play a greater role and can obtain more

accurate crop distribution information, which is of great

significance to the development and application of crop

monitoring technology. UAV remote sensing has features such

as high resolution, simple operation, fast data acquisition, and

low cost. It can quickly perform image collection for a certain

area and combine ground actual measurement data to quickly and

accurately complete the crop planting information monitoring

task. It can be used as a useful complement to satellite remote

sensing and aerial remote sensing, and provides accuracy

verification for large-scale remote sensing (Del Pozo et al, 2014a)

At present, many scholars have done a lot of research on the

classification of crops based on drones, and put forward many

techniques and methods. Pena used object-oriented methods

based on the obtained six-band multispectral images to achieve

weed mapping during early corn growth (Pena et al, 2013a). Gini

used the maximum likelihood method and used the obtained

multispectral images to classify different trees (Gini et al, 2016a).

Sona used multi-spectral images for soil and crop mapping to

realize the application of multi-spectral data in precision

agriculture (Sona et al, 2016a). Rypochi Doi performed color

synthesis of multi-spectral images, and found that this method

can increase the distinguishability of similar pixels (Rypochi Doi

et al, 2015a). Oumer used multi-spectral images and RGB images

to achieve land cover and crop classification using Random

Forest Algorithm (Oumer et al, 2017a). Xiuliang Jin used an

optical camera mounted on a drone to acquire RGB images and

used the SVM algorithm to estimate the planting density of

winter wheat. The maximum likelihood method and SVM

The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-3, 2018 ISPRS TC III Mid-term Symposium “Developments, Technologies and Applications in Remote Sensing”, 7–10 May, Beijing, China

This contribution has been peer-reviewed. https://doi.org/10.5194/isprs-archives-XLII-3-1023-2018 | © Authors 2018. CC BY 4.0 License.

1023

Page 2: UAV-based Crops Classification with joint features from ......An aerial image classification method based on feature selection was proposed to realize the recognition of flooded areas

algorithm were used to classify the shadows, indicating that the

SVM classifier had a better classification effect in classification

process (Xiuliang Jin et al, 2017a). Adrien Michez used drones

to obtain multi-temporal and hyperspectral images and used a

random forest algorithm to classify the tree types and health

status of the riparian forest zone (Adrien Michez et al, 2017a). J.

Senthilnath used the K-means generator and the expected

maximum method to classify crop mapping and canopy mapping.

An aerial image classification method based on feature selection

was proposed to realize the recognition of flooded areas and

roads (J. Senthilnath et al, 2017a). Han Wenting used multi-

spectral cameras to achieve the extraction of field canal

distribution information using an object-oriented feature

combination hierarchical classification method (Han Wenting et

al, 2017a). However, the use of multi-spectral devices has

disadvantages such as high cost, low flight efficiency, small work

area, and difficulty in matching with visible light images.

Gao Lin realized the estimation of winter wheat leaf area index

based on digital images of drones (Gao Lin et al, 2016a). Guo

Peng used visible light images and selects brightness, saturation,

and red second-order moments as the best classification features

to classify crops, which is significantly higher than the color

index method (Guo Peng et al, 2017a). Kim used fixed-wing

drones to obtain orthophotos, multi-spectral images, and digital

surface models, using a random forest approach to classify land

cover types (Kim et al, 2017a). Wang Xiaoqin adopted the visible

light band image and proposed a new visible vegetation index

with visible light bands to achieve healthy green vegetation

information extraction. The visible light band can be used as a

data source for extracting crop planting information (Wang

Xiaoqin et al, 2015a). However, from the above studies, it is

found that the use of multi-spectral equipment is relatively

expensive, the flight efficiency is low, the work area is small, and

it is difficult to match the visible light image.

Yang Qi adopted a low-altitude remote sensing platform to carry

a high-definition digital camera and collected high-resolution

digital images of sugarcane during the whole growth period. The

CSMs at each growth stage were established and the plant heights

were extracted with and without ground control points. The

results showed that this method is feasible and the plant heights

extracted by CSMs have high accuracy (Yang Qi et al, 2017a).

Bendig conducted several studies based on a digital camera

platform equipped with a drone to obtain barley plant height,

confirming that the plant height extracted from CSMs has good

accuracy, and established an estimation model of barley plant

height and biomass (Bendig et al, 2014a). It can be seen that the

height of crops extracted from visible bands can be used as a

feature of crop classification.

In this experiment, we aimed to compare and analyse the

classification features of crop species by using high-resolution

images of UAVs and digital surface models to obtain farmland

cover classification maps. In particular, in this experiment, the

filter processing was performed on the DSMs, and the filtered

image was added to the classification process as the height

features of the crop, thereby improving the classification

accuracy. Finally, we compared and analysed the results of

different combinations of RGB features, texture features, and

height features.

2. AREA TEST AND DATA CAPTURE

The research area is the experimental base of Chinese Academy

of Agricultural Sciences, located in Minzhu Township, Daowai

District, Harbin. A variety of crops were planted in this research

area, and the terrain is undulating.

The data from this study was collected from the drone remote

sensing test from August 3 to August 4, 2017. The ground control

point data was marked and measured on August 3, 2017. The

drone image data collection date is August 4, 2017 using Fixed-

wing UAV with Sony Digital Camera.

3. CROP CLASSIFICATION METHOD AND PROCESS

This experiment was based on spectral, texture, and spatial

information and generated crop classification maps using the

SVM classifier. The specific steps are shown in Figure 1.

Figure 1. Research Process

3.1 The generation of DOM and DSMs

In Smart3D, Digital Orthophoto Map(DOM) and digital surface

models(DSMs) were produced using POS data and ground

control point data. The DOM that we chose had a spatial

resolution of 10 cm and 7500*7500.

Figure 2. DOM(left) and DSMs(right)

3.2 Feature Analysis and Combination

There are 12 types of cropland cover, including rice, corn, unripe

wheat, ripe wheat, harvested wheat, soybeans, trees, grassland,

bare land, roads, greenhouses and houses. After normalization,

trained SVM models and predicted pixel labels in LIBSVM.

Selected RBF kernel in this paper. The number of samples of

each type of feature is shown in Table 3.

The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-3, 2018 ISPRS TC III Mid-term Symposium “Developments, Technologies and Applications in Remote Sensing”, 7–10 May, Beijing, China

This contribution has been peer-reviewed. https://doi.org/10.5194/isprs-archives-XLII-3-1023-2018 | © Authors 2018. CC BY 4.0 License.

1024

Page 3: UAV-based Crops Classification with joint features from ......An aerial image classification method based on feature selection was proposed to realize the recognition of flooded areas

Label Name Training data

(pixels)

Reference

data(pixels)

1 Road 1500 200

2 Rice 1300 200

3 Greenhouse 600 200

4 Building 500 150

5 Corn 900 200

6 Tree 2000 200

7 Unripe wheat 1200 200

8 Ripe wheat 1000 100

9 Harvested land 600 100

10 Bare soil 1000 50

11 Grassland 920 100

12 Soybean 1220 100

Total 12740 1800

Table 3. Types of training data

3.2.1 Texture feature extraction

Texture feature is extracted by ENVI software. The mean,

variance, homogeneity, contrast, dissimilarity, entropy, second

moment, and correlation of RGB bands and DSM band are

obtained. Then the mean, variance (D), coefficient of variation

(V), and coefficient of difference between classes (𝐷𝑤 ) were

statistically analyzed for each type of sample. Selecting the

feature with a small coefficient of variation and a large difference

in classification coefficient and combined RGB to form new

training features.

𝐷 = 𝑆2 (1)

𝑉 =𝐷

M× 100% (2)

𝐷𝑤 =𝑀1−𝑀2

𝑀2× 100% (3)

where D = Variance

S= Standard deviation

Dw = coefficient of difference between classes

M = Mean

M1, M2= Mean of sample 1 and sample 2

Due to the lack of spectral bands and complex types of objects, a

single feature cannot completely distinguish all crops. Through

comparative analysis, we found that the second-order moment

texture of the green band had a smaller variance and coefficient

of variation, and can better distinguish between twelve different

types of ground objects and can be used as an effective feature.

Take the road as an example, calculate the eight features in the

green band, and calculate the difference coefficient with the other

eleven types of ground features(Table 4).

Table 4. Comparison of roads and other features in green band

features

Fea

ture

index

Eval

uat

ion V

alue

M

D

V

Dw(R

ice)

Dw(G

reen

house

) D

w(B

uil

din

g)

Dw(C

orn

) D

w(T

ree)

Dw(U

nri

pe

whea

t)

Dw(R

ipe

whea

t)

Dw(H

arves

ted

land)

Dw(B

are

soil

) D

w(G

rass

land)

Dw(S

oy

bea

n)

Mea

n

46.7

44 4

4.0

86

94%

163%

22%

24%

135%

101%

120%

86%

44%

49%

109%

101%

Var

iance

13.1

04 7

07.2

34 5

397%

186%

-6

5%

-4

5%

-6

8%

-7

2%

-2

3%

-6

5%

-4

1%

24%

-5

6%

-7

0%

Hom

ogen

eity

0.7

35

0.0

23

3%

38%

145%

17%

284%

103%

123%

165%

74%

35%

133%

192%

Contr

ast

2.6

60

21.6

33

813%

-2

8%

-8

1%

-5

5%

-9

3%

-7

7%

-7

7%

-8

7%

-5

7%

-5

6%

-8

1%

-9

0%

Dis

sim

ilar

ity

0.7

45

0.3

92

53%

-4

3%

-7

4%

-3

8%

-8

4%

-7

0%

-7

1%

-7

8%

-5

8%

-4

9%

-7

4%

-8

0%

Entr

opy

2.8

06

1.0

84

39%

-2

2%

-4

3%

-1

4%

-5

0%

-4

6%

-4

3%

-4

8%

-4

0%

-2

5%

-4

6%

-4

9%

Sec

ond

Mom

ent

0.1

49

0.0

14

9%

183%

1246%

40%

3348%

1623%

1310

%

1866%

1192%

220%

1510%

2746%

Corr

elat

ion

0.8

37

0.0

12

1%

47%

20%

18%

56%

-4

%

30%

13%

0%

15%

15%

17%

The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-3, 2018 ISPRS TC III Mid-term Symposium “Developments, Technologies and Applications in Remote Sensing”, 7–10 May, Beijing, China

This contribution has been peer-reviewed. https://doi.org/10.5194/isprs-archives-XLII-3-1023-2018 | © Authors 2018. CC BY 4.0 License.

1025

Page 4: UAV-based Crops Classification with joint features from ......An aerial image classification method based on feature selection was proposed to realize the recognition of flooded areas

3.2.2 Height Feature Extraction

Table 5. Comparison of roads and other features in DSMs

features

By comparison and analysis, the features of DSMs filter, variance

and contrast can better distinguish crop types and can be used as

effective distinguishing features. During the processing, the

DSMs contrast feature and the DSMs variance feature were

calculated using the 9*9 and 27*27 processing window. Take

the road as an example, calculate the eight features in the DSMs

filters images, and calculate the difference coefficient with the

other eleven types of ground features(Table 5).

After comparison and analysis from the above table, four kinds

of feature combinations are selected: the first one is to use RGB

features for classification, the second is to use RGB features and

second moments of green band for classification, the third

experiment is to use RGB features and DSMs for classification,

the fourth experiment is to use RGB, the second-order moment

of green band, and DSMs contrast and variance filter features for

classification.

3.3 Image classification by SVM

In the case of limited sample information, support vector machine

can well balance the complexity and learning of the model, and

has a good generalization ability. Therefore, it has been widely

used in remote sensing image classification research in recent

years, and has achieved good effect. First, the test combines the

features of the spectrum, texture, and height to determine the

combination of features. The composite features are normalized,

input SVM to train, and then use the particle swarm optimization

and grid search to find the optimal parameters and cross-

validation. Finally, the classification model was used to classify

different features and calculate the classification accuracy (Liu,

2017).

4. CLASSIFICATION RESULT

Figure 6. Original image and RGB feature

Figure 7. RGB and Green second moment features(left),RGB

and DSMs features(right)

Figure 8. RGB and DSMs Variance filter (left:9*9 processing

window, right 27*27 processing window)

Fea

ture

index

Eval

uat

ion V

alue

M

D

V

Dw(R

ice)

Dw(G

reen

house

) D

w(B

uil

din

g)

Dw(C

orn

) D

w(T

ree)

Dw(U

nri

pe

whea

t)

Dw(R

ipe

whea

t)

Dw(H

arves

ted

land)

Dw(B

are

soil

) D

w(G

rass

land)

Dw(S

oy

bea

n)

Mea

n

18.4

69 3

07.8

46 1

667%

309%

70%

-1

9%

-3

2%

-4

9%

-6

1%

-6

1%

18.4

69

307.8

46

1667%

1667%

Var

iance

0.1

24

0.2

90

23

5%

207%

-9

3%

-7

9%

23%

-9

7%

208%

291%

0.1

24

0.2

90

235%

235%

Hom

ogen

eity

0.9

85

0.0

05

1%

0%

3%

3%

1%

9%

0%

0%

0.9

85

0.0

05

1%

1%

Contr

ast

0.0

31

0.0

13

43%

171%

-7

3%

-6

2%

-4

%

-92%

152%

245%

0.0

31

0.0

13

43%

43%

Dis

sim

ilar

ity

0.0

22

0.0

02

11%

95%

-7

3%

-6

2%

-3

1%

-8

9%

81

%

148%

0.0

22

0.0

02

11%

11%

Entr

opy

0.2

66

0.1

62

61%

60%

-8

0%

-6

8%

-3

4%

-8

6%

55

%

98%

0.2

66

0.1

62

61%

61%

Sec

ond

Mom

ent

0.8

57

0.0

46

5%

-5

%

127%

50%

11%

254%

-5

%

-7%

0

.85

7

0.0

46

5%

5%

Corr

elat

ion

0.8

97

0.0

27

3%

-3

%

-5%

2%

10%

-4

%

-1%

-2

%

0.8

97

0.0

27

3%

3%

The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-3, 2018 ISPRS TC III Mid-term Symposium “Developments, Technologies and Applications in Remote Sensing”, 7–10 May, Beijing, China

This contribution has been peer-reviewed. https://doi.org/10.5194/isprs-archives-XLII-3-1023-2018 | © Authors 2018. CC BY 4.0 License.

1026

Page 5: UAV-based Crops Classification with joint features from ......An aerial image classification method based on feature selection was proposed to realize the recognition of flooded areas

Figure 9. RGB and DSMs contrast filter(left,27*27 processing

window),RGB, DSMs features and Green second moment

features (right,27*27 processing window)

Figure 10. RGB, DSMs, DSMs contrast and variance filter

The above figures show the classification results of various

combinations of features: When using only RGB features for

classification, many trees were mistakenly classified as corn, the

shadow of the greenhouse was wrongly divided into greenhouses,

and some buildings were mistakenly divided into roads. In the

classification results with the addition of green band texture

features, the shadows of many trees are mistakenly classified as

roads. With the addition of altitude information, the shadows of

trees and buildings are reduced, and the classification within the

block is more consistent. After adding the green band and DSMs

filtering, there are fewer fine points in the class, and there are

some trees that are mistakenly divided into buildings on the edge.

Combination of features overall

accuracy kappa

RGB 72.94 0.7

RGB, Second Moment of green band 83.39 0.82

RGB, DSMs 91.44 0.9

RGB, DSMs, DSMs variance (9*9) 92.56 0.92

RGB, DSMs, DSMs variance (27*27) 93.33 0.93

RGB, DSMs, DSMs contrast (27*27) 92.78 0.92

RGB, DSMs, DSMs variance (27*27),

DSM contrast (27*27) 93.11 0.92

RGB, DSMs, Second Moment of

green band, DSMs variance (27*27),

DSMs contrast (27*27)

94.5 0.94

Figure 11. Classification accuracy and kappa of different feature

combinations

In order to evaluate the classification results, the classification

accuracy and the confusion matrix are calculated, and kappa

coefficients are calculated from the confusion matrix(Figure 11).

First of all, the test classification accuracy based on the support

vector machine classification of the visible RGB band is 73.166%

and Kappa is 0.70. Second, after adding the second moment

feature of the green band, the accuracy of the classification

accuracy is 83.38% and Kappa is 0.82. Third, the four-

dimensional feature formed by adding DSMs test accuracy is

91.44% and Kappa is 0.90. Based on the RGB and DSMs bands,

addition of DSMs contrast filtering and variance filter. The

accuracy of the variance filter with window 9*9 added is 92.25%

and Kappa is 0.92, and the accuracy of the variance filter with

window 27*27 is 93.35% and Kappa is 0.93. The contrast filter

test accuracy of the 27*27 window is 92.77% and Kappa is 0.92.

The variance and contrast filtering accuracy of the 27*27

addition window is 93.11% and Kappa is 0.92. Finally, after

adding the second moment feature of the green band, the DSMs

variance and the contrast filtering feature, the detection accuracy

is 94.5% and Kappa is 0.94. Although the overall classification

accuracy is highest after adding the second moment of green band,

DSMs, DSMs variance and contrast features, the edges of the tree

are misclassified into buildings, and the second moment of green

band cannot provide effective improvement after adding DSMs.

The optimal feature combinations are RGB, DSMs, DSMs

variance and contrast features.

5. CONCLUSION

In this paper, we use drones equipped with a digital camera to

collect RGB images of the test site, and then those images are

processed in order to generate an orthoimage and a DSM of the

entire area. Differ to traditional UAV-based remote sensing

methods which mainly rely on spectra features from the RGB

images, spatial features extracted from the DSM data are

emphasized in our method. We adopt a typical SVM

classification framework where texture and spatial features

extracted from the orthoimage and DSM data are combined

together. Experiments results indicated that the overall

classification accuracy is dramatically improved when the spatial

features especially that of the altitude demission are introduced.

According to the experiments results, the contributions of spatial

features were as follows:

1. Besides the overall classification accuracy, the homogeneity

of each category was also improved at the same time;

2. Classes which are hard to separate in colour space, such as

tree, grass and crops, became distinguishable to each other

when features in the altitude demission was introduced. The

reason is, height value and its variance of each class is

different according to statistical analysis.

3. Misclassifications caused by shades in the RGB images was

largely reduced with the assistance of spatial features.

Although classification accuracy was much improved with the

proposed method, there are still tackle issues remained in this

field. For example, misclassification occurred in the edge of the

class due to the poor data quality of the DSM data. Usually at the

edge of objects, noises of the height value increased sharply,

which is quite common in such kind of data. Moreover, random

noises can be seen in almost every category in the final

classification results, because of a pixel by pixel classification

approach was used. Further research will be conducted to solve

those issues mentioned above. For example, number of features

by neighbourhood analysis should be increased in order to

overcome the drawbacks of pixel by pixel classification pattern.

Furthermore, other classification approaches such as deep

learning method should also be investigated in future. Finally, we

also plan to add other sensors such as low cost multi-spectral

sensor to increase the feature space for the sake of a better

performance in classification accuracy of crops.

REFERENCES

Adrien Michez, Hervé Piégay, Jonathan Lisein, Hugues

Claessens, Philippe Lejeune, 2016a. Classification of riparian

forest species and health condition using multi-temporal and

hyperspatial imagery from unmanned aerial system. Environ

Monit Assess, 188, pp. 148-165.

The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-3, 2018 ISPRS TC III Mid-term Symposium “Developments, Technologies and Applications in Remote Sensing”, 7–10 May, Beijing, China

This contribution has been peer-reviewed. https://doi.org/10.5194/isprs-archives-XLII-3-1023-2018 | © Authors 2018. CC BY 4.0 License.

1027

Page 6: UAV-based Crops Classification with joint features from ......An aerial image classification method based on feature selection was proposed to realize the recognition of flooded areas

Cao Weibin, Yang Bangjie,Pei Zhiyuan,Wang Qingfa and Wu

Quan, 2004a. Investigation and analysis of agricultural

condition information demand in China. Transactions of the

CSAE, 20(1), pp. 147-151.

Catur Aries Rokhmana, 2015a. The Potential of UAV-based

Remote Sensing for Supporting Precision Agriculture in

Indonesia. Procedia Environmental Science, 24, pp. 245-253.

Chen Zhongxin, Ren Jianqiang, Tang Huajun, Shi Yun, Leng

Pei, Liu Jia, Wang Limin, Wu Wenbin, Yao Yanmin, Hasi Tuya

2016a. Progress and perspectives on agricultural remote sensing

research and applications in China. Journal of Remote Sensing,

20(5), pp. 748-767.

Francisco Javier Mesas-Carrascosa, María Dolores Notario-

García, José Emilio Meroño de Larriva, Manuel Sánchez de la

Orden, Alfonso García-Ferrer Porras, 2014a. Validation of

measurements of land plot area using UAV imagery.

International Journal of Applied Earth Observation and

Geoinformation., 2, pp. 270-279.

Gao Lin, Yang Guijun, Li Hongjun, Li Zhenhai, Feng Haikuan,

Wang Lei, Dong Jinhui, He Peng, 2016a. Winter wheat LAI

estimation using unmanned aerial vehicle RGB-imaging.

Chinese Journal of Eco-Agriculture, 24(9), pp. 1254-1264

Giovanna Sona, Daniele Passoni, Livio Pinto, Diana Pagliari,

Daniele Masseroni, Bianca Ortuani, Arianna Facchi.(2016):

UAV MULTISPECTRAL SURVEY TO MAP SOIL AND

CROP FOR PRECISION FARMING APPLICATIONS. In:

ISPRS Annals of the Photogrammetry, Remote Sensing and

Spatial Information Sciences XLI-B1, pp. 1023-1029

Guo Peng, Wu Fadong, Dai Jianguo , Wang Haijiang, Xu

Liping, Zhang Guoshun, 2017a. Comparison of farmland crop

classification methods based on visible light images of

unmanned aerial vehicles. Transactions of the CSAE, 33(13),

pp. 112-119.

Han Wenting, Zhang Liyuan, Zhang Haixin, Shi Zhiqiang,

Yuan Mengchan, Wang Zijun, 2017a. Extraction Method of

Sublateral Canal Distribution Information Based on UAV

Remote Sensing. Transactions of the Chinese Society for

Agriculture Machinery, 48(3), pp.205-214.

Huang Qing, Wang Di, and Liu Jia, 2009a. Investigation and

analysis of agricultural condition information demand in China.

Chinese Journal of Agricultural Resources and Regional

Planning, 30(2), pp. 13-17.

J. Senthilnath, Manasa Kandukuri, Akanksha Dokania, K.N.

Ramesh, 2017a. Application of UAV imaging platform for

vegetation analysis based on spectral-spatial methods.

Computers and Electronics in Agriculture, 140, pp. 8-24.

Juliane Bendig, Andreas Bolten, Simon Bennertz, Janis

Broscheit, Silas Eichfuss and Georg Bareth, 2014a. Estimating

Biomass of Barley Using Crop Surface Models(CSMs) Derived

from UAV-Based RGB Imaging. Remote Sens, 6(11), pp.

10395-10412.

Kim, Gu Hyeok, Choi, Jae Wan, 2017a. Land Cover

Classification with High Spatial Resolution Using Orthoimage

and DSM Based on Fixed-Wing UAV. Journal of the Korean

Society of Surveying, Geodesy, Photogrammetry and

Cartography, 35(1), pp. 1-10.

Liu Jia, Wang Limin, Teng Fei, Li Dandan, Wang Xiaolong,

and Cao Huaitang, 2015a. Crop area ground sample survey

using Google Earth image-aided. Transactions of the CSAE,

31(24), pp. 149-154.

Oumer S. Ahmed, Adam Shemrock, Dominique Chabot, Chris

Dillon, Griffin Williams, Rachel Wasson, Steven E. Franklin,

2017a. Hierarchical land cover and vegetation classification

using multispectral data acquired from an unmanned aerial

vehicle. Journal of Remote Sensing, 38(8-10), pp. 2037-2052.

Patricia K.Freeman, Robert S.Freeland, 2015a. Agricultural

UAVs in the U.S.: potential, policy, and hype. Remote Sensing

Applications: Society and Environment, 2, pp. 35-43.

Pena, J.M., Torres-Sanchez, J., de Castro, A.I., Kelly, M. and

Lopez-Granados, F., 2013a. Weed mapping in early-season

maize fields using object-based analysis of unmanned aerial

vehicle (UAV) images. PLoS One, 8(10), pp. e77151.

Rossana Gini, Daniele Passoni, Livio Pinto, Giovanna Sona,

2014a. Use of Unmanned Aerial Systems for multispectral

survey and tree classification: a test in a park area of northern

Italy. European Journal of Remote Sensing, 47(1), pp. 251-269.

Ryoichi Doi, 2016a. Improved discrimination among similar

agricultural plots using red-and-green-based pseudo-colour

imaging. Int. Agrophys, 30(2), pp. 151-163.

Susana Del Pozo, Pablo Rodríguez-Gonzálvez David

Hernández-López and Beatriz Felipe-García, 2014a. Vicarious

Radiometric Calibration of a Multispectral Camera on Board an

Unmanned Aerial System. Remote Sens, 6(3), pp. 1918-1937.

Wang Xiaoqin, Wang Miaomiao, Wang Shaoqiang, Wu

Yundong. 2015a. Extraction of vegetation information from

visible unmanned aerial vehicle images. Transactions of the

CSAE, 31(5), pp. 152-157.

Xiuliang Jin, Shouyang Liu, Frédéric Baret, Matthieu Hemerlé,

Alexis Comar, 2017a. Estimates of plant density of wheat crops

at emergence from very low altitude UAV imagery. Remote

Sensing of Environment, 198, pp. 105-114.

Yang bangjie.Lu Denghuai, Pei Zhiyuan,Zhao Hanjie, and Wu

Yanyao., 1997a. The Structure Design of a National Crop

Condition Monitoring System. Transactions of the CSAE, 13(1),

pp. 22-25.

Yang Qi, Ye Hao, Huang Kai, Zha Yuanyuan, Shi Liangsheng,

2017a. Estimation of leaf area index of sugarcane using crop

surface model based on UAV image. Transactions of the CSAE,

33(8), pp. 104-111.

Zhang Jiankang, Cheng Yanpei, Zhang Fawang, Yue Depeng,

Guo Xiaoxiao, Dong Hua, Wang Jiping, Tang Hongcai, 2012a.

Crops planting information extraction based on multi-temporal

remote sensing images. Transactions of the CSAE, 28(2), pp.

134-141.

The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-3, 2018 ISPRS TC III Mid-term Symposium “Developments, Technologies and Applications in Remote Sensing”, 7–10 May, Beijing, China

This contribution has been peer-reviewed. https://doi.org/10.5194/isprs-archives-XLII-3-1023-2018 | © Authors 2018. CC BY 4.0 License.

1028