SEReGAP Land Cover Mapping Summary and Results

Southwest Regional GAP ProjectArizona, Colorado, Nevada, New Mexico, Utah

US-IALE 2004, Las Vegas, Nevada: Transdisciplinary Challenges in Landscape Ecology

R. Douglas Ramsey, John Lowry, Jessica Kirby, Wendy Rieth, Lisa Langs

Remote Sensing and GIS LaboratoryUtah State University

Logan, Utah

Park ValleyPark ValleyPark Valley

AGRICULTURE

GRASSLAND

GREASEWOOD

LOWLAND RIPARIAN

PICKLEWEED BARRENS

SAGEBRUSH

SAGEBRUSH/PERENNIAL GRASS

SALT DESERT SCRUB

WATER

PERENNIAL GRASSLAND

ANNUAL GRASSLAND

PLAYA

BIG SAGEBRUSH SHRUBLAND

XERIC MIXED SAGEBRUSH

MIXED SALT DESERT SCRUB

SEMI-DESERT SHRUB STEPPE

SEMI-DESERT GRASSLAND

LOWER MONTANE RIPARIAN WOODLAND AND SHRUBLAND

GREASEWOOD FLAT COMPLEX

CULTIVATED CROPS/IRRIGATED AG

PASTURE/HAY/NON-IRRIGATED AG

ANNUAL FORBLAND

1995 GAP 30 M 2003 GAP 30 M1995 GAP 100 Ha

Edgematching

ACCURACY ASSESSMENT• Accuracy assessment is based on a 20% sample of

training sites for each mapped type.

• Since the 20% sample is not independent, this is not considered a true accuracy assessment.

• The 20% sample or validation set is not used in the classification tree model development.

• In addition to a validation with a 20% sample, we have supported our findings with a manual 5-fold cross validation with sample replacement to test model stability and help understand the effects of validation sample size on the model stability results.

Each site is considered correctly classified if the majority of pixels agree with sample polygon

Validation: Lower Wasatch Range MZ

Overall Accuracy: 65% - 24 typesKappa: 0.61Standard error of kappa: 0.01Z-Score for kappa: 46.73

Accuracy for Mapping Zone UT04

Accuracy for Mapping Zone UT02

Overall Accuracy: 69% - 51 types

Overall Accuracy: 60% - 32 typesKappa: 0.56

Standard error of kappa: 0.008

Z-Score for kappa: 66.61

Accuracy for Mapping Zone UT03

D06 D08 D09 S009 S012 S015 S023 S024 S034 S040 S046 S050 S054 S055 S065 S071 S065 S079 S086 S093 S096 S100

trial A 0.27 0.23 0.22 0.13 0.00 0.62 0.50 0.71 0.73 0.79 0.47 0.00 0.50 0.25 0.53 0.07 0.00 0.33 0.22 0.17 0.38 0.75trial B 0.20 0.18 0.06 0.00 0.33 0.61 0.50 0.75 0.64 0.76 0.21 0.17 0.53 0.25 0.42 0.20 0.00 0.32 0.22 0.17 0.31 0.75trial C 0.21 0.23 0.17 0.25 0.00 0.46 0.25 0.38 0.82 0.75 0.07 0.00 0.53 0.30 0.51 0.27 0.00 0.32 0.13 0.17 0.33 0.33trial D 0.07 0.33 0.17 0.13 0.00 0.76 0.50 0.75 0.80 0.76 0.36 0.00 0.60 0.36 0.57 0.13 0.00 0.24 0.00 0.17 0.26 0.60trial E 0.33 0.15 0.11 0.38 0.33 0.69 0.25 0.63 0.55 0.81 0.20 0.17 0.64 0.32 0.55 0.13 0.00 0.27 0.00 0.00 0.21 0.60mean 0.22 0.22 0.15 0.18 0.13 0.63 0.40 0.64 0.71 0.78 0.26 0.07 0.56 0.30 0.52 0.16 0.00 0.30 0.11 0.13 0.30 0.61sd 0.09 0.06 0.06 0.13 0.16 0.10 0.12 0.14 0.10 0.02 0.14 0.08 0.05 0.04 0.05 0.07 0.00 0.04 0.10 0.07 0.06 0.15CV 0.40 0.27 0.38 0.73 1.22 0.16 0.31 0.22 0.15 0.03 0.52 1.22 0.09 0.15 0.10 0.42 0.00 0.12 0.86 0.50 0.19 0.25Sample Size Comparison

n Trial A 15 39 18 8 3 26 4 7 11 85 15 6 96 44 112 15 0 63 23 6 42 4n Trial B 15 40 17 8 3 28 4 8 11 85 14 6 96 44 112 15 0 63 23 6 42 4n Trial C 14 39 18 8 3 28 4 8 11 85 14 6 96 44 112 15 0 63 23 6 42 3n Trial D 14 40 18 8 2 25 4 8 10 85 14 6 95 44 112 15 0 63 22 6 42 5n Trial E 15 40 18 8 3 26 4 8 11 85 15 6 96 44 112 15 0 63 23 6 42 5mean 14.6 39.6 17.8 8.0 2.8 26.6 4.0 7.8 10.8 85.0 14.4 6.0 95.8 44.0 112.0 15.0 0.0 63.0 22.8 6.0 42.0 4.2

Summary Statistics, 5-fold Validation

This is a 5-fold cross validation with sample replacement to test model stability and help understand the effects of validation sample size on the model stability results.

Sample Size vs Accuracy and Coefficient of Variation

0.00

0.20

0.40

0.60

0.80

1.00

1.20

1.40

0.0 20.0 40.0 60.0 80.0 100.0 120.0

Average Sample Size

Accuracy and CV

Mean Accuracy Accuracy CV

0

20

40

60

80

100

120

0.00% 2.00% 4.00% 6.00% 8.00% 10.00% 12.00% 14.00% 16.00% 18.00%

% of Area

Validation Sample Size

% of area mapped vs. available sample size

(Great Salt Lake Desert)

0

10

20

30

40

50

60

70

80

90

100

0.00% 2.00% 4.00% 6.00% 8.00% 10.00% 12.00% 14.00% 16.00% 18.00%

% of Area

Validation Level (~Accuracy)

< 2% of area mapped > 2% of area mapped

% of area mapped vs. accuracy level

Users AccuracyMapping Zones UT2, 3, and 4

D04

D06

D08

D09

S001

S002

S010

S011

S023

S024

S025

S028

S030

S032S036

S039S040

S042

S043

S045

S046

S050

S052

S056

S059

S065S071

S078

S079

S081

S083

S085

S090

S091

S093

S096

S102

S133

S136

S031

S012

S054

S006

S004

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

% Accuracy

< 20 samples> 20 samples> 40 samples

Conclusions• Current validation shows between 60-65% “accuracy” for completed

mapping zones.

• Variance of accuracies is directly linked to validation sample size.

• A validation set of 40 points seems to provide some stability to model results.

• A minimum of 40 samples for validation per cover type should be a target size. This is difficult due to the preponderance of “rare” types and the lack of training sites.

• Using a 20% validation set, we need a minimum of 200 field sites per mapped type. This is difficult given the availability of unique sample sites for a particular cover type.

• Overall map “accuracy” seems to coincide with expected results given the map detail.

• Rare ecological systems and systems that consist of a wide range of cover types (I.e. shrub steppe types) tend to lower accuracies.

Acknowledgements