Is 80% Accuracy

Good Enoughpresented by

Charles E. Olson, Jr.Senior Image Analyst

Michigan Tech Research InstituteAnn Arbor, Michigan

Prepared for the ASPRS Semi-Annual Conference, Denver, CO, November 2008

Accuracy can mean different things to different people.

This paper addresses only accuracy of thematic data.

Accuracy can mean different things to different people.

This paper addresses only accuracy of thematic data.

The Interagency Steering Committee on Land Use Interpretation and Classification wrote:

“The minimum level of interpretation accuracy in the identification of land use and land cover categories from remote sensor data should be at least 85 percent.”

(Anderson, et al., 1971).

Accuracy means different things to different people.

This paper addresses only accuracy of thematic data.

The Interagency Steering Committee on Land Use Interpretation and Classification wrote:

“The minimum level of interpretation accuracy in the identification of land use and land cover categories from remote sensor data should be at least 85 percent.”

(Anderson, et al., 1971).

At the time this proposal was made, this level of accuracy was routine and it was essentially a “lowest common denominator” approach.

Washtenaw County Mapping Project

Photo Scale: 1:24,000

Compilation Scale: 1:48,000

Classification Scheme

Based on U.S. Geological Survey Circular 671

Hierarchical to Level IV

Nine Level I Classes

Twenty-eight Level II Classes

Fifty-nine Level III Classes

More than one thousand Level IV Classes

Accuracy

96% at Level II

92% at Level III

How did we do it?

By using all of the Image Interpretation “tools”

Existing maps - sources of existing data about the terrain.

Stereoscopes - viewing the photos and seeing the terrain in 3D.

Scales - for measuring and determining feature size.

Terrain sense - understanding the multiple facets of any terrain.

- animals, especially the human animal

- soils and rocks

- vegetation

- topography

- water

The EIIs - elements of image interpretation

Computer screens are today’s medium of choice

Human interpreters do on their computer screens what we

used to do with paper prints.

But the interpretation process remains the same.

Computer users can drape maps over their image data, can measure and determine sizes, can view the terrain in 3-D, can use the EIIs (but seldom do), and seldom have the broad sense of terrain required of a good interpreter.

Shape Shadow Site

Size Pattern Association

Tone Texture Resolution

Of these nine EIIs, computer-based interpretation algorithms

rely almost completely on just one: Tone

The Elements of Image Interpretation (Olson, 1960)

Shape Shadow Site

Size Pattern Association

Tone Texture Resolution

Of these nine EIIs, computer-based interpretation algorithms

rely almost completely on just one: Tone

The Elements of Image Interpretation (Olson, 1960)

When used with the deductive process that Bob Colwell (1954) called “… the convergence of evidence … “ useful results almost always follow.

Agricultural Fields near Ann Arbor, MI September 11

University of Michigan photo

Map Date: 1954

Photo Date: 1952

What happened to these small “ponds?”

and

Are they significant?

Agricultural Fields near Ann Arbor, MI September 11

University of Michigan photo

Low oblique photos taken in 1983.

Changing Relationships

Rapid growth of computer based interpretation systems has resulted in lowering minimum level of accuracy to 80%.

Cost of computer hardware and software, and the personnel to keep it running, often leaves no funds to do it any other way.

If the computer can’t do it, it can’t be done.

Rapid rise in GIS usage.

As long as it fits neatly into the data base, it’s OK.

Computer processing is often faster than manual processing.

Is that speed really necessary when it comes at the cost of reduced accuracy?

How Much Accuracy Do We Really Need?

Is 80% actually good enough?

The market place says YES.

Clients are willing to buy it.

What is the long-range cost?

A 1% increase can be worth $10 million.

Will providing poorer data to decision makers come back to haunt our profession?

The market says YES,

because clients buy it.

The market says YES,

because clients buy it.

But we can do better.

95% accuracy used to be routine.

The market says YES,

because clients buy it.

But we can do better.

95% accuracy used to be routine.

If we didn’t get 95% accuracy we didn’t get paid.

Thank you for listening.

I will be happy to answer questions if time permits, or after the session is over.

Prepared for the ASPRS Semi-Annual Conference, Denver, CO, November 2008