IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL ...€¦ · IDENTIFICATION OF...

46
IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL NETWORKS: A CONTEXTUAL APPROACH Julian E. Boggess Computer Science Department, Mississippi State Univer P. O. Drawer CS, Mississippi State, MS 39762, U.S.A. (601) 325-2756 [email protected] Abstract - Humans can fairly easily identify roads in remote sensing images, but this has turned out to be a difficult task fo computers. Most previous work in this area has utilized statist and rule-based techniques, which depended primarily upon s information. However, it appears that spectral information alon insufficient to identify roads in Landsat Thematic Mapper satel imagery, since soils have the same spectral signature in the dat roads, and that contextual information is required. In this application, artificial neural networks are found to be superior several previous techniques due in part to their ability to inco both spectral and contextual information. However, a number factors cause problems for the network, and further work must done to include additional information; it is suggested that a h system might alleviate most of these difficulties. Key words : Artificial neural networks, Satellite imagery classification, Artificial intelligence

Transcript of IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL ...€¦ · IDENTIFICATION OF...

Page 1: IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL ...€¦ · IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL NETWORKS: A CONTEXTUAL APPROACH

IDENTIFICATION OF ROADS IN SATELLITE IMAGERYUSING ARTIFICIAL NEURAL NETWORKS:

A CONTEXTUAL APPROACH

Julian E. Boggess

Computer Science Department, Mississippi State University,P. O. Drawer CS, Mississippi State, MS 39762, U.S.A.

(601) [email protected]

Abstract - Humans can fairly easily identify roads in remote

sensing images, but this has turned out to be a difficult task for

computers. Most previous work in this area has utilized statistical

and rule-based techniques, which depended primarily upon spectral

information. However, it appears that spectral information alone is

insufficient to identify roads in Landsat Thematic Mapper satellite

imagery, since soils have the same spectral signature in the data as

roads, and that contextual information is required. In this

application, artificial neural networks are found to be superior to

several previous techniques due in part to their ability to incorporate

both spectral and contextual information. However, a number of

factors cause problems for the network, and further work must be

done to include additional information; it is suggested that a hybrid

system might alleviate most of these difficulties.

Key words: Artificial neural networks, Satellite imagery

classification, Artificial intelligence

Page 2: IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL ...€¦ · IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL NETWORKS: A CONTEXTUAL APPROACH

INTRODUCTION

The purpose of this paper is to report the results of applying

artificial neural networks to the task of detecting road pixels in

satellite images. It was found that providing the neural network

with just the spectral data for a given pixel is insufficient evidence

for the network to be able to judge whether the pixel is a road pixel

or not. However, if contextual information is provided, in the form of

a 5-pixel by 5-pixel square surrounding (and including) the pixel to

be classified, a neural network is capable of achieving useful results.

This report will begin with a brief summary of previous

attempts to identify roads in aerial and satellite imagery, using

various techniques, as well as efforts to employ artificial neural

networks for land-use classification from satellite imagery, followed

by a discussion of previous efforts to use neural networks to identify

roads from satellite imagery. Next, the designs of the experiments

reported in this study will be discussed, and their results presented.

The results will then be analyzed, and four problem areas will be

discussed. The paper will conclude with suggestions for improving

the performance of the road extraction system and proposals for

future work.

MOTIVATION

Road networks in satellite images are generally readily

discerned by the eye of the average human observer. Unfortunately,

Page 3: IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL ...€¦ · IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL NETWORKS: A CONTEXTUAL APPROACH

roads are difficult to extract automatically from satellite imagery,

and numerous man-hours are required to extract them by hand. In

the current world political climate, it is important be able to provide

military and other groups with accurate, up-to-date maps of the road

networks in any region of the world. For this reason, there is strong

motivation to develop more powerful algorithms for automatic road

identification.

Artificial Intelligence algorithms derive their power from the

use of domain-specific knowledge. Several different types of

Artificial Intelligence approaches to road detection are available,

including Expert Systems, Fuzzy Set Theory, Heuristic Search, and

Artificial Neural Networks. The current study extends previous work

on finding roads using artificial neural network techniques by

incorporating knowledge of spatial relationships (context) with the

more-normally used spectral information. Although some useful

results are obtained, it is probable that a complete solution to the

problem will require the integration of several AI techniques into a

hybrid system; some possible hybrid approaches are described at the

end of this report.

Several types of satellite-based remote sensing data are now

available. Landsat Thematic Mapper (TM) data was used for this

study because it is more readily available and cheaper than SPOT

data, and because the TM data has 7 bands of spectral information as

compared to 3 for the SPOT data.

Page 4: IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL ...€¦ · IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL NETWORKS: A CONTEXTUAL APPROACH

BACKGROUND

Previous attempts to find roads in satellite imagery generally

have approached the problem by using one of two types of

information about roads: their spectral characteristics, and their

linear spatial structure. Unfortunately, roads share spectral

characteristics with other types of terrain in satellite images (plowed

fields, clear-cut forest areas, any bare ground), and thus cannot

reliably be identified from spectral data alone. Moreover, many

linear features occur in satellite images (barges, wakes, furrows,

clearings for electrical transmission lines, etc.), and therefore

techniques which identify linear features cannot reliably find roads.

Even a merger of these two approaches can be misled by river banks,

furrows, dry stream beds, field boundaries, levees, river meander

scars, and the like. For this reason, previous attempts to identify

roads in satellite images have met with only moderate success, and

the problem continues to inspire a considerable amount of research.

Detecting roads and linear structures in remote sensing imagery

Quite a few papers have appeared in the last 15 years

concerning the problem of detecting roads and other linear

structures in aerial photographs and satellite images. It is possible to

detect edges in images using artificial neural networks (for example,

see Bhatia et al., 1991), as well as statistical analysis (see Lashlee and

Naugle, 1990), but these techniques in and of themselves do not

identify roads, since many other types of lines and edges appear in

satellite images. Moreover, techniques which find any linear feature

Page 5: IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL ...€¦ · IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL NETWORKS: A CONTEXTUAL APPROACH

are generally not optimal for identifying roads; algorithms restricted

solely to identifying roads would be more useful.

A number of previous studies which provide algorithms for

detecting roads in aerial or satellite imagery supplement the

information used in the algorithm with data which is not available

strictly from the image itself. For example, Van Cleynenbreugel

(1990) used Geographical Information Systems (GIS) information

(e.g., topographic map data) to aid in delineating roads in SPOT

images. Fischler et al. (1981) employed a type of expert system

approach, integrating information from several sources in order to

improve the ability of the system to detect roads. Maillard and

Cavayas (1989) use extant maps as a starting point for the extraction

of new roads from SPOT images, as part of the process of updating a

cartographic database. Since these techniques require additional

information which may not be available in particular situations, they

are not suitable as general solutions to the road identification

problem.

Others studies (Ton et al. 1989) utilize only the data provided

in the original image. Duda (Duda and Hart, 1973), for example,

defines a road operator which identifies lines that contrast with their

surroundings. Bajcsy and Tavakoli (1976), define roads by their

function, and use what they thereby deduce must be true about the

appearance of the roads in ERTS-1 satellite images (plus spectral

information) to identify the roads. Wharton (1987) presents a set of

rules for detecting water and vegetative areas, thereby reducing the

number of pixels that must be examined as possible road pixels.

Vasudevan et al. (1988) describes a multi-level system of heuristics

Page 6: IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL ...€¦ · IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL NETWORKS: A CONTEXTUAL APPROACH

in which each higher level contributes to the construction and

identification of roads in Landsat TM images from information

provided by the previous level. Wang and Newkirk (1988) and

Moller-Jensen (1990) take these approaches a step further and

attempt to find roads in Landsat TM images by using knowledge-

based expert system techniques.

Use of artificial neural networks in remote sensing

Several papers have appeared recently in which artificial

neural networks are used to identify or classify some components

appearing in aerial photographs or satellite images. Dreyer (1993)

reported on the successful use of optimized neural networks to

classify land-cover into four categories (such as forest, meadow, etc.).

Ryan et al. (1991) utilized neural networks to extract shoreline

features in aerial photographs. Dawson et al. (1993) discussed the

use of neural networks to retrieve various characteristics, including

roughness, of a surface such as sea ice.

Hepner and Ritter (1989) demonstrated that neural networks

could successfully determine land-cover classes; Ritter and Hepner

(1990) applied neural network techniques in land-cover

classification of TM imagery. Augusteijn et al. (1993) showed that

neural networks performed well whether given extracted features as

input or given non- (or slightly-) preprocessed data as input.

Augusteijn and Dimalanta (1992) utilized a variation of the

backpropagation neural network (the Cascade-Correlation network)

to classify cloud cover categories; they also investigated the use of

the Kohonen map to determine the categories inherent in the data,

Page 7: IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL ...€¦ · IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL NETWORKS: A CONTEXTUAL APPROACH

but found it to be of no help. Lee et al. (1990) and Slawinski et al

(1991) also employed neural networks to assist in cloud detection

and classification.

Liu and Xiao (1991) felt that a standard backpropagation

neural network suffered from slowness of convergence when being

trained to classify TM data, but that a Blocked Backpropagation

network performed better. Safavian and Tenorio (1991) also found a

standard backpropagation network to be useful for ground cover

classification with Airborne Multi-spectral Scanner (MSS) images and

other data, but felt that it did not work as well as a self-organizing

type of neural network.

A comparison of neural network techniques with the more

standard statistical classification methods was discussed in a number

of studies. Key et al. (1989) reported that a neural network out-

performed a maximum-likelihood classifier in identifying four

surface and eight cloud classes. Kanellopoulos et al. (1992) also

found that a neural network used to classify land-cover in SPOT

imagery performed significantly better than a maximum-likelihood

classifier on the same data. Hepner et al. (1990) showed that

backpropagation neural network techniques performed better than

conventional supervised classification techniques, given a relatively

small amount of training data. Benediktsson et al. (1990) found that

neural network techniques could have some advantages over

standard statistical classification methods, although the standard

methods might be superior in some circumstances. Omatu and

Yosida (1991) reported that a neural network approach produced

Page 8: IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL ...€¦ · IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL NETWORKS: A CONTEXTUAL APPROACH

better results than Bayesian classification in experiments classifying

Landsat TM data.

Use of neural networks for identification of roads

No previously-published literature was found which deals

primarily with the use of neural networks for the extraction of roads

from satellite imagery. One unpublished paper (Mason et al., 1993)

reports some preliminary data on this topic, but the results are not

very robust as of yet. One interesting study (Smith and Austin,

1992) discusses the use of an associative memory artificial neural

network to distinguish a road from a field or urban background; this

was only one of three tasks described, so details are sketchy, but this

technique merits further investigation. Attempts to survey previous

work in this area are continuing.

CURRENT STUDY

Scope

The scope of the study reported below was to investigate the

use of artificial neural networks in identifying roads in Landsat

Thematic Mapper images. One goal of the project, in order to have a

method of visually determining whether the results were

satisfactory, was to devise an automated process which could accept

as input a multi-spectral TM image and produce as output an image

of the road network suitable for overlaying on top of the original

image.

Page 9: IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL ...€¦ · IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL NETWORKS: A CONTEXTUAL APPROACH

A Landsat TM image of the Vicksburg, Mississippi, area taken

on 1 April 1991 served as a testbed. This area was chosen due to its

proximity to the investigator and the consequent ease with which

ground-truthing could be performed. A 400-by-400-pixel section of

the original image was used for the experiments described below; the

terrain within this section included several roads of different types

(four-lane interstate highway I-20, U. S. highways 61 and 80, state

and county highways, urban and suburban streets, and small local

roads), as well as various types of terrain, including hills, flat delta

farmland, bare and waterlogged soils, meander scars from the

Mississippi river, forests, lakes, streams, linear areas cleared for

electrical transmission lines, and other types of land features. Data

from all seven TM spectral bands were used, including the far-

infrared thermal band, band 6, which has a spatial resolution of 120

meters per pixel; the other six spectral bands have a resolution of 30

meters. The 120-meter pixels in band 6 were resampled to produce

equivalent 30-meter pixels, for compatibility with the other bands.

Landsat TM spectral data have a dynamic range of 8 bits and

thus theoretically can range in integer values from 0 to 255,

indicating the intensity of response of the sensor within a range of

wavelengths to light reflected from a small area on the earth's

surface. However, the data obtained for each band may not contain a

full range of responses; if the data for one of these bands is displayed

as an image, the image may appear totally dark, or with only a very

small range of brightness (low contrast) over most of the image.

Therefore, a histogram equalization is often performed to permit the

brightness range of the data for this band to be stretched in

Page 10: IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL ...€¦ · IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL NETWORKS: A CONTEXTUAL APPROACH

proportion to the frequency with which the data occurs in a

particular image; when stretched in this way, contrast is enhanced

and the features in the image are much more easily discernible. This

is a commonly-used procedure in image processing, and image

processing software usually provides a means for performing this

procedure quickly and easily. The data for each of these bands was

processed in this manner.

Figures 1 through 7 show the appearance of the experimental

test area in the seven TM spectral bands after histogram processing.

Table 1 shows the spectral sensitivity of the seven TM bands.

BAND BOUNDARIES (micrometers)COLOR

1 0.45 - 0.52 blue

2 0.52 - 0.60 green

3 0.63 - 0.69 r e d

4 0.76 - 0.90 reflective-infrared

5 1.55 - 1.75 mid-infrared

6 10.40 -12.50 thermal (emission) infrared

7 2.08 - 2.35 mid-infrared

Characteristics of the Landsat Thematic Mapper Spectral Bands

(adapted from Jensen, 1986, p. 34, and Richards, 1986, p. 13)

Table 1

Page 11: IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL ...€¦ · IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL NETWORKS: A CONTEXTUAL APPROACH

Methodology

Two types of artificial neural networks were used, including

three versions of the backpropagation network (a hand-coded

version, the Aspirin/MIGRAINES system from Mitre Corporation, and

NevProp, a version of Scott Fahlman's Quickprop), plus a self-

organizing feature map (Kohonen's SOM-PAK). None of the versions

of backpropagation differed greatly in results from any of the others.

In addition, a rule-based technique (equivalent to a simple expert

system) was devised to illustrate how well spectral-based

characteristics alone could serve to identify roads. These artificial

intelligence techniques were compared to results obtained by

standard statistical techniques employed by professional remote

sensing personnel.

Backpropagation Neural Network without Context

The first experiment was designed to determine whether or not

spectral information alone would be sufficient to allow a

backpropagation network to accurately ascertain whether a given

pixel should be classified as a road pixel or not. Five hundred pixels

were hand-sampled from the 160,000 pixels in the 400 by 400 test

image; of these 500 pixels, half were road and half were non-road.

The pixels were chose to represent all of the major terrain features

and land-use classes appearing in the image. Road pixels were

selected for use as training data only if they were clearly parts of

roads; pixels representing road margins were not selected, nor were

pixels from road segments which, for various reasons, did not stand

out clearly from the surrounding terrain. A number of pixels were

Page 12: IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL ...€¦ · IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL NETWORKS: A CONTEXTUAL APPROACH

selected from areas which were suspected of having a high

probability of being confused with roads, including patches of soils,

river banks, field boundaries, and the like.

Ninety percent, or 450 (consisting of 225 road and 225 non-

road pixels), were used as the training set, while 10 percent, or 50

pixels (25 road and 25 non-road), were reserved for use as members

of the test set. Trial runs showed that the order of presentation of

the training data did not significantly affect the results produced by

the network, so most runs were performed by presenting the road

pixels first, followed by the non-road pixels, for each epoch of

training.

The input data for this experiment consisted of values from all

seven TM spectral bands for each of the pixels in the training set.

The range of values was stretched using the histogram analysis

technique described above. The integer values from 0 to 255 were

then normalized to real numbers between 0.0 and 1.0 in order to be

used as input to the neural network.

A backpropagation network with 7 input units, 50 hidden

units, and 1 output unit was constructed; the weights of the

connections between the layers were randomly initialized, as usual.

As the network was trained, each of the 7 input units received as

input the normalized value from a different TM band for the next

pixel in the training set; thus input 1 would receive the normalized

value from TM Band 1 for the first pixel in the training set, while

input 2 would receive the value from Band 2, and so on. These

values would then be propagated through the network, producing an

output at the single output unit which would have a value between

Page 13: IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL ...€¦ · IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL NETWORKS: A CONTEXTUAL APPROACH

0.0 and 1.0, with the value reflecting the network's "best guess" as to

whether the pixel was a road pixel or not, or, more accurately, the

degree of confidence the network had that the pixel was a road. The

training set also contained data which specified whether a given

pixel was, in fact, a road pixel or not, and the network would

compare this information to its output and then propagate the error

information back though the layers of the network, adjusting the

weights of the connections in order to reduce the error when the

network was exposed to the same or similar input at a later time.

The network was exposed to all 450 items of the training data

one time (which constituted an epoch of training), and the root mean

square (RMS) error was computed, which indicated how far the

network was from an exact solution. The training process was

repeated for a number of epochs, during which the RMS error was

observed to decrease more or less steadily. When, after many

epochs of training, the RMS error began to increase, training was

terminated, and the network was tested.

As was expected, the network performed poorly when given

only the spectral information from the 7 TM bands for a single pixel;

the accuracy rate on the test set was only 55%. One reason for this

relatively poor performance is that road pixels and soil pixels

frequently have the same spectral reflectance characteristics, which

makes it impossible for this network to distinguish between roads

and soils; a number of soil pixels were deliberately included in the

test set, and they evidently confused this network. Moreover, road

pixels almost always actually consist only partially of road surface,

with the rest of the 30 by 30 meter area containing vegetation,

Page 14: IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL ...€¦ · IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL NETWORKS: A CONTEXTUAL APPROACH

water, or other substances; the average spectral reflectance

characteristics of these road pixels may well resemble the

characteristics of the other, predominant substances in the area more

closely than those of roads. For these reasons, presenting a neural

network with only the spectral information about a single pixel does

not allow it to accurately determine whether it is a road or a non-

road pixel.

Backpropagation Neural Network with Context

Humans are very good at identifying roads in remote sensing

images, and apparently depend upon context to enable them to

disambiguate and identify confusing image segments. For example,

the human visual system can identify roads in remotely-sensed

images by noticing the characteristic contrast of a narrow ribbon of

road with the larger patches (usually similar to one another) of

vegetation or other material through which the road passes. Roads

can be identified in this manner even when the roads have the same

spectral reflectance as soils or other materials in the image, and even

when different roads have different spectral characteristics.

The context which enables human beings to identify roads may

be a much broader context than the local one mentioned above, or

there may be multiple contexts, which together may serve to

disambiguate the labeling of a linear feature as a road or something

else, even if no single context would be sufficient to permit the

identification of a linear terrain feature as a road. For example, an

economic context might specify that the costs of constructing roads

should be kept low, so that roads tend to follow contour lines and

Page 15: IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL ...€¦ · IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL NETWORKS: A CONTEXTUAL APPROACH

ascend and descend gentle slopes, rather than taking routes which

would be more expensive for construction. Social context usually

prescribes that roads serve the societal purpose of transportation

between socially-significant entities, such as cities, factories, houses,

and the like; roads will not be built in segments which connect

nothing of any social importance. Another context might be the

physical limitations of the vehicles (and their drivers) which will

utilize the road; these limitations require that roads be fairly straight

in order to permit rapid travel, but may also require an occasional

curve, even when not required by the terrain, in order to prevent

the divers from getting bored. All of these contexts may be taken

into consideration by a human interpreter of an image. However,

only the local context was utilized by the artificial neural network in

this experiment.

If the pixel to be classified as a road or non-road pixel is

considered to be the target pixel, then, in order to include the local

context as part of the input to the neural network, the pixels

surrounding the target pixel would also have to be included as part

of the input. Initially, it was thought that including just the 8 pixels

surrounding and contiguous to the target pixel would be sufficient.

However, it was discovered that roads occasionally straddled a pixel

boundary, contributing their spectral reflectance characteristics to

both the chosen target pixel, plus one or more of the surrounding

pixels, whose contrast was supposed to provide the context necessary

to identify the target pixel correctly. Moreover, scattering of the

light rays as they pass back through the atmosphere to the airborne

sensors contaminates the immediately contiguous pixels with the

Page 16: IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL ...€¦ · IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL NETWORKS: A CONTEXTUAL APPROACH

spectral signature of the road pixel, thus reducing the contrast.

Consequently, it was decided to include the 16 pixels surrounding the

8 pixels contiguous to the target pixel as part of the input, resulting

in the input pixels consisting of a 5-pixel by 5-pixel square centered

around the target pixel (see Figure 8).

The same 500 pixels which had been selected as the training

and test pixels for the first experiment were utilized for this one as

well, with the selected pixels serving as the target pixel around

which the context of 25 pixels was defined. All seven bands of data

for each pixel were presented as input, producing a total of 175 input

units (7 times 25). Each input unit originally had an integer value

between 0 and 255; this was normalized to be a real number

between 0 and 1.

Various sizes and configurations of networks were trained and

tested. The network which performed best on the test set had a

configuration of 175-15-5-1 (175 input units, a hidden layer of 15

units, another hidden layer of 5 units, and a single output unit); it

achieved an accuracy rate of 94%, making only three errors in

classifying the 50 examples in the test set, only one of which was an

error in classifying a non-road as a road. Networks with a single

hidden layer performed nearly as well; networks with single hidden

layers of 7 and 50 units performed at the rate of 92 % accuracy.

Now that several networks had been trained to identify roads,

they were tested against the entire image. The trained networks

were exposed to all of the pixels in the 400 by 400 test image, with

the exception of a two-pixel layer around the border of the image,

for which network output could not be generated due to the lack of a

Page 17: IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL ...€¦ · IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL NETWORKS: A CONTEXTUAL APPROACH

complete context for them; white pixels were substituted for the

output of these border pixels. The trained networks generated an

output value for each pixel that varied from 0 to 1; this value

represented the network's degree of confidence that the pixel was a

road pixel, with a value near 1 representing a high degree of

confidence, and a value near 0 representing a low degree of

confidence. These output values were displayed in the same format

as the original 400 by 400 pixel image, with all values below 0.5

displayed as white, and all values over 0.5 displayed as black. The

resulting images, for the single-hidden-layer and two-hidden-layer

networks, are shown in Figures 9 and 10 respectively. The images

do not differ very much from one another, demonstrating that both

the single and the double hidden layer networks are learning the

same things about the data, and are learning to discriminate in the

same way between road and non-road pixels.

Upon inspection of these two images, it may be observed that

the neural networks are very conservative; almost no no-road pixels

are identified as road pixels, and even a good many road pixels are

not identified as road pixels. On the one hand, the fact that the

network makes very few errors in classifying non-road pixels (some

isolated soil pixels, and a few riverbank pixels) demonstrates the

effectiveness of the combined spectral/contextual approach; in

discriminating effectively between roads and soils, the neural

networks far out-perform the traditional statistical approaches which

use spectral information alone. On the other hand, the roads

delineated by the neural networks often appear as dotted lines, with

segments of roads unable to be classified confidently as roads by the

Page 18: IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL ...€¦ · IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL NETWORKS: A CONTEXTUAL APPROACH

networks. In some cases it is difficult to follow the line of the road

unless one knows already knows where the road should be located;

the bridges over the Mississippi River, for example, do not show up

at all.

The problems which cause the dotted-line effect are discussed

at some length below, but the basic problem is a lack of adequate

information. Enlargement of the original images in certain areas

(Figures 11 and 12, illustrating the effects of scattering and occlusion,

respectively) demonstrates that information about the pixels in these

areas is rather inconclusive; neither spectral nor local contextual

information provides much of a clue as to whether the pixels in these

areas are road pixels or not. A more global context might well enable

humans to delineate a road through these areas, but consideration of

these larger contexts is beyond the scope of this study.

Analysis of the weights of the connections in the 175-7-1

network revealed that the network depended primarily upon the

information from bands 1, 3, 5, and 7, used band 4 a little, and

almost totally ignored the information from bands 2 and 6.

Moreover, it was often the case that information from only one pixel

in any direction was utilized by the network; that is, if the network

placed heavy emphasis on the influence of the upper-left pixel of the

outer ring of pixels, it was almost sure to ignore almost completely

the information provided by the upper-left pixel of the inner ring,

and vice versa. This indicates that it should be possible to reduce the

size of the input to the network while still retaining its ability to

discriminate road from non-road pixels, although such a reduction

was not attempted as a part of this study.

Page 19: IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL ...€¦ · IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL NETWORKS: A CONTEXTUAL APPROACH

Statistical Rule-based Systems

Statistical techniques were the tools previously most

frequently used to classify areas and features within remote sensing

images. Although some statistical techniques (e.g., principal

components analysis; see Naugle et al., 1991) did not appear to be

very useful for automated extraction of roads, others appeared to be

potentially valuable. Three statistical approaches to identifying

roads were tried, and the results compared to the neural network

approach.

Thematic Mapper spectral signatures were derived for several

different types of roads in the test image area by J. D. Lashlee of the

U. S. Army Corps of Engineers Waterways Experiment Station

(personal communication, 1993), including an interstate highway and

two different segments of a U. S.. highway. These signatures were

derived by sampling many pixels along stretches of each of these

roads, then determining the mean spectral reflectance of each road in

each of the 7 TM bands. Using this signature information, an image

of the test area was constructed, with the color of the pixels being

proportional to the distance from the mean spectral values of the

average road signature values; thus pixels whose spectral signature

was very close to that of a road were displayed as very dark, while

pixels whose spectral signature was quite different from that of a

road were displayed as very light. The resulting image is presented

as Figure 13.

Most roads in this image, with the obvious exception of the

bridges over the Mississippi River, are clearly visible. Unfortunately,

Page 20: IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL ...€¦ · IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL NETWORKS: A CONTEXTUAL APPROACH

many other non-road features are also displayed, including medians

and grassy patches around the interstate highway, and especially

patches of soils north and west of the Mississippi, which the

signature technique classifies very strongly as roads. Moreover,

breaks appear in many of the smaller roads, such as the roads in the

National Military Park; in some cases the breaks are so long (e.g.,

South Washington Street) as to make it difficult to discern the

existence of a road in that area.

Another statistical approach is to perform an analysis of

Thematic Mapper data to try to find a set of rules which will identify

everything which is not a road; these rules could then be used to

eliminate non-road features, leaving only the roads. Wharton's

(1987) analysis revealed a set of rules that eliminate much water

and vegetation from TM images, leaving roads and urban features

such as parking lots. Unfortunately, Wharton's approach does not

eliminate patches of soils (including fields and riverbanks), and even

heavily-silted water remains in the processed image; moreover,

many of the smaller roads are eliminated from the image. Figure 14

demonstrates the effects of applying Wharton's rules to the test

image.

Inspired by these techniques, it was thought that perhaps a

rule-based system which looked at the boundaries of the spectral

reflectance of roads in all 7 TM bands simultaneously might be able

to identify road pixels. Visual inspection and hand-processing of the

test image in each of the bands except band 6 revealed lower and

upper bounds in spectral reflectance for road pixels in each band

(see Table 2).

Page 21: IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL ...€¦ · IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL NETWORKS: A CONTEXTUAL APPROACH

Band Lower Bound Upper Bound

1 1 4 6 1

2 1 9 5 7

3 1 5 8 4

4 2 7 1 6 5

5 1 3 1 0 1

6 N/A N/A

7 8 9 4

Lower and Upper Limits for Spectral Values for Road Pixels in TM

Bands

Table 2

Vegetation often has a lower spectral value than roads in bands 1, 2,

and 3; water has a lower spectral value than roads in bands 4, 5, and

7. Hot and bright objects (factories, roofs of metal buildings, large air

conditioning units, etc.) have higher spectral values than roads in all

bands.

Versions of the test image were produced for each of the

processed bands, displaying in white the pixels which are outside the

range (both below and above) of road pixels in that band; these

images are presented in Figures 15 through 20. It may be observed

that different bands eliminate different types of land features. For

example, the rules for band 3 are effective at eliminating a

substantial amount of vegetation and bright objects. On the other

Page 22: IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL ...€¦ · IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL NETWORKS: A CONTEXTUAL APPROACH

hand, the rules for band 4 eliminate virtually all water areas,

including water-logged soils.

A program was written which combined all of the rules devised

for the 6 processed bands, eliminating any pixel whose values fell

outside of the range of road pixels in any of the bands. The resulting

image is displayed as Figure 21, with white representing non-road

areas, grey probable non-road areas, and black possible road areas.

Unfortunately, many soil areas are still classified as possible roads,

and a number of the smaller roads are marked as probably or

definitely not a road. However, this technique does seem superior to

Wharton's, in that fewer areas of soils are included as possible roads,

and fewer roads are discarded erroneously.

When the above technique is combined with Wharton's rules,

the result is the elimination of almost all features except major roads

and some soil patches and shorelines; see Figure 22. Once again, the

spectral similarity between roads and soils in the Thematic Mapper

data has restricted the ability of a technique which depends strictly

upon spectral information to distinguish one from the other. Clearly,

the contextual information in this data set will be required to

discriminate roads from soils.

Self-organizing Feature (Kohonen) Map

Another type of artificial neural network, called a self-

organizing feature map (or a Kohonen map), can discover the

categories inherent in a set of data, without needing to be provided

with anything other than the actual data itself (unlike the

backpropagation network, which needs to be told what the correct

Page 23: IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL ...€¦ · IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL NETWORKS: A CONTEXTUAL APPROACH

answer is for each input during training). An experiment using a

Kohonen map was executed to determine whether or not this

technique could categorize the data in such a way as to produce a

"roads" category; if so, then the defining characteristics of this

category could be determined by examination of the weights of the

connections between the nodes in the network and extracted for use

in an expert system to find roads.

A data set was constructed which consisted of the spectral

characteristics (ranging from 0 to 255) in all seven bands of each

pixel in the 400 by 400 pixel testbed image. Using Kohonen's SOM-

PAK software, two self-organizing feature map programs were

created, one with a 5 by 3 map (15 categories), and one with a 10 by

8 map (80 categories). These Kohonen maps were exposed to the

data in steps, ranging from one iteration of the data set up to 10,000

iterations. Apparently the data was inherently too disparate to

allow the network to construct tight categories; the average error per

category was 17, indicating quite a bit of variance between members

of the same category.

The results for the 5 by 3 map are displayed in Figure 23. The

Kohonen map categorizes soils and roads as being rather similar,

vegetation as somewhat different, and water as quite different.

Clearly the spectral characteristics of a given pixel, by themselves,

are insufficient to allow the self-organizing map to discover natural

categories, inherent in the data, which separate roads from all other

types of terrain. Even when given a larger number of classes (80)

into which to categorize the data, the self-organizing map does no

better at separating roads from soils.

Page 24: IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL ...€¦ · IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL NETWORKS: A CONTEXTUAL APPROACH

The results of this experiment are supported by those of

Augusteijn and Dimalanta (1992), who found a Kohonen map to be

unhelpful in trying to automatically determine land use classes in

satellite imagery. Additional studies are planned, using the 25-pixel

combined spectral/contextual data utilized in the successful

backpropagation neural network approach described above.

ANALYSIS OF RESULTS AND DISCUSSION OF PROBLEMS

Pixel resolution

It is fairly difficult to accurately position roads using Thematic

Mapper data. One of the primary sources of difficulty is the size of

the pixel compared to the size of the road; six of the seven TM bands

have a nominal spatial resolution of 30 meters by 30 meters, while

the thermal infrared band has a resolution of 120 meters by 120

meters. In contrast, two-lane local and state highways have lane

widths which range from 8 to 12 feet (an average of about three

meters), while interstate and some other four-lane highways have a

width of 12 to 13 feet (approximately four meters) per lane. The

shoulders on the sides of the road (which may not have the same

spectral reflectance characteristics as the hard, smooth road surface)

range from 10 to 12 feet for the outside shoulders of interstate

highways (inside shoulders range from 6 to 8 feet) down to almost

no shoulder for some smaller roads. Thus, even when the shoulders

are included, the average road may well cover less than half of the

surface area represented by a single TM image pixel. The widest

Page 25: IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL ...€¦ · IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL NETWORKS: A CONTEXTUAL APPROACH

roads normally encountered, interstate highways with little or no

grassy median strip, may be up to 92 feet wide; this is just about the

width of a single 30-meter resolution pixel.

In the data received from the satellite sensors, each pixel is a

spot of light of a single uniform intensity. If a road extends over less

than half the surface area of a pixel, then the numerical value of that

pixel will be determined less by the spectral reflectance

characteristics of the road than by the other land-cover class or

classes present in that pixel. This may well make it difficult - or, in

some cases, impossible - to detect the presence of a road in a pixel.

Moreover, even if a road can be detected as being present in a pixel,

but does not seem to be present in a neighboring pixel, it may

indicate only that more than 50% of the road's surface is in the

former and less than 50% is in the latter. This means that, in theory,

the centerline of a road can be located with 15-meter accuracy.

However, in actual tests Togliatta et al. (1988) judged the accuracy of

TM data to be somewhat worse than this - plus or minus 25 meters -

and recommends that TM images not be used for topographic

mapping on a 1:50,000 scale, as he found that "only a small

percentage of medium/small roads could be traced" and that

occasionally "even large roads were missing [from the TM image] or

abruptly disappeared" (p. 1684). In other cases a road's image may

blur due to the effects of atmospheric scattering and spread over

several adjacent pixels; this also makes the centerline of a road hard

to locate.

Page 26: IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL ...€¦ · IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL NETWORKS: A CONTEXTUAL APPROACH

Cowen et al. (1991) maintain that much better than 30-meter

pixel resolution is required for the accurate detection of roads. They

state:

The general rule in digital image processing is that in order to

positively detect a feature it is necessary for the sensor system

to include at least two pixels in both the x and y directions....

This theoretically means that a minimum pixel resolution of ...

about four meter resolution is required to identify [residential]

roads. (pp. 35-36)

Benjamin and Gaydos (1990) studied the ground resolution required

to adequately detect and extract roads in a California suburb; using

aerial photography they found that "a 3-meter pixel resolution

appears to be the best choice" (p. 93).

One solution to the problem of pixel resolution may appear to

be obvious; obtain satellite imagery with a pixel resolution of at least

four meters. In some cases, even higher resolution than 4 meters

may be required, since, in many of the places where we do not

already have access to accurate maps of road networks (third-world

countries, etc.), the roads may be even narrower than 8 meters.

Unfortunately, no satellite imagery with this degree of resolution is

commercially available at present; even the SPOT Panchromatic data,

with 10 meter resolution, is considered insufficient to locate most

roads within the tolerances of standard map accuracy.

Moreover, reliance on higher-resolution imagery may not be a

practical solution to the problem. In many cases, the area covered

by a high-resolution image is too small to be of use for mobility-

evaluation purposes; a number of these images would have to be

Page 27: IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL ...€¦ · IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL NETWORKS: A CONTEXTUAL APPROACH

obtained to encompass the same area as provided by a single

readily-available Landsat TM image. Moreover, the amount of effort

required to process these high-resolution images may be too large,

and the time too long; for some purposes, the TM images may

produce adequate results, with considerably less time and effort

required.

Scattering

A related problem is that of scattering. Roads in many cases

(although certainly not in all) are lighter than their surroundings;

they reflect more light back to the satellite-borne sensor. As the

light travels back up through the atmosphere it is scattered to the

sides of its true path, and impinges not only on the sensor directly

above a given road pixel but also on the sensors which are picking up

the light reflected by the neighboring pixels. If these neighboring

pixels are darker than the road pixel, the reflected light from the

road will cause the neighboring pixels to appear brighter than they

should be. They will then share some of the spectral characteristics

of the nearby road pixel and will seem to be more road-like, and

thus more soil-like, since roads and soils have the same basic

spectral reflectance characteristics.

One of the ways in which the neural network devised for this

study identifies road pixels is by detecting a contrast between the

center pixel of the 5 by 5 pixel square and the context surrounding

the center pixel. Scattering reduces this contrast, and may make the

entire square resemble a patch of relatively homogeneous soil-like

Page 28: IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL ...€¦ · IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL NETWORKS: A CONTEXTUAL APPROACH

pixels rather than having the distinctive liner pattern of a road

passing through a contrasting patch of terrain.

Spectral similarity

Another problem in detecting roads in TM imagery is that

roads have spectral characteristics similar to those of other types of

land-cover classes. Specifically, roads are often confused with soils.

Paved roads are composed of gravel and sand with a cement binder,

or crushed rock and sand with an asphalt binder, and gravel and dirt

roads consist of soil components directly, so it is not surprising that

classification algorithms that depend primarily or solely on the

spectral characteristics of the pixels in an image would group roads

and soils together. As a result, however, these algorithms almost

always misclassify soil regions (and patches of bare soil in particular)

as roads. Even hybrid algorithms that include a component to detect

linear structures can confuse furrows, field boundaries, meander

scars, levees, dry stream beds, shorelines, pipeline construction, and

the like with roads.

Moreover, housing tracts and developed urban areas may

present a problem. Industrial areas, malls, and shopping centers

may have large areas consisting of parking lots and sidewalks. Even

residential areas may cause confusion due to the fact that roofs are

often composed of fiberglass, asphalt, and mineral particles. These

materials may have the same sort of spectral signature as roads, and

thus may be misclassified as roads.

The backpropagation neural network approach discussed in

this paper does very well at discriminating roads from other soil

Page 29: IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL ...€¦ · IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL NETWORKS: A CONTEXTUAL APPROACH

types; by combining spectral and contextual information, the

network is able to distinguish roads from patches of soils, which lack

the linear structure with contrast on either side which is

characteristic of most roads. The network also seems to perform well

in distinguishing between roads and other linear features which have

the spectral signatures of soils, such as river banks.

The problem of spectral similarity apparently may be able to

be solved completely by utilizing a hyper-spectral scanner.

Christiansen et al. (1993) use neural networks with JPL's AVIRIS

sensor system, which processes information from 224 separate

spectral bands; their results indicate that the spectral signatures of

various types of materials may be determined so precisely by the

AVIRIS system that paved roads can be identified based on their

spectral characteristics alone. Unfortunately, the spatial resolution of

AVIRIS is 20 meters, which means that problems of occlusion still

affect the hyperspectral data. In addition, the AVIRIS system is not

a satellite-borne system, so it may not be possible to obtain AVIRIS

data from hostile or remote areas.

Occlusion

When the roadway in a given pixel is occluded, it is impossible

to reliably classify that pixel as a road pixel. One major cause of

occlusion during the growing season is vegetative canopy. Where

vegetation overhangs the road, it obscures the road and presents a

vegetative spectral signature to the scanning device, resulting in

pixels in the image which have the characteristics of vegetation

instead of road.

Page 30: IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL ...€¦ · IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL NETWORKS: A CONTEXTUAL APPROACH

Another source of occlusion is vehicular traffic. A

concentration of vehicles in one spot on the road will give the pixels

in that location the spectral characteristics of painted metal.

For areas in which only part of the road is obscured, higher-

resolution images may be able to distinguish road from non-road

areas. However, when the road is almost totally occluded, even

higher resolution will not solve the problem. Tunnels, for example,

totally hide their roads, and vegetative canopy can also obscure a

road completely for a significant distance.

Where a road pixel is occluded to such an extent that it no

longer has the spectral signature of a road, it would be imprecise to

classify it as a road pixel. However, other evidence (such as the

presence of road pixels on either side of it) may suggest that the

spectral evidence be ignored or overridden, and the pixel classified

as a road anyway. In these cases, care should be taken to indicate

that the spectral evidence does not support the classification of the

pixel as a road pixel.

CONCLUSION

The most readily apparent conclusion that may be drawn from

this series of studies is that no matter what analysis technique is

used, be it neural networks (either backpropagation nets or self-

organizing feature maps) or standard statistical or rule-based

techniques, spectral information alone is insufficient for a system to

be able to accurately determine whether a pixel is a road pixel or

Page 31: IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL ...€¦ · IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL NETWORKS: A CONTEXTUAL APPROACH

not. The use of context or other domain-specific knowledge is

essential for successful classification.

Secondly, it is clear from an examination of Figure 11 versus

Figures 14 and 22 that use of neural networks to identify roads in

satellite imagery compares favorably with the best previously-used

techniques (use of statistical analysis to generate spectral signatures

characteristic of roads, and use of expert-system-like sets of rules to

eliminate non-road areas) and shows promise of being able to

provide the framework for a complete solution to the problem. The

results also seem to indicate, however, that backpropagation neural

networks alone may not be able to provide a complete solution; it

may be necessary to complement the backpropagation technique

with other techniques in order to more closely approach a complete

solution, using the results of the backpropagation technique as a

reliable starting point for the application of other road-completion

techniques. This may be done in two ways: through a multi-stage

neural network approach, in which a series of neural networks,

perhaps of different types, are applied to the results of the previous

stage, increasing the accuracy of the results each time; and by means

of a hybrid neural network system, which incorporates other types

of artificial intelligence techniques, such as a small expert system or

a probabilistic system, with the backpropagation approach.

Future work

Projected work will involve construction of a hybrid system

that will include the combination of the backpropagation neural

network information with information derived from statistical

Page 32: IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL ...€¦ · IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL NETWORKS: A CONTEXTUAL APPROACH

analysis of the data. The neural network may be thought of as being

very conservative, and the pixels it identifies as road pixels generally

do belong to roads; however, its inherent conservatism causes it to

refuse to identify a pixel as a road pixel when it is not certain of its

identity, resulting in a series of dashes in the place of what should

appear as solid lines. On the other hand, the statistical techniques do

not eliminate enough pixels, in particular including many soil-type

pixels along with the more-continuous lines of road pixels. It is

expected that the combined use of both sources of information will

have a synergistic effect, mutually compensating for the weaknesses

of each technique.

The hybrid system will employ a matrix the same dimensions

as the image, in which a belief factor is associated with each pixel in

the image. When the image is processed by the neural network, a

value between 0 and 1 is produced for each pixel, which may be

interpreted as the "belief" that the neural network has that the pixel

is a road pixel; the higher the value, the stronger the belief. The

hybrid system will use these high belief factor pixels as "seeds", or

known starting points from which longer continuous stretches of

roads may be extended, or "grown". During each step ("generation")

of the process, each seed pixel will look at the information about its

neighboring pixels, provided by the statistical techniques, and will

determine the likelihood that a neighboring pixel is a road pixel or

not. Some pixels (such as water pixels) may be identified clearly by

the statistical analysis as non-roads, and will always have a belief

factor of 0. Other pixels will have larger belief factors. Neighboring

pixels which are computed to have a high probability of being roads

Page 33: IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL ...€¦ · IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL NETWORKS: A CONTEXTUAL APPROACH

will be labeled tentatively as roads, and can serve as seed pixels

during the next generation of road growth. Directions of orientation

will be determined for each of the road segments, and pixels located

in these directions will have their probability rating enhanced;

similarly, roads which are near one another and have their axes

oriented in the same direction will be encouraged to grow together.

The process will continue for a number of generations, much like a

cellular automaton, until all road pixels have at least two road

neighbors (or are bounded by the edges of the image), or some

predetermined termination condition is reached. The amount of

search required to find potential neighbors may suggest the use of a

heuristic search technique in order to reduce the amount of time

required and increase the efficiency of the search.

This algorithm may be thought of as a process of hypothesis

generation and testing. The evidence provided by the neural

network concerning the location of known (or highly probable) road

pixels suggests the hypothesis that some of the neighboring pixels

might also be road pixels. The hypothesis is tested by examination of

the results of the statistical analysis and conclusions are drawn about

the probability that certain of the neighboring pixels are road pixels.

These conclusions can then serve as new evidence, generating new

hypotheses about the next set of pixels, and so on. The information

about the pixels can be stored in a belief factor knowledgebase, with

the information indexed in such a way that it would be possible to

tell how the information was derived. It may also be necessary in

some cases to retract some conclusions reached earlier - if they seem

to contradict other evidence, or other more strongly supported

Page 34: IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL ...€¦ · IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL NETWORKS: A CONTEXTUAL APPROACH

conclusions - and to backtrack to an earlier state of processing; the

data stored in the knowledgebase should enable this to be done.

Edge detecting and line following algorithms will be pursued as

possible adjuncts to this and other techniques. While these

algorithms alone cannot discriminate roads from other linear

features, they can serve to suggest possible areas which should be

investigated to see if they have the other characteristics of roads,

besides linearity; the edge-enhancing filters discussed in Naugle

(1991) may be well suited to this purpose. The identification of

pixels as belonging to lines and edges could easily serve as input to

the hypothesis-generating algorithm proposed above, in that the

belief factor of a pixel could be increased if it were part of a line or

edge.

Additional sources of information, such as digitized maps, 10-

meter panchromatic SPOT images, or radar imagery (see Lawton

al., 1985), may be very helpful in correctly classifying the terrain

features in a satellite image. A number of studies, mentioned above,

have already successfully utilized multiple sources of information to

improve the performance of image processing techniques. One

potential source that might prove especially useful would be

digitized topographic maps of the region under investigation; altitude

information, contour lines (including stream delineation), and the

display of previously-existing road networks would aid in the task of

identifying the current road networks.

There are a number of additional approaches to finding roads

in satellite images which should be considered as having sufficient

potential that they are worthy of further research.

Page 35: IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL ...€¦ · IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL NETWORKS: A CONTEXTUAL APPROACH

First, if context is so important in identifying road pixels, then a

larger context may prove to be more successful than the 5-pixel by

5-pixel context currently used. A 6-by-6, or even larger, context

should be investigated for use with the backpropagation neural

network. To reduce the size of the network, perhaps selected

portions of some of the interior squares of pixels could be discarded,

since analysis shows that not all pixels are serving as equally

important sources of information. Further study would be required

to determine the most appropriate size and configuration of the

surrounding context.

Second, standard statistical techniques could be expanded to

use spatial as well as spectral information. Use of spatial information

in conjunction with spectral information does not appear to have

been attempted very often, and may well provide useful results.

Third, the amount of work that is required of the

backpropagation neural network may be reduced, in hopes of

obtaining better results. This may be done by training a different

net to identify different types of roads; one net might be trained just

to identify an interstate highway, another would be used to classify

4-lane U. S.. highways, a third would be trained on good two-lane

roads, and a fourth network would identify small roads with

overhanging vegetation. These results could be merged, or

incorporated within a more general network which could serve as a

framework upon which to anchor the results of the other networks.

Fourth, the type of neural network used could be changed.

Optimized backpropagation networks tend to generalize better, and it

may be worth the effort to construct an optimized network for this

Page 36: IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL ...€¦ · IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL NETWORKS: A CONTEXTUAL APPROACH

task, to see if it would perform better. Recent work suggests that

cascaded networks, or network "committees", may improve

performance; these consist of a sequence of networks in which each

network handles only the data that the pervious networks

misclassify, much like a decision tree.

Fifth, classification and regression trees should also be explored

as possible alternatives to neural networks. In a classification tree, a

set of rules is progressively discovered which best classifies the data

which remains after the application of the previous rules. The

amount of remaining unclassified data is reduced at each step, until

all of the data is classified or some irreducible remainder of data is

left. This is a very powerful technique and shows much promise.

Summary

Although humans can readily trace road networks in satellite

images, it is difficult for computers to do so. Current techniques have

concentrated primarily on the use of spectral data alone; the results

of the current report indicate that use of contextual or spatial

information is essential. Artificial intelligence techniques, which use

domain-specific knowledge to solve problems - and artificial neural

networks, in particular - may be applied successfully to the problem

of extracting roads from remote sensing imagery. Additional

research is needed to improve the performance of these artificial

intelligence techniques; hybrid approaches, which utilize several

sources of information, appear to offer the most promise of success.

Page 37: IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL ...€¦ · IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL NETWORKS: A CONTEXTUAL APPROACH

Acknowledgements - This work was supported by the U. S. Army

Corps of Engineers Waterways Experiment Station, Geotechnical

Laboratories, Mobility Systems Division, Analytical Studies Branch,

under the auspices of the U.S. Army Research Office Scientific

Services Program administered by Battelle (Delivery Order 856,

Contract No. DAAL03-91-C-0034).

Page 38: IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL ...€¦ · IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL NETWORKS: A CONTEXTUAL APPROACH

REFERENCES

Augusteijn, M. F. and Dimalanta, A. S., 1992, Feature detection in

satellite images using neural network technology: Telematics and

Informatics, v. 9, nos. 3/4, p. 131-144.

Augusteijn, M. F., Shaw, K. A., and Watson, R. J., 1993, A study of

neural network input data for ground cover identification in

satellite images: Unpublished manuscript, University of Colorado

at Colorado Springs.

Bajcsy, R. and Tavakoli, M., 1976, Computer recognition of roads from

satellite pictures: IEEE Transactions on Systems, Man and

Cybernetics, v. 6, no. 9, p. 623-637.

Benediktsson, J. A., Swain, P. H., and Ersoy, O. K., 1990, Neural

network approaches versus statistical classification methods in

classification of multisource remote sensing data: IEEE

Transactions on Geoscience and Remote Sensing, v. 28, p. 540-

552.

Benjamin, S. and Gaydos, L.. 1990, Spatial resolution requirements for

automated cartographic road extraction: Photogrammetric

Engineering and Remote Sensing, v. 56, no. 1, p. 93-100.

Page 39: IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL ...€¦ · IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL NETWORKS: A CONTEXTUAL APPROACH

Bhatia, P., Srinivasan, V., and Ong, S. H., 1991, Single-layer edge

detector with competitive unsupervised learning: Proc. 1991 IEEE

International Joint Conference on Neural Networks, Vol. 1,

Piscataway, New Jersey, p. 634-639.

Carpenter, G. A. and Grossberg, S., eds., 1992, Neural Networks for

Vision and Image Processing: MIT Press, Cambridge,

Massachusetts.

Christiansen, J. H., Zawada, D. G. Simunich, K. L., and Slater, J. C., 1993,

A hyperspectral image analysis workbench for environmental

science applications: submitted to Ninth IIPS Conference, Argonne

National Laboratory.

Cowen, D. J., Jensen, J. R., and Halls, J.1991, Maintenance of TIGER files

using remotely sensed data: Proc. 1991 ACSM-ASPRS Annual

Convention, Baltimore, Maryland, p. 31-40.

Dawson, M. S., Fung, A. K., and Manry, M. T., 1993, Surface parameter

retrieval using fast learning neural networks: Remote Sensing

Reviews, v. 7, no. 1, p. 1-17.

Dreyer, P., 1993, Classification of land cover using optimized neural

nets on SPOT: Photogrammetric Engineering and Remote Sensing,

v. 59, no. 5, p. 617-621.

Page 40: IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL ...€¦ · IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL NETWORKS: A CONTEXTUAL APPROACH

Duda, R. O., and Hart, P. E., 1973, Pattern recognition and scene

analysis, Wiley, Ney York.

Fischler, M. A., Tenenbaum, J. M., and Wolf, H. C., 1981, Detection of

roads and linear structures in low-resolution aerial imagery using

a multisource knowledge integration technique: Computer

Graphics and Image Processing, v. 15, p. 201-223.

Hepner, G. F., Logan, T., Ritter, N., and Bryant, N., 1990, Artificial

neural network classification using a minimal training set:

Comparison to conventional supervised classification:

Photogrammetric Engineering and Remote Sensing, v. 56, no. 4, p.

469-473.

Hepner, G. F. and Ritter, N., 1989, Application of an artificial neural

network to land cover classification of Thematic Mapper imagery:

Jet Propulsion Laboratory Internal Technical Report.

Jensen, J. R., 1986, Introductory Digital Image Processing: A Remote

Sensing Perspective: Prentice-Hall, Englewood Cliffs, New Jersey.

Kanellopoulos, I., Varfis, A., Wilkinson, G. G., and Mégier, J., 1992,

Land-cover discrimination in SPOT HRV imagery using an

artificial neural network - a 20-class experiment: International

Journal of Remote Sensing, v. 13, no. 3, p. 917-924.

Page 41: IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL ...€¦ · IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL NETWORKS: A CONTEXTUAL APPROACH

Key, J., Maslanik, J. A., and Schweiger, A. J., 1989, Classification of

merged AVHRR and SMMR Arctic data with neural networks:

Photogrammetric Engineering and Remote Sensing, v. 55, no. 9, p.

1331-1338.

Lashlee, J. D. and Naugle, B. I., 1990, Statistical Analysis of Edge

Components Associated with Geologic Linear Features Mapped

from Landsat Thematic Mapper Data: Technical Report GL-90-6,

U. S.. Army Corps of Engineers Waterways Experiment Station,

Vicksburg, Mississippi.

Lawton, D. T., Glicksman, J., Conner, G. D., and Drazovich, R. J., 1985,

Linear Feature Extraction from Radar Imagery: Technical Report

ETL-0405, U. S. Army Corps of Engineers, Engineer Topographic

Laboratories, Fort Belvoir, Virginia.

Lee, J., Weger, R. C., Sengupta, S. K., and Welch, R. M., 1990, A neural

network approach to cloud classification: IEEE Transactions on

Geoscience and Remote Sensing, v. 5, p. 846-855.

Liu, Z. K. and Xiao, J. Y., 1991, Classification of remotely-sensed image

data using artificial neural networks: International Journal of

Remote Sensing, v. 12, no. 11, p. 2433-2438.

Maillard, P. and Cavayas, F., 1989, Automatic map-guided extraction

of roads from SPOT imagery for cartographic database updating:

Page 42: IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL ...€¦ · IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL NETWORKS: A CONTEXTUAL APPROACH

International Journal of Remote Sensing, v. 10, no. 11, p. 1775-

1787.

Mason, G. L., Embrechts, M. J., and Grivas, D. A., 1993, Roadway

identification on LANDSAT imagery using neural networks:

Unpublished manuscript, US Army Corps of Engineers Waterways

Experiment Station, Vicksburg, Mississippi.

Naugle, B. I., Lashlee, J. D., and Robinson, J. H., 1991, Digital Image

Enhancement of Satellite Data for Transportation Network

Extraction: Technical Report GL-91-25, US Army Corps of

Engineers Waterways Experiment Station, Vicksburg, Mississippi.

Moller-Jensen, L.1990, Knowledge-based classification of an urban

area using texture and context information in Landsat-TM

imagery: Photogrammetric Engineering and Remote Sensing, v.

56 , no. 6, p. 899-904.

Omatu, S. and Yosida, T., 1991, Pattern classification for remote

sensing using neural network: Proc. 1991 IEEE International Joint

Conference on Neural Networks, Vol. 1, Piscataway, New Jersey, p.

653-658.

Richards, J. A., 1986, Remote Sensing Digital Image Analysis: An

Introduction: Springer-Verlag, New York.

Page 43: IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL ...€¦ · IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL NETWORKS: A CONTEXTUAL APPROACH

Ritter, N. D. and Hepner, G. F., 1990, Application of an artificial neural

network to land-cover classification of Thematic Mapper imagery:

Computers & Geosciences, v. 16, no. 6, p. 873-880.

Ryan, T. W., Sementilli, P. J., Yuen, P. and Hunt, B. R., 1991, Extraction

of shoreline features by neural nets and image processing:

Photogrammetric Engineering and Remote Sensing, v. 57, no. 7, p,

947-955.

Slawinski, O., Kowalski, J., and Cornillon, P., 1991, A neural network

approach to cloud detection in AVHRR images: Proc 1991 IEEE

International Joint Conference on Neural Networks, Vol. 1,

Piscataway, New Jersey, p. 283-288.

Togliatti, G., Lechi, G., and Moriondo, A., 1988, Tests of topographic

mapping with Thematic Mapper images: International Journal of

Remote Sensing, v. 9, nos. 10 & 11, p. 1681-1686.

Ton, J., Jain, A. K., Enslin, W. R., and Hudson, W. D., 1989, Automatic

road identification and labeling in Landsat 4 TM images:

Photogrammetria, v. 43, p. 257-276.

Van Cleynenbreugel, J., Fierens, F., Suetens, P., and Oosterlinck, A.,

1990, Delineating road structures on satellite imagery by a GIS-

guided technique: Photogrammetric Engineering and Remote

Sensing, v. 56, no. 6, p. 893-898.

Page 44: IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL ...€¦ · IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL NETWORKS: A CONTEXTUAL APPROACH

Vasudevan, S., Cannon, R. L., Bezdek, J. C., and Cameron, W. L., 1988,

Heuristics for intermediate level road finding algorithms:

Computer Vision, Graphics and Image Processing, v. 44, p. 175-

190.

Wang, F. and Newkirk, R.1988, A knowledge-based system for

highway network extraction: IEEE Transactions on Geoscience and

Remote Sensing, v. 26, no. 5, p. 525-531.

Wang, J. F. and Howarth, P. J., 1987, Automated road network

extraction from Landsat TM imagery: Proc. 1987 ASPRS-ACSM

Annual Convention, Vol. 1., Baltimore, Maryland, p. 86-95.

Wharton, S. W., 1987, A spectral-knowledge-based approach for

urban land-cover discrimination: IEEE Transactions on Geoscience

and Remote Sensing, v. GE-25, no. 3, p. 272-282.

Page 45: IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL ...€¦ · IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL NETWORKS: A CONTEXTUAL APPROACH

FIGURES

Page 46: IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL ...€¦ · IDENTIFICATION OF ROADS IN SATELLITE IMAGERY USING ARTIFICIAL NEURAL NETWORKS: A CONTEXTUAL APPROACH

7-Band, 5 -by-5 Pixel Input Array for Context-sensitive Neural

Network (Showing Road Passing Through Center Pixel)

Figure 8