Digital Fictitious Datafor Automatic Mapping Research *

]. D. NEWTON and H. F. DODGEt

Graphic Data Redudion Dept.IBM, Kingston, New York

(Abstract on page 169)

INTRODUCTION

T HE purpose of this paper is to describe anIBM 7094 Data Processing System pro­

gram designed to prod uce fictitious photo­graphic data. Such data are now being used asan important tool in carrying out research indigital automatic mapping at IBM.

In this research, an independent effort isbeing made to produce and verify techniquesfor automatically generating contoured ortho-

known. The fictitious terrain data are com­posed of both the gray-scale val ue and theexact geographic position (X, Y, Z) forground points as close together horizontallyas 0.1 foot. From such terrain data and fromknowledge of exact position, attitude, andprincipal distance of the camera, values ofwhich are arbitrarily chosen, fictitious photo­graphic data with points as close to eachother as one micron can be obtained. Allphoto coordinate data are accurate within the

A stereoscopic pair of "photographs" of a hypothetical terrainmodel can be simulated, stored on magnetic tape, and printed forvisual inspection.

photographs from control-annotated stereodiapositives, ground control data, cameracalibration information, exposure stationposition estimates, and output specifications.Current activity is largely concerned withfinding a solution to the problem of auto­matically associating corresponding images onthe several photos of the strip or block onwhich the images appear. Another objectiveis to establish specifications for practicalspecial purpose equipment which will carryout orthophoto compilation automaticallyat high speed. One approach to an automaticmapping system is described in [1].

PURPOSE OF FICTITIOUS DATA

The function of the fictitious data is toestablish strong technical control over thecorrelation research. Digital means have beendevised for generating fictitious terrain andphotographic data about which everything is

limits of round-off error. No presently avail­able electronic scanning equipment can ex­tract such error-free photo data. Moreover, itis impractical to attempt to obtain accurategeodetic positions for such a multitude ofground points in a real photo model.

The primary use of such error-free data is tofacilitate the evaluation of the correlationprogram and to establish various decision­making criteria in the correlation process.

The main function of the fictitious photo­generation process is to introduce stereoscopicor binocular parallax into two photographs.I n human stereoscopic vision, binocularparallax is the principal depth cue [2]. All otherdepth cues, binocular or monocular, such aslinear perspective, monocular movementparallax, shadow patterns, interposition, etc..are of much less importance and merely serve

* Presented at 30th Annual Meeting of the Society, Hotel Shoreham, Washington, D. c., March17-20, 1964.

t Mr. Dodge is now Principal Engineer, Recon. & Mapping Dept., Aeronutronics Div., Ford MotorCo., Newport Beach, Calif.

166

GENERATING DIGITAL FICTITIOUS STEREOGRAMS

PHOTO [Md °2

PLANES ~p-ff-~_

FICTITIOUSMODEL

FIG. 1. Geometry of projection of model element to photo-planes. (Text on page 168.)

167

to help stereoscopic vision. The use of arandom gray-scale pattern in a pair of ficti­tious stereo photographs demonstrates thatstereo vision can be accomplished with onlybinocular parallax; a random gray-scalepattern is devoid of all other depth cues.

Therefore, any correlation process per­formed on the fictitious photos will have toutilize the property of binocular parallax inorder to derive height information. No otherdepth cues need to be present in the photos.In fact, it does not appear possible or practicalto devise a correlation process which will takeadvantage of depth cues other than binocularparallax, except in very restricted cases (e.g.,in determining the height of moon cratersfrom shadow cues on a single photograph).

If binocular parallax is the only depthindicator required in fictitious photographs,it is clear that the gray-scale pattern need notcorrespond to the elevation data in the groundterrain model in order to establish stereo­scopic vision. This correspondence is likewisenot required for curren t photo data correla­tion processes which make use of only thebinocular parallax property.

Realistic errors can be inserted into thedata. These errors fall into two categories:(1) gray-scale; (2) geometric. Gray-scale

errors can be used to simulate noise, simulateuneven transmission of light through theatmosphere or lens, and simulate differencesin reflectivity from the ground. The use ofsuch errors tests the ability of the correlationprogram to adapt itself to normal photo­graphic variations in film densities. Geomet­ric errors which can be easily determined ordescribed, such as residual aerotriangulationerror and errors inherent in an electronicscanner, can also be introduced.

]. D. NEWTON

168 PHOTOGRAMMETRIC ENGINEERING

\Vhen a satisfactory correlation program isdeveloped, it is planned to introduce thesearbitrary errors into the data. For design ofan optimum mapping system, a series of testscan be run to determine the allowable errorsin the electronic gear, the required spot size,and the gray scale quantization needed forany desired final accuracy. Trade-offs of thesediA'erent factors can be investigated.

I n all these tests, the fictitious data pro­vide the framework within which controlledexperiments can be conducted.

EVALUATION OF THE CORRELATION PROCESS

As previously mentioned, the fictitiousdata consist of ground terrain elevation data,two stereo photographs, the principal distanceof the camera, and "aerotriangulation" data(the positions and attitudes of the two cam­eras at the time of exposure). In the auto­matic mapping program, only the two ficti­tious stereo photographs and the principal­distance of the camera are provided as inputto the correlation process in its function ofassociating corresponding images. The qual­ity of this process of association or correla­tion is checked by an evaluation routine.Evaluation proceeds as follows for each pairof correlated points:

1. The object rays are projected in accord­ance with the aerotriangulation data,with the camera principal-distance, andwith the photo coordinates found by thecorrelation process; in this way, theposition of the virtual ground intersec­tion is found (i.e., horizontal and ver,tical position).

2. The ground model data are interrogatedfor the true terrain elevation at thehorizontal ground position.

3. A comparison of the true elevation withthat computed from the correlationprocess provides the basis for determin­ing if \'ertical accuracy requirementsha ve been met.

4. The original horizontal position of thatground point corresponding to one of theobject rays, as selected by the correla­tion process, is obtained from theficti tious data.

5. A comparison of the original horizontalposition with that found as a result ofthe correlation process forms the basisfor determining if horizontal accuracyrequirements have been met.

6. Horizontal and vertical errors are sum,marized by the evaluation routine.

To verify the fideli ty of the test loop priorto the testing of a highly complex correlationprogram, it is expected that the loop will beclosed first by using an "a-priori correlationprogram." This program will select, from thefictitious photo data, matched pairs of pointsas they are generated. \Vhen the entire testloop has been proved, the "a-priori" correla­tion program will be replaced by the actualcorrelation programs to be evaluated.

MECHANICS OF DIGITAL FICTITIOUS

DATA GENERATION

The geometry involved in producing afictitious stereogram, such as those given inFigures 5, 6, 10, 11, and 13, is illustrated inFigure 1 (shown on page 167). It will beshown how, over each photo plane, a discretegray-scale picture pattern corresponding to aperspective photograph of the model can begenerated. Each photo coordinate system isgiven any desired spatial position Randattitude M. (M is an orientation matrixwhose values are direction cosines of thephotographic coordinate system taken rela­tive to the master ground X, Y, Z coordinateframe.)

The grou nd model is an arbi trary discretesurface defined on a square grid on the X- Yplane. Elevation information is given at eachX, Y grid intersection. Figure 1 shows suchelevations for a typical square element in themodel. Such terrain data may be computedfrom a mathematical surface or interpolatedfrom contour map data for a realistic effect.Any grid interval can be chosen for the model.

The gray-scale val ues for the model may beany arbitrary density pattern; in general, thegray values are obtained by electronic scan­ni ng of an aerial photograph. The gray val uesare assigned on the basis of one to a groundgrid square. In the computer program, capa­bility exists for carrying along 1 to 6 gray­scale values per ground square; thus, 1 to 6different gray-scale density picture patternsmay be assigned at one time to the reliefmodel. \\"hen carrying out the stereo-projec­tion process (see below), the 1 to 6 gray-scalevalues per ground grid square are carriedalong as a single entity. Hence, after the en­tire photo-plane projection process has beencompleted, 1 to 6 pairs of stereograms areobtained. These are all derived from the sameground terrain but have different picturepatterns.

GENERATING DIGIT.\!. FICTITIOUS STEREOGR.\MS 169

To generate a fictitious photograph, squareelemen ts along each row of the model areprojected in sequence by the program to thephoto plane (on which is superimposed asquare photo grid). The projection equationsused are those originated by von Gruber(1932) :

computer program for setting the photo-gridspacing (resolution) to any desired value. Ingeneral, a 1-1 ratio between the number ofphoto-grid intersections and the correspond­ing number of ground-grid intersections ismaintained.

:t =

y=

_ jMII(XO - Rx) + M 21 (Yo - Rl·) + M,,(Zo - R z)

M 13 (Xo - Rx) + M 23(YO - RJ·) + M 33(ZO - R z )

_ jMI2 (XO - Rx) + M 22(YO - R y) + M 32(ZO - Rz)

M 13 (XO - Rx) + M 23(YO - Ry) + M 33(Zo - Rz)

in which

x, yare the photo-coordi nates of the pro­jected point

f is the principal distance of the camera

Because the photographic axes are notnecessarily parallel to the master coordinatesystem and because of the usual presence ofrelief in the model, a sort-and-merge routine

ABSTRACT: Techniques for generating digital fictitious stereograms on an IE ill7094 Data Processing System are described. The stereograms are obtained by aperspective projection of a ground model with known elevation and gray shadedetail upon two "photo" plcmes with known spatial position and orientation.Realistic photographic and geometrical errors can be introduced into the dataif desired. Such fictitious data are very useful for evaluation of digital auto­matic mapping programs now being developed and perfected at JEll!.

M is the orientation matrix of the photo­plane

(Xa, Ye, Ze) is the position of the groundpoint

(Rx , Ry , Rz , components of R) is the loca­tion of the camera exposure station

The four rays from the corner elevations ofthe ground-square element form the cornersof a quadrilateral on the photo-plane as in­dicated in Figures 1 and 2. Next, the vertical(or horizontal) grid line intercepts (see in­tercept points, Figure 2) on the sides of thequadrila teral are obtained. The intersectionsof the square photo-grid within the pro­jected quadrilateral boundary then consist ofthe points lying in between each correspond­ing pair of intercepts. These enclosed gridintersections (photo-points) are assigned thegray-scale density value (or values) of thecorresponding ground-square element.

The photo-grid intersections with theirassociated gray-scale densities (spots) simu­late the scan-by-scan and spot-by-spot dis­crete sampling of a photograph by an elec­tronic-optical scanner. The photo-grid spac­ing corresponds to the spot interval (or spotsize) of the scanner. Capability exists in the

is required to order the gray-scale densityvalues of the generated photo by spots in xand by scans in y. This routine takes up thebulk of computer lime in generating a ficti­tious stereogram.

An a-priori correlation routine is includedin the program to save the x and y coordinatesof a large number of points projected up toeach photo-plane, as corners of quadrilaterals.Matched pairs of points on the two photos arethus retained for later usc in checking outother programs in the closed simulation loop.

The computer program provides for se­quential numbering of the ground gridsquares. These numbers are carried to onephoto-plane along wi th the gray-scale valuesassociated with the same ground-grid squaresthroughout the entire simulation process­stereo-projection, correlation, and evaluation.In this way, a horizontal check can be madeon the correlation process. As a resul t of thecorrelation progra m, matched pairs of photopoints are obtained; each pair of the~e isassociated with a gray-scale value and with aground-grid square number corresponding toone of the photo points. The horizontal error

170 PHOTOGRAMMETRIC ENGINEERING

PHOTO GRID COORDINATE SYSTEM

FIG. 2. Assignment of photo-grid pointsto a quadrilateral.

due to correlation is then given by the differ­ence in position between the original ground­grid square (denoted by the grid square num­ber) and the ground-grid square containingthe position computed from the photo-to­ground projection of the correlated pair ofphoto points.

The fictitious stereograms are idealized inthat gray-scale density variations do not existfor corresponding regions in the two pictures-if both pictures of a pair are derived from areiief model with the same density pattern.Inreal photography, such gray-scale densitydifferences do exist because of differences inlight reflectivity on the ground and differentamounts of atmospheric haze for the twoexposure stations; in addition, the corners of aphotograph are made darker than the rest ofthe photograph because less light passesthrough the camera lens for these regions. Tosimulate real'photography, the computer pro­gram incorporates a routine to interpolate fineground-grid gray-scale density changes(± val ues) from gross grid data determined

manually. The ground grid gray-scale densitypattern may then be modified by these gray­scale changes prior to the generation of afictitious photograph.

Because of electronic, optical, and me­chanical errors in a scanner, linear and non­linear x, y errors in positioning the light beamresul t. This type of error can be measured andeither entered in tabular form or approxi­mated by 1-variable or 2-variable polynomials.

Rotation and translation errors occur whenthe photographic axes (fiducial marks) are notlocated where they are thought to be whensetti ng up the photographic plates for scan­ning.

In addition to these gross changes made togray-scale density, noise may also be intro­duced into the fictitious photo data. For thispurpose, a random-number generator routinewas devised. The muliplicative congruentialscheme in use at MIT was adopted. It pro­duces normally distributed noise (± values)with any desired mean and variance whichbecome the changes to be made to the grayscale density pattern of a photograph.

The following geometric errors, present inany digital system, can also be introducedinto the fictitious data to determine how theyaffect output accuracy:

1. Rotation error2. Translation error3. Linear and non-linear x, y position

errors.

Aerial photographs often have featureswhich appear in one photograph but are notvisible in a second overlapping photograph.These "hidden" features occur wheneverground terrain slope is sufficient to obscurethe terrain from a camera exposure station.

~ PROJECTEDCORNERS

JlI INTERCEPT

* ENCLOSED GRIDINTERSECTION

.-/1-

~v

\ ...-'~1--'

-

.~~- - - ~

y

I( »1REGION 2

PHOTO 2-7"'---+--"2

A'

A"

NADIR

-y

+I

~-t-

+----t­+---------t-+---------t-+---------t-

x

y REGION OFMODEL

FIG. 3. Hidden features. FIG. 4. Projection sequence of model elements.

GENERATING DIGITAL FICTITIOUS STEREOGRAMS 171

FIG. 5. Fictitious stereogram derived from a photo pattern superimposed on mathematical model.

FIG. 6. Fictitious stereogram (with enlarged section) derived from arandom number pattern superimposed on mathematical model.

In Figure 3, for example, region 1 is hiddenfrom photo 1 and region 2 from photo 2. Inthe region of a hidden feature, three groundterrain points (two at points of tangency)project along a single object ray to a singlepoint on the photo (e.g., points A, A', A"project to point a, Figure 3), whereas onlyone point, point A, is visible. To insure thatonly the visible points are projected onto thephoto, the computer program always pro­jects ground-square elements in a sequencemoving inward toward the ground nadir

point (for the given photo-plane) from theoutermost corner of the model, inward inspots along a scan, then inward to the nextscan, etc. (Figure 4). In the computer pro­gram, each photo grid intersection receivesonly the last gray-scale density value that isassigned to it. I n the two dimensional ill us­trations of Figure 3, point a on photo 1 isassigned first the gray-scale value from pointA", then from A', and finally from A, whenthe projection sequence is taken from right toleft toward the nadir point N. The projection

TOP VIEW

(4) s::J['Z~--

SIDE VIEW

FIG. 7. Elevation model with mathematical surfaces used for stereogral1ls in Figures 5 and 6.

172 PHOTOGRAMMETRIC ENGINEERING

Real photography has certain propertieswhich are not simulated by the fictitious datagenerator. These are:

1. Multi-valued elevation data. In realphotography, many situations exist inwhich three elevations are required todescribe an object at a given horizontalposition. For example, detail under atree or under an overhanging roof of abuilding can be seen. The fictitious datagenerator, as designed, can only repro-

FIG. 8. Ground random number pattern forstereograms in Figures 10 and 11.

sequence therefore assures that only thevisible gray-scale density values are trans­ferred to the photo-plane.

Surface

EllipsoidX cylinderInclined planeElliptical paraboloid

TABLE 1

Surface Numbers

1,3,5216All remaining surfaces

100' JAGGED PEAKS

90'

(14)

60'

35':>

(5)c

401

("'~--:'(:::-2):-----..:..------t...-.~~::.I..---------_4

0 1

FIG. 9. Terrain model for stereograms in Figures 10, 11, and 13.

FIG. 10. Fictitious stereogram generated from the terrain model of Figure 9.

GENERATING DIGITAL FICTITIOUS STEREOGRAMS 173

F[G. 11. Stereogram of Figure 10 modified by noise and gross gray scale changes.

duce ground terrain with:slopes of lessthan ± 90 degrees.

2. Shadow effects. With additional pro­gramming, shadows probably can beintroduced into fictitious photo data, ifdesired.

3. Individual local gray shade variation dueto differences in reflectivity. This prop­erty is simulated only crudely by the useof gross gray-scale changes and normallydistri bu ted noise.

4. Those photographic errors which cannotbe adequately defined.

A most realistic fictitious model, in whichthe photo patterns correspond to the terrai ndata, can be generated. I t would make use of adigital photo-data correlation program asfollows:

1. The correlation program is used togenerate two outputs upon correlatinga given stereo pair of real photographs;these are:(a) terrain elevation data(b) an orthophoto based on orthocorrec­

tion of one of the two photographs.2. Using a terrain model made up from

these two outputs of the correlationprogram, two stereo fictitious photos aregenerated.

I t does not matter if the correlation pro­gram produces elevation data which is inac­curate with respect to the actual terrain be­cause these data are now being defined as being"true" terrai n da ta for the ficti tious model.

A second way to obtain a realistic modelrequires a large amount of manual work. Thesteps required are:

1. Contour lines from a contour map aretransferred to one of the two photo­graphs while viewing the diapositives instereo.

2. Fine grid elevation data are interpolatedfrom gross grid elevations extracted fromthe warped contour map overlay on thegiven photograph.

3. Using a terrain model consisting of thefine grid elevation data and the givenphotograph gray pattern, a ficti tiousstereogram is genera ted.

EXAMPLES OF FICTITIOUS STEREOGRA 'IS

Figures 5, 6, 10, II, and 13 are fictitiousstereo photographs prepared on an I B i\l7094-II computer. All are vertical photo­graphs. All gray-scale patterns have a rangeof eight shades of gray (0 to 7).

Figures 5 and 6 are fictitious stereogramsprepared with the same terrain elevation databut with the different gray-scale patternsshown. The elevation data is composed of thesix mathematical surfaces indicated below andappearing in Figure 7, from left to right:

1. Side plane2. Elliptical paraboloid3. Ellipsoid4. Elliptical paraboloid

FIG. 12. Ground photo pattern for stereogramin Figure 13.

174 PHOTOGRAMMETRIC ENGINEERING

FIG. 13. Stereogram with photo pattern modified by noise and gross gray scale changes.

5. Elliptical cylinder rUllning up and down6. Inclined plane in upper right corner

The use of a random number pattern inFigure 6 shows that stereoscopic parallax pro­vides the only depth cue in the stereogram.

Figure 8 shows the basic ground randomnumber pattern used for the stereograms inFigures 10 and 11. The terrain model forFigures 10, 11, and 13 is shown in Figure 9.In this figure are depicted the rough outlinesof surfaces which can be seen in the stereo­grams. The elevation model consists of anupper and lower section. The upper section isfine grid elevation data which was inter­polated from gross grid elevations man uallyassigned and is generally self-explanatory.The lower part consists of mathematicalshapes given in Table 1.

Figures 10 and 11 are the same stereogramexcept that normally distributed noise (with

mean 0 and standard deviation 1.2) and grossgray-scale changes (ranging from +2 to -3)have been introduced into the left and rightphotos of Figure 11, respectively.

For the stereogram of Figure 13, the basicground pattern used was that of Figure 12.The same terrain data (Figure 9) used forFigures 10 and 11, is used in Figure 13. As inFigure 11, noise and gross gray-scale changesare included in Figure 13.

While all features in Figures 5 and 6 in boththe left and right photos are clearly visible tothe eye, there are ground features which arehidden from both photos in Figures 10, 11,and 13. This may be seen by comparing thelatter with Figures 8 and 12, the originalground gray-scale patterns. Close examina­tion of Figure 10 will disclose features visiblein the left photo but not in the right photo.

TABLE 2

Generate unsorted photo data 10.5 16.0Merge and sort photo data 36.0 36.0Introduce gross gray scale

changes 3.6Introduce gray scale noise 3.0Prepare data for recording on

film 13.6 13.6

Total for stereo photo(minutes) 63.1 69.2

Time (Minutes)

Generate ground model elevationdata (640 X 480 spots)

LeftPhoto

6.3

RightPhoto

Because (1) the elevation changes in theterrain model (Figure 9) are quite large and(2) the model is located in the upper leftquadrant relative to the ground nadir pointsof both camera stations, large binocularparallaxes result. Since it is quite difficult toview stereograms 10, 11, and 13 in stereo andbecause they overlap, they are not mountedfor easy stereo viewing. evertheless, thesestereograms are interesting when viewedmonocularly. The terrain features are clearlyidentifiable. (This is particularly true in thecase of Figure 10.) It is apparent that whenhidden features are present, the monoculardepth cue of interposition (the superimposingof near objects upon far objects) comes intoplay [2J.

Total running times on an IBM 7094-II forthe stereograms in Figures 10, 11, and 13 areshown in Table 2.

GENERATING DIGITAL FICTITIOUS STEREOGRAMS 175

SUMMARY

This paper has shown how ficti tious stereophotography can be generated on a general­purpose digital computer. A fictitious terrainmodel must first be constructed from anarbitrary gray-scale pattern and an elevationmodel. The elevation data can be interpolatedfrom a contour map or from manuallyassigned elevations, or it can be computedfrom mathematical shapes. A fictitious stereo­gram is then easily obtained by the mathe­matical projection of the terrain model upontwo tilted photo-planes in space.

Such fictitious data can be utilized as apowerful tool in research on digital correlationtechniques. Since most correlation techniquesunder investigation today for use in automaticmap compilation employ only the binocularparallax property as a clue to depth, the lackof correspondence between gray-scale patternand elevation data, the absence of shadows,

and the inability to reproduce the widevariety of tonal differences between the twophotographs (which are present in real pho­tography) are not detrimental to correlationresearch. Neither is the inability to generatemulti-valued terrain data considered a sig­nificant disadvantage.

ACKNOWLEDGMENTS

The authors are grateful to Mr. C. I.Homan for his major contribution in pro­gramming and in the generation of fictitiousdata and to Mr. A. L. Anthony for his work ingeneration of mathematical terrain models.

REFERENCES

1. Dodge, Hugh F. "Automatic Mapping SystemDesign," paper presented at the 1963 Semi­annual Meeting of the American Society ofPhotogrammetry, Wellesley Island, New York.

2. Julesz, Bela, "Binocular Depth Perception ofComputer-Generated Patterns," Bell SystemTechnical Journal, September 1962, vol.XXXIX, No.5, pp. 1125-1162.

BOARD OF DIRECTION FOR 1964

WILLIAM A. FISCHER .

GOMER T. McNEIL .

CAPT. LAWRENCE W. SWANSON .C. EARL PALMER .

G. C. TEWINKEL... . .

THEODORE W. NORCROSS ....•..

MRS. BARBARA N. MUIR .....

. .President. First Vice President

· .Second Vice President· Secretary- Treasurer

. Editor. Editor Emeritus· .. Advertising Manager

Term expires in 1964CLARICE L. NORTON

MARVIN B. SCHER

HELMUT H. SCHMID

CHARLES THEURER

GLENN C. WELDEN

DIRECTORS

Term expires in 1965HUBERT C. CAIN

JOHN O. EICHLER

HEINZ GRUNER

EUGENE]. SCHLATTER

PAGE TRUESDELL

PAST PRESIDENTS

JAMES P. WEBB AND R. S. QUACKENBUSH, JR.

Term expires in 1966PAUL H. ALBAN

FREDERICK J. DOYLE

WILLIAM C. MAHONEY

ARTHUR R. SHOPE, JR.

JOHN H. WICKHAM, JR.

EXECUTIVE COMMITTEE, 1964

PRESIDENT-William A. FischerVICE PRESIDENTs-Gomer T. McNeil and Lawrence W. Swanson

DIRECTORs-Faul H. Alban, F. J. Doyle, Charles Theurer, Glenn C. Welden

COUNCIL OF THE SOCIETY, 1964

CHAIRMAN-Gomer T. McNeilDEPUTY CHAIRMAN-R. S. Quackenbush

SECRETARy-Glenn A. HarperCOUNCIL REPRESENTATIVES-Clarice L. Norton, Marvin B. Scher, Helmut H. Schmid

AND PRESIDENTS OF ALL LOCAL REGIONS