J. theor. Biol. (1972) 36, 105-l 17

Iterative Methods for the ‘lb&dime&and Recm~n of an Object f?om Projectiom

F%TER GILBERT

Medical Research Council’s Laboratory of Molecular Biology Hills Road, Cambridge CB2 2QH, England

(Received 1 September 1971)

A method of reconstruction (ART) has recently been proposed (Gordon, Bender & Herman, 1970) which consists in Qeratively changing a trial structure until its projections are consistent tith the original projections of the unknown structure. It is shown that in general ART produces errcneous reconstructions. An alternative itetative method is proposed which will give correct reconstructions under ‘certain conditions. One of the potential applications of this method is in &&mining the three- dimensional structure of objects from electron8 micrographs.

1. IntrodlIction

The optical density of an electron micrograph enables the projected density of a structure to be determined when certain tinditions are fulfSd (for a review see Valentine, 1961). The plane of the cknsity projection is normal to the electron beam, and by tilting the specimen stage of the microscope a series of projections of the object in different &k&ions can be obtained.

DeRosier & Klug (1968) first showed how a three-dimensional density distribution could be reconstructed from the projection data contained in a set of electron micrographs. Their method consists in determining the Fourier transform of the unknown object from the Fourier transforms of its observed projections, and then transforming back to obtain the structure. They give a relation between the number of projections used (equally spaced in angle around a common axis) and the resolution of the final reconstruction. Crowther, DeRosier & Klug (1970) extended the method to apply to pro- jections in arbitrary orientations relative to one another.

The problem of reconstructing an object from its projections has arisen in fields of research other than electron microscopy (for example X-radio- graphy and radio-astronomy) and Bracewell & Riddle (1967) and Berry & Gibbs (1970) have proposed reconstruction methods which are direct in the sense that no Fourier transformations are required. Direct methods of

105

106 P. GILBERT

reconstruction are also described by Crowther et al. (1970), Vainshtein (1970) and Gilbert (1970). These methods are discussed in detail elsewhere (Gilbert, 1972) and in particular the method of Bracewell & Riddle is shown to be equivalent to the Fourier method of DeRosier & Klug (1968) when applied to projections equally spaced in angle about an axis.

As well as these direct methods, another direct method of reconstruction, an Algebraic Reconstruction Technique (ART), has been proposed by Gordon, Bender & Herman (1970). This method has been the subject of some controversy (Crowther & Klug, 1971; Bellman, Bender, Gordon & Rowe, 1971) because the authors claim it has considerable advantages over the Fourier method. It will be demonstrated that the claims made for ART algorithms are not justified and that, in general, far from producing better reconstructions than the Fourier method, considerably worse results are obtained. The reason for the successful reconstructions of the model examples given by Gordon et al., is that they used an unjustified procedure for generat- ing the data used for the reconstructions. In this paper a different iterative method is proposed which gives results which are in close agreement with those produced by the Fourier method.

The Fourier method is rigorous and the conditions for a correct recon- struction are understood, and it might therefore appear that there is no need to produce other methods of reconstruction. The main reason for investigating direct methods is to try to use them to produce better reconstructions of an object from its projections than can be obtained with the Fourier method. This may be possible with direct methods by introducing into the reconstruc- tion procedure information about the object which is not used in the Fourier method. Thus the Fourier method does not utilise the fact that the density of the object is everywhere positive. Theoretically it would be possible to use this condition of positivity in the Fourier method because certain constraints are placed on the Fourier transform of the density distribution (Karle & Haupt- man, 1950) but in practice these constraints are difficult to apply, and have not yet been used in a Fourier reconstruction. In certain direct methods, however, it is very easy to ensure positivity in reconstructions, as will be demonstrated.

2. Tbe Algebraic Reeoustruction Technique

For convenience, a special case of reconstruction will be considered in which the planes of all the projections intersect along a single line, but all the methods to be described can readily be general&d to apply when there is no such common line of intersection. The density in any plane of the object perpendicular to this line will only contribute to the projections in this plane,

ITERATIVE METHODS OF RECONSTRUCTION 107

The reconstruction of the object therefore consists of a set of two-dimensional reconstructions to be carried out in planes perpendicular to the line of inter- section of the projections. Thus we need only consider the reconstruction of a two-dimensional object from a number of one-dimensional projections.

Consider the reconstruction of a roughly isometric object where we wish to tidthedensityp,,onagridofnxnpoints(i,J),i= 1,2 ,..., n;j= 1,2 ,..., n. The field of this grid must be slightly greater than the dimensions of the object (the linear dimensions of the object to be reconstructed can be deduced approximately from the projection data). Using the terminology of Gordon et al. (1970) a ray of a projection (or view) at angle 8 is defined as a band of width w across the plane at that angle (see Fig. 1). Let R,, be the sum of the densities pu contained within the ray k of the projection at angle 6. Thus

R, = r, Pii? k=12 rO. , ,***, (0 points (W in w mn

The number of rays r, in a projection at angle 8 depends on the width of the rays and the size of the object whose density is reconstructed. The points k on the projections are at the midpoints of the rays (see Fig. 1). Let the true projected densities of the object at these points be Pke. The problem of reconstruction is to determine the pi, from Pke.

The total density, T, of the object can be deduced approximately from the sum of the projected densities Pke for any projection at angle 13. The degree of approximation depends on the sire of the sampling interval between the points PM on the projections. The procedure of Gordon et al. is iterative, and initially all the pu are set equal to the approximate mean density of the structure. Thus

p$ = T/n2 i,j=1,2 n. , *a*, (2)

These initial densities are now iteratively changed by one of two different procedures. Let Rfo be the sum of the densities pf, at the qth iteration con- tained within ray k of projection 8. Then

R& = c pfi k = l,...,r,. (3) the NM points (id in my (k.0)

In their multiplicative method the density of a point (iJ) is changed by the rule

PFj+l = (Pke/%) Pfj (4) for (i, j) in ray (k, t9), and in the additive method by

PFJ+ = max bf, + cpk, - R%)/Nm 01. (5) These formulae (4) or (5) are applied to all rays of a given projection at angle 8 and then used for the next projection, and so OIL In equations (4) and (5) and subsequent equations the rays are taken to have a width equal to the spacings

108 P. GILBERT

FIG. 1. Densities p,, of the object are reconstructed from projection data Pke at points on the array shown. The figure shows the kth ray of width w and at an angle 8. The length, Lke of the ray is ad, and the sum of densities at the points within ubcd at the qth iteration of the reconstruction algorithm is RR@. The projection of the density of the object along the middle of the ray (shown as a broken line) is Pke.

between points on the reconstruction grid. The projected densities Pke are expressed in terms of the same unit of distance.

The form of algorithm (5) ensures that all the reconstructed densities will be positive. Similarly with the multiplicative algorithm (4) all the reconstructed densities will be positive if the projected densities Pke are all positive.

Gordon ef al. (1970) have tested these formulae with model examples. However they apparently generated their projection data by measuring the densities pii of two-dimensional objects and then assuming that the true projected densities Pke were given by R,,, the sum of the densities Pij con- tained within the rays. Obviously if the ray width and the spacings between the points pii both tend to zero, then R,, will tend to Pke, but, when these distances are finite Pke is only approximately equal to R,,. Using these approximate projected densities Rke (which will be referred to as pseudo- projection data) they showed that with successive iterations the pt converged

ITERATIVE METHODS OF RECO~NSTRUCTION 109

to stable values after about 20 cycles. For both the additive and multiplicative methods the root mean square deviation of the reconstructed density from the original density also decreased with successive iterations. This is illustrated in Fig. 2(a) and (b) for the reconstruction of the model structure shown in

Number of iterations

FICA 2. (a) The deviation 6 [equation (6)] vs. number of iterations for the reconstruction of the object shown in Fig. 3(a), using 10 equaliy-spaa?d projections. The results of the multklicative ART. additive ART and additive SIRT ahtorithms when auolied to the true projekon data PMram shown by curves (1). (2) and (9, respectively. l%r curve (4) the additive ART algorithm was used for the reconstruction8 with the pseudo-projection data & (see text).

(b) As for (a) using 25 equally spaced projections for the reconstructions.

Fig. 3(a), using the additive method, and 10 and 25 equally-spaced projections respectively. The discrepancy 6 between the original density and the re- constructed density, where

(6)

is shown in Fig. 2 for each successive round of iteration. The summations in equation (6) are taken over the array of points contained within the circular boundary shown in Fig. 3(a)-the reconstructions also assumed that there was no density outside the circular region shown.

However, very different results were obtained with the multiplicative and additive ART methods when they were used with true projection data. Each of the circular regions in the model structure shown in Fig. 3(a) was taken to have uniform density and the projections were calculated analytically. The projected density values Pks were determined for each projection at points corresponding to the midpoints of the rays. F&ure 2(a) and (b) show the deviation 6 of the reconstructed densities from the original for successive iterations using 10 and 25 projections respectively. With both ART methods

110 P. GILBERT

the reconstructed density becomes closer to the original density in the first few cycles, and then starts to deviate from it more and more with subsequent iterations. Although the two-dimensional reconstructed densities diverge from that of the original structure, the one-dimensional projections R& of the reconstructed densities agree more and more closely with the projections Pke of the original structure as the iterations are continued. Since only the projection data are available when an unknown density is reconstructed there is no criterion for stopping the iterations when an unknown structure is reconstructed and very poor reconstructions could result [Figs 3(c), 3(d), ‘WI.

It is thus apparent that when the ART algorithms (4) or (5) are used with the true projection data considerably worse reconstructions are produced than when the pseudo-projection data R,, are used. Crowther & Klug (1971) pointed out that generating pseudo-projection data from the original density, using the same procedure for the determination of the sums of the densities in the rays as for the reconstruction, was not a valid test of the algorithms. Thus using Rke as data for the reconstruction leads to an exactly consistent set of equations for the densities Pij; the ART algorithms appear to be able to determine the Pij only when the equations are consistent and the slightest deviation from consistency causes erroneous reconstructions. Even the ART reconstructions corresponding to the iterations at which 6 reaches its minimum value are very poor, when the true projection data are used, and moreover are not the best reconstructions that can be obtained with the data, as will be demonstrated later. If random Gaussian errors are introduced into the projection data to simulate the effects of experimental error then the multi- plicative and additive ART methods give even worse reconstructions (especially the multiplicative method), with larger discrepancies 6 between the reconstructed and original densities.

Because both the multiplicative and additive ART algorithms make the densities in the rays R,, converge towards the true projected densities Pke, the reconstructed densities produced by these techniques must bear some relation to the true density. The methods appear to reconstruct the true density “peppered” with random errors at each density point. These errors increase with the number of iterations for a particular reconstruction, and also increase as larger numbers of projections are used in a reconstruction. Of course it is impossible to interpret satisfactorily a reconstruction containing arbitrarily large errors and not much reliance can be placed on the “great deal of structural detail” contained in the reconstruction of a ribosome by the multiplicative method (Bender, Bellman & Gordon, 1970).

A good method of reconstruction should not introduce any false detail into the reconstructed density, and the density values at adjacent points

ITERATIVE METHODS OF RECONSTRUCTION

Fra. 3. (a) The model object used for the reconstructiohs, with the density values shown. In calculating the true projections to be used for reconstructions the density was taken to be uniform within the ckcular regions indicated.

(b) A reconstruction of the object shown in (a) using the true projection data Pk, from equally-spaced projections after 20 cyck of iteration of the additive SIRT algorithm. The recunstruetion is eontoured at a level of 10 density units, which is intermediate in level between the background density and the circular regions of high density shown in (a).

(c) As for (b) but using the additive ART algorithm. (d) As for (5) but u&g the multiplicative ART algorithm.

should vary as smoothly as possible in a manner which is consistent with the projection data available. ART does not reconstruct in this way when the true projection data are used. On the other hadd the Fourier method does reconstruct in this manner b~yse a resolqtioq limit is determined from the

P. GILBERT

FIG. 4. (a) A reconstruction of the object shown in Fig. 3(a) using true projection data from 25 equally-spaced projections, and the method of Bracewell & Riddle (1967) which is equivalent to the Fourier method (see text).

(b) As for (a) but using 20 cycles of iteration of the additive SIRT algorithm for the reconstruction.

(c) A reconstruction using 20 cycles of iteration of the additive ART algorithm and pseudo-projection data Rke (see text) from 25 equally-spaced projections.

(d) As for (c) but using the true projection data Pke.

nature of the projection data available (Crowther et al., 1970), and no Fourier components with a frequency higher than this limit are allowed into the Fourier synthesis which reconstructs the object.

ITERATIVE METHODS OF RECONSTRUCTION

3. The Slnmltraeorro Iterative R-on Teddqae

113

An iterative algorithm can be formulated which gives reconstructions which are very similar to those produced by the Fourier method. The method can conveniently be called the Simultaneous Iterative Reconstruction Tech- nique (SIRT) because at each iteration the densities are changed by using data from all the projections simultaneously, in contrast to ART where densities are altered using data from one projection at a time. Using the same notation as above and setting & to the mean density, then the additive SIRT algorithm is :

max (7)

where L, is the length of the ray R,, (see Fig. 1). The multiplicative SIRT algorithm is

The summations in equations (7) and (8) are over all the projection points (k, 0) to which the density point pu contributes, i.e. the number of terms in each summation equals the number of projections used.

After algorithms (7) or (8) have been applied to all the densities pfi then the densities #,,’ l are scaled so that their sum over all grid points is equal to the sum of p&

With the true projection data, Pko, of the test object shown in Fig. 3(a) the additive and multiplicative SIRT algorithms gave very similar reconstructions. The deviation 6 of the reconstruction from the original with successive iterations is shown in Fig. 2(a) and (b) for lol and 25 equally-spaced pro- jections, respectively, using the additive method. Although the reconstruc- tions using this method converge more slowly towards the original than with the ART method in the first few iterations, they go on converging with increasing iterations, while the ART reconstructions start to diverge from the original. The SIRT reconstructions have a minimum discrepancy 6 from the original after about 10 cycles of iteration for the object shown. At this mini- mum the reconstruction is considerably closer to the original than the best ART reconstructions (Fig. 2), and with increasing numbers of projections better reconstructions are obtained, which is not the case with ART algorithms. After this minimum is reached the SIRT reconstructions start to diverge from the original but much more slowly than the ART reconstructions. Thus after 20 iterations 6 is still within 1% of its minimum for both the reconstructions illustrated [Figs 3(b), 4(b)]. A s can be seen these reconstructions are much better than those produced by ART, and they have discrepancies 6 which are

T.B. 8

114 P. GILBERT

within a few percent of those for reconstructions produced by the method of Bracewell & Riddle (1967).

In the presence of errors in the projection data the SIRT algorithms give satisfactory reconstructions, which the ART algorithms do not. After about 10 cycles of iteration the reconstructions have discrepancies 6 from the original object which are very close to the values of S for Fourier reconstruc- tions. Thus when random errors with a Gaussian distribution and a standard deviation of 10% of the projected density values were introduced into the projection data the ART reconstructions had 6 > O-97 after 10 cycles using both 10 and 25 projections. The corresponding additive SIRT reconstructions had 6 = 0.63 (O-66) and 6 = 0.52 (O-54) for 10 and 25 projections, respectively (the values of 6 using the Fourier method are given in parentheses).

Both the ART and SIRT reconstructions are affected by the boundaries which are chosen for the reconstruction grid. Obviously the boundary must enclose the object to be reconstructed or there will be serious errors in the reconstruction. If there are a large number of projections of the object, equally spaced in angle around an axis, then it is possible to estimate the boundary of the object quite accurately from the projection data, and little error is introduced into the reconstruction from this source. However, if only a few projections in a small range of directions are available then it is not possible to estimate the boundaries of the object accurately, and large errors can be introduced into the reconstruction.

There are two features of the SIRT algorithms which distinguish them from the ART algorithms, and which are responsible for them giving better reconstructions. One of these differences between the SIRT and ART algorithms is a major one, and is responsible for the increased stability of the SIRT algorithms in the presence of errors in the projection data, while the other difference is comparatively minor. The major difference is that, in each cycle of iteration of SIRT, data from all the projections are applied simultaneously to changing the density ~7, of a point on the reconstruction grid, whereas for ART the data from projections are applied sequentially during an iteration cycle.

The less important difference between the SIRT and ART algorithms follows from an effect pointed out by Dr R. A. Crowther (private com- munication). In the ART algorithms serious errors are introduced because the number of points Nke in a ray of given length can vary widely according to the angle 0 of the projection. Thus a ray of unit width and length L,, will contain roughly Lke points pii at 8 = 0 (see Fig. l), but when 6 = x/4 the ray can contain as many as JZL,, points or as few as (1 /Jz)L, points depending on the position of the ray relative to the reconstruction grid. There is a similar effect for rays of projections at angles between these two extremes,

ITERATIVE METHODS OF RECONSTRUCTION 115

This phenomenon causes substantial errors in the algorithms (4) and (5) when rays of unit width are used, because the sum of points pli in a ray RM can never be a good estimate of the true projected density PM except when 0 is close to 0 or x/2. One way of correcting this source of error would be to use rays of a large width, compared to the distance between points on the re- construction grid, in equations (4) and (5). This problem is overcome in the SIRT algorithms by dividing the projected densities Pre by the lengths of the rays he, and the ray densities RM by the numbers of points N, in the rays. This modification could also be introduced into the ART algorithms, in which case equation (4) would be replaced by

P$- = U’,N,IL&bl pf.9 (9 and (5) would become

Pf:l = ma bfj + (PM/L, - WNd 01. (10) Although algorithms (9) and (10) when applied to perfect projection data do in fact give considerably better reconstructions than the ART algorithms (4) and (5), they do not give such good reconstructions as the SIRT algorithms. But above all algorithms (9) and (10) do not remedy the main defect of the ART algorithms since they are just as vulnerable to errors in the projection data as the ART algorithms. This was shown empirically by adding random errors with a Gaussian distribution and a standard deviation of 10% of the projected density values to the projection data. Under these conditions the reconstructions produced with algorithms (9) and (10) all had 6 > 0.98 after 10 cycles of iteration when both 10 and 25 projections of the model object shown in Fig. 3(a) were used (the values of 6 when the SIRT and ART algorithms were used in a similar trial are given above).

It was explained previously that the ART and SIRT algorithms were tested with model objects composed of circular regions of uniform density in order that the projection data could be calculated analytically and not by inter- polation. The algorithms were also tested with another class of objects composed of squares of uniform density with the projection data again being calculated analytically. The results obtained were very similar to those already described for the model example composed of circular discs, with the SIRT algorithms producing satisfactory reconstructions and the ART algorithms giving unsatisfactory results.

Therefore, of all the iterative methods described, the SIRT algorithms give the best reconstructions, and are the least sensitive to errors in the projection data. However, some care is required in the application of the SIRT algorithms because, as shown above for a particular modeI structure, the discrepancy 6 between the original and reconstructed densities reached a minimum after about 10 cycles of iteration and then started to increase gradually with

116 P. GILBERT

subsequent iterations. This is a similar effect to that which occurs for the ART reconstructions, except that the increase of 6 with an increasing number of iterations using SIRT is considerably smaller than the increase with ART. Because it is such a small effect a rigorous criterion for terminating the SIRT procedure after a certain number of iterations is not crucial in order for satisfactory reconstructions to be produced. Thus it was found, for a range of model examples, that terminating after 10 to 15 iterations gave reconstruc- tions very similar to those obtained by the Fourier method (with the S values for reconstructions produced by the two methods agreeing within a few percent). As iterations continue beyond the point where a minimum value of 6 is reached it appears that random errors are added to the reconstructed density at each grid point, so that the reconstructed density varies less and less smoothly between adjacent grid points with an increasing number of iterations. Preliminary results indicate that this effect can be reduced, or even eliminated completely, by decreasing the spacing between points on the reconstruction grid. Work is in progress to determine the optimal spacing between points on the grid and of the width of rays to ensure that the SIRT algorithms give the smoothest reconstructions consistent with the data available after any number of iterations.

The “peppering” of the reconstructions described above is similar to that which would occur in a Fourier reconstruction if it were used without the correct resolution criterion. Thus in the Fourier method a smooth recon- struction is only assured by imposing a resolution limit which is dependent on the size of the object, the number and distribution of views available, and on the quality of the data in each projection; if an attempt is made to solve a structure to a higher resolution than is given by this criterion then not only will the reconstructed density contain errors, but also any errors in the original projection data will be magnified in the fural reconstruction (Crowther et al., 1970).

In conclusion, therefore, a new iterative method of reconstruction, SIRT, has been proposed. This method produces reconstructions very similar in quality to those obtained by the Fourier method, although the exact rules governing its application have not yet been determined. It has also been demonstrated that the ART algorithms (Gordon et al., 1970) give very poor reconstructions, which are much worse than those produced by either the SIRT algorithms or the Fourier method.

I would like to thank Drs R. A. Crowther and A. Klug for many helpful dis- cussions about this work.

ITERATIVE METHODS OF RECONSTRUCTION 117

REFERENCES BELLMAN, S. H., BENDER, R., GORDON, R & ROWE, J. 1. (1971). J. theor. Biol. 32,205. BENDER, R., BELLMAN, S. H. & GORDON, R. (1970). J. t&eor. Biol. 29,483. BERRY, M. V. & GIBBS, D. F. (1970). Proc. R. Sot. A 314,143. BRACEWELL, R. N. & &DDLE, A. C. (1967). Astrophyz J. 150,427. CROWTHER, R. A., DBROSER, D. J. & I&JO, A. (1970). Proc. R. Sot. A 317,319. CROWTEDX, R. A. & KLUO, A. (1971). J. theor. Bill. 32,199. DsRosm, D. J. C RL.UG, A. (1968). Nature, Land. 217,130. GILBERT, P. F. C. (1970). Ph.D. Thesis, CambrLIge University, England. GILBERT, P.F.C. (1972). Proc. R. Sot. A (ii pnzss). &XtDON, R., BENDER, R. & HERMAN, G. T. (1970). L theor. Biol. 29,471. K.uua,J.&H~~mmm, H. (1950). Actu crystulfogr. 3, 181. VAINSHTEIN, B. K. (1970). Kristallgrafia 13,894. Translated in Soviet Phys. Crystalliogr. 15,

781 (1971). VALENTINE, R. C. (1961). A&. Yirus Res. 8,287.