A resolution criterion for electron tomography based on...

Journal of

www.elsevier.com/locate/yjsbi

Journal of Structural Biology 151 (2005) 117–129

StructuralBiology

A resolution criterion for electron tomography basedon cross-validation

Giovanni Cardone a, Kay Grunewald b, Alasdair C. Steven a,*

a Laboratory of Structural Biology, National Institute of Arthritis, Musculoskeletal and Skin Diseases, National Institutes of Health,

Bethesda, MD 20892, USAb Department of Molecular Structural Biology, Max Planck Institute of Biochemistry, 82152, Martinsried, Germany

Received 18 March 2005, and in revised form 27 April 2005Available online 31 May 2005

Abstract

Despite much progress in electron tomography, quantitative assessment of resolution has remained a problematic issue. The cri-teria that are used in single particle analysis, based on gauging the consistency between density maps calculated from half data sets,are not directly applicable because of the uniqueness of a tomographic volume. Here, we propose two criteria based on a cross-val-idation approach. One, called FSCe/o, is based on a Fourier shell correlation comparison between tomograms calculated from theeven and odd members of a tilt series. The other, called noise-compensated leave-one-out (NLOO), is based on Fourier ring corre-lation comparisons between an original projection and the corresponding reprojection of the tomogram calculated from all the otherprojections, taking into account the differing noise statistics. Plotted as a function of tilt angle, they allow assessment of the angulardependence of resolution and quality control over the series of projections. Integrated over all projections, the results give a globalfigure for resolution. Tests on simulated tomograms established consistency between these criteria and the FSCref, a correlation coef-ficient calculated between a known reference structure and the corresponding portion of a tomogram containing that structure. Thetwo criteria—FSCe/o and NLOO—are mutually consistent when residual noise is the major resolution-limiting factor. When the sizeof the tilt increment becomes a significant factor, NLOO provides a more reliable criterion, as expected, although it is computation-ally intensive. Applicable to entire tomograms or selected structures, NLOO has also been tested on experimental tomographic data.Published by Elsevier Inc.

Keywords: Fourier shell correlation coefficient; Anisotropic resolution; Signal-to-noise ratio

1. Introduction

Electron tomography (ET) is a rapidly developingthree-dimensional imaging technique that bridges theresolution gap between cryo-EM/single-particle analysis(SPA) and optical microscopy (Grunewald et al., 2003a;McEwen and Koster, 2002). Its most notable feature isits general applicability: ET may be used to study thestructural organization of complex molecular objectsin situ (e.g., Medalia et al., 2002; Perkins et al., 2001)or of individual isolated complexes (e.g., Nitsch et al.,

1047-8477/$ - see front matter. Published by Elsevier Inc.

doi:10.1016/j.jsb.2005.04.006

* Corresponding author. Fax: +1 301 443 7651.E-mail address: [email protected] (A.C. Steven).

1998). On the other hand, the capability to visualize sin-gle specimens imposes limitations for vitrified speci-mens, in terms of the maximum tolerable electron doseand other technical issues.

In this context, quantitative assessment of resolutionremains a key issue. Standard SPA criteria cannot be ap-plied to electron tomograms in a straightforward way.In the Fourier shell correlation (FSC) (Harauz andvan Heel, 1986) and the differential phase residual(DPR) (Frank et al., 1981)—two measures commonlyused in SPA—the input projections are randomly divid-ed into half-sets, and resolution is measured in terms ofthe consistency between the resulting density maps. Theefficacy of these measures relies on the (usually high)

mailto:[email protected]

118 G. Cardone et al. / Journal of Structural Biology 151 (2005) 117–129

redundancy of the input data: when the number of pro-jections is limited, as in tomography, they may beexpected to underestimate the resolution obtainablefrom the full data set.

If a tomogram happens to contain a recognizablecomponent whose structure is already known (fromstructural analysis by SPA or crystallography) to higherresolution than is likely for the tomogram, it may beused to estimate the resolution of the tomogram bycalculating a FSCref correlation coefficient (Grunewaldet al., 2003b). (Similar bootstrapping is done in resolu-tion estimation for SPA in situations in which a recon-structed complex contains a component for which ahigh resolution crystal structure exists e.g., Ludtkeet al., 2001; San Martin et al., 2001.) This approach isintuitively appealing but is limited to tomograms thatcontain a suitable reference structure, which will not ingeneral be the case.

To date, there have been few formal measurements ofresolution in electron tomographic studies and investi-gators have tended to be guided by visual clues concern-ing the visibility or otherwise of certain features in thetomogram. Recently, some approaches to resolutionassessment have been proposed (Penczek, 2002; Tayloret al., 1997; Unser et al., 2005—see Section 4). However,no such measure has yet entered widespread use.

In statistics, the term cross-validation (Efron, 1979)covers a wide class of model evaluation methods whichhave in common the separation of the input data intotwo sets—one for performing the analysis, and the otherfor assessing its goodness-of-fit. This approach has beenpursued in many fields. In crystallography, it underliesthe R-factor, RFree (Brunger, 1992), which gives an unbi-ased measure of the consistency between atomic modelsand diffraction data. In EM, cross-validation has beenused to quantify the bias introduced by a reference mod-el in projection-matching (Shaikh et al., 2003). Theseauthors suggested a procedure to derive a Free-FSCcurve for SPA, i.e., a curve theoretically free from refer-ence bias. Here, we present a method for assessing theresolution of tomograms based on cross-validation inthe projection domain.

2. Theory

2.1. Definitions

A tomographic tilt series consists of Np projectionimages. We define X ðiÞ

m;n as component (m,n) of theFourier transform of projection i. The correspondingcomponent for the reprojection calculated from thetomogram is ~X

ðiÞm;n, and ~X

�ðiÞm;n for the reprojection of

the tomogram from which projection i was excluded.The Fourier ring correlation (Saxton and Baumeister,1982) between transforms F(i) and G(i) is given by

FRCðiÞFGðkÞ¼

Pm;n2RðkÞRe F ðiÞ

m;nGðiÞ�m;n

n oP

m;n2RðkÞ FðiÞm;n

�� 2� � Pm;n2RðkÞ G

ðiÞm;n

�� 2� �� 1=2;

ð1Þ

where the asterisk denotes the complex conjugate, k isthe radial frequency, and R (k) refers to an annular zonein Fourier space of mean radius k. Its width is given bythe reciprocal of the minimum dimension of the volumeunder analysis. The Fourier shell correlation for three-dimensional data is analogous to Eq. (1), except thatin this case R (k) is a shell and the frequency componentsof a transform have three indices (m,n,p).

2.2. The ideal experiment

We first need to define a correlation-based measure ofresolution that is applicable in tomography. Operation-ally, resolution is limited by noise as well as by resolvingpower. In this context, the Fourier shell correlation(FSC) between two maps of the same volume, calculatedfrom independent but statistically equivalent data sets, isrelated to the signal-to-noise ratio (SNR) of the maps by(Frank and Al-Ali, 1975; Penczek, 2002)

FSCðkÞ ¼ SNRðkÞSNRðkÞ þ 1

. ð2Þ

We define resolution in terms of the FSC comparing twosuch maps. This definition is idealized in that it requirestwo reconstructions from independent �full� data sets,which is not usually possible. However, we show belowthat approximations to this measure of resolution maybe obtained by defining proper criteria.

2.3. The even/odd Fourier shell correlation

In SPA, the FSC compares two density maps calcu-lated from half data sets of single projections of manynominally identical particles. Since, in the general case,a tomogram represents a unique volume, there will notbe two such maps to compare. However, independenttomographic representations of the same volume maybe calculated from subsets of projections in a tilt series.A natural division is to select the even and odd projec-tions, respectively. This strategy guarantees the mostuniform coverage in the Fourier domain and retainsthe property of equally spaced projections. If the num-ber of projections (angular increment) is the resolu-tion-limiting factor, the resolution of thesubtomograms should be lower by a factor of two thanthat of the full tomogram, according to the formula

r# � d# � D; ð3Þwhere D is the diameter of the particle and d# is the tiltincrement in radians (Bracewell and Riddle, 1967;

G. Cardone et al. / Journal of Structural Biology 151 (2005) 117–129 119

Crowther et al., 1970). This formula assumes a noise-free situation. In practice, noise may place a morestringent limit on the resolution of tomograms, andthe limited range of accessible tilt-angles imposes a fur-ther constraint (Frank, 1992).

In SPA, maps from half data sets are necessarily nois-ier than the map obtained from the full data set: accord-ingly, the FSC under-estimates the resolution of thelatter map. To compensate for this effect, considerationsof signal-to-noise (Rosenthal and Henderson, 2003)allow one to derive a relationship between the FSC(k)calculated in this way and FSC 0(k), a measure thatapproximates the ideal experiment defined above, i.e.,

FSC0ðkÞ ¼ 2FSCðkÞFSCðkÞ þ 1

. ð4Þ

This equation assumes that the SNR for each map froma half data set is half that of the map from the full dataset. This assumption is reasonable in SPA because of theexpected redundancy of input projections for each orien-tation, while it is less likely to be satisfied in electrontomography. Nevertheless, we empirically adopt thiscorrection in defining the even/odd FSC, i.e., first wecalculate a correlation coefficient FSC*(k) from thetwo half tilt series tomograms, and then correct it asfollows

FSCe=oðkÞ ¼2FSC�ðkÞ

FSC�ðkÞ þ 1. ð5Þ

Since each subtomogram has some information contentthat is missing from the other, according to the centralsection theorem (Kak and Slaney, 1988), it follows thatthe accuracy of FSCe/o (k) in estimating the resolution ofthe full tomogram depends on the degree of overlap inthe Fourier domain between adjacent projections inthe tilt series: the larger the tilt increment or the diame-ter of the structure under analysis, the greater is the res-olution underestimate given by FSCe/o (k). However, wehave observed that the estimate provided by FSCe/o is inagreement with that given by other criteria that are notlimited in this way to the same extent (see below), pro-vided that that the resolution obtained, r, is inferior tothe critical resolution, according to

r > rcrit ¼ 2 � r#; ð6Þwhere the limit r# is posed by the angular increment ofthe full tilt series Eq. (3).

2.4. The leave-one-out method

Since the number of projections is likely to limit res-olution in ET, at least for moderately thick biologicalspecimens, halving this number is a drawback, despitecompensatory approximations (above). An alternativeapproach to assessing self-consistency is to compare asingle projection with the corresponding reprojection

of a tomogram calculated from all the other projectionsexcept the projection in question, which is omitted toavoid bias. Since only one projection is missing, the res-olution of this tomogram should be very close to that ofthe complete tomogram. This approach, which we referto as leave-one-out (LOO), represents another exampleof cross-validation. It is, in fact, an extreme case ofthe many possible ways in which the full tilt series couldbe partitioned and analyzed.

Since there are Np input projections, an equal numberof tomograms should, in principle, be calculated. How-ever, an adequate assessment of resolution may beachieved with a subset of LOO calculations (see below).Although this approach is computationally intensive, ithas compensations. Plotting resolution against tilt angleallows one to assess the dependence of resolution upondirection and to appraise the tilt series.

2.5. 2D noise-compensated leave-one-out estimation

Since projections and reprojections have differingnoise statistics, one must allow for this distinction inusing them to define a measure of resolution. We assumethat the noise in the input projections is additive. Thereprojections also have a noise component, but its vari-ance is reduced in proportion to the number of projec-tions used to calculate the tomogram. Furthermore,the tomogram can be calculated only for a partial thick-ness of the specimen, or we may be interested only in aspecific structure confined to a subvolume. In the lattercases, reprojections from the selected subvolume containjust the signal of interest, while the corresponding inputprojections also have contributions from adjacent fea-tures. In the following, contributions to the input projec-tions from any object that is not part of the (sub)volumeselected for resolution analysis is treated as noise.

On this basis, we estimate the noise as the differencebetween the input projection and the correspondingreprojection from the tomogram generated from all

the input projections. The discrepancy between an inputprojection and the corresponding reprojection from atomogram reconstructed without that projection canbe quantified by means of a normalized square differ-ence. Thus, exploiting the relationship between correla-tion and normalized square difference (Appendix A), wedefine the noise-compensated leave-one-out in two-di-mensions (NLOO-2D) measure as

NLOO-2DðiÞðkÞ ¼FRCðiÞ

X ~X�ðkÞ

FRCðiÞX ~X

ðkÞ; ð7Þ

where FRCðiÞX ~X

�ðkÞ and FRCðiÞX ~X

ðkÞ are defined in Eq. (1).Here, the denominator is a noise compensation term.This measure, NLOO-2D, closely approximates thatwhich would be obtained in the ideal experiment,i.e., by comparing the entire tomogram, via FSC, to a


hypothetical additional tomogram calculated from anindependent but statistically equivalent tilt series(Appendix B).

Since the 2D Fourier transform of a projection formsa slice of the 3D Fourier transform of that volume, eachNLOO-2D(i) (k) curve can be assumed to provide a res-olution for the corresponding slice of the reconstructedvolume.

2.6. Noise-compensated leave-one-out estimation in 3D

We may generalize NLOO-2D(i) to define a measureof resolution for the whole tomogram,

NLOO-3DðkÞ ¼

PNp

i

Pm;n2RðkÞ

Re X ðiÞm;n

~X�ðiÞ�m;n

� �ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiPNp

i

Pm;n2RðkÞ

X ðiÞm;nj j2

q ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiPNp

i

Pm;n2RðkÞ

~X�ðiÞm;n

�� 2qPNp

i

Pm;n2RðkÞ

Re X ðiÞm;n

~XðiÞ�m;n

� �ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiPNp

i

Pm;n2RðkÞ

X ðiÞm;nj j2

q ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiPNp

i

Pm;n2RðkÞ

~XðiÞm;n

�� 2qð8Þ

(see Section 2.1 for definitions). This expression is ob-tained from (7) by simply performing further summa-tions over both tilt series. (For the sake of clarity,redundant terms in (8) have not been eliminated). Thus,NLOO-3D may be interpreted as the ratio between twoFourier shell correlations.

2.7. Dependence of resolution on tilt-angle and location

within the volume

The quality of a tomogram is usually not homoge-neous. In particular, the limited angular range leavesan unsampled portion of Fourier space—the ‘‘missingwedge’’ for single-axis tilt series, or ‘‘missing pyramid’’for dual-axis tilt series—that results in inherently aniso-tropic resolution (Frank, 1992). NLOO-2D, applied toeach projection, provides a straightforward way to as-sess resolution as a function of the tilt-angle. Theincreasing thickness of the specimen at higher tilt-angles,as well as focal gradients and uncertainties in alignmentparameters, also contributes to non-uniformity of reso-lution. To assess it, both criteria may be applied toselected subvolumes.

Fig. 1. A projection of the HBV capsid (Conway et al., 1997) viewedalong a 2-fold symmetry axis, before (left) and after (right) addition ofGaussian noise (SNR = 1). Bar, 100 A

3. Results

To explore the performance of NLOO-3D andFSCe/o as measures of resolution, we compared themwith results given by a reference FSC. This was donein two ways. First, we tested them on synthetic tomo-grams calculated from tilt series that were derived com-putationally from a reference density map. In thisexperiment, the reference structure occupied the fulltomographic volume. Then we applied the measures to

a real cryo-ET data set, for which a reference map wasavailable for one (minor) component of the reconstruct-ed volume. In both cases, we calculated the referenceFSC over the Fourier domain covered by the tilt series.

All resolution estimates were determined at a thresh-old of 0.3, unless otherwise specified. According to Eq.(2), this choice corresponds to a SNR(k) close to 0.5(0.429), i.e., the resolution is identified as the (highest)radial frequency at which the spectral density of the sig-nal falls below half that of the noise. All reconstructionswere performed using the R-weighted back-projectionalgorithm as implemented in IMOD (Kremer et al.,1996). Both NLOO-3D and FSCe/o have been imple-mented in ELECTRA (ELECtron Tomography Resolu-tion Assessment), a software package written in C thatrelies on Bsoft (Heymann, 2001). Running on manyUnix-like platforms, it is freely available from theauthors, on request.

3.1. A model experiment

A density map of hepatitis B virus (HBV) capsid, aspherical particle of �310 A in diameter, was used togenerate tilt series. This map has an isotropic resolutionof 9.1 A, according to the FSC with a threshold of 0.5(Conway et al., 1997), and is sampled at 2.63 A. Projec-tions were generated around a single tilt axis, coveringthe range �68� to +68�. Three series were calculated,corresponding to angular steps of 1�, 2�, and 4� (137,69, and 35 images). White Gaussian noise was addedto the projections at a SNR of 1 (e.g., Fig. 1). A SNRof 1 is unrealistically high for many cryo-ET applica-tions, but the effect of this parameter is considered fur-ther below. Tomograms were calculated from thesedata and their resolutions measured by all three criteria.

This experiment is idealized in the sense that realtomograms are also compromised by other factors,including imperfect alignment of the projections, varia-tions in defocus, phase contrast effects, radiation dam-age, and the modulation transfer function of thecamera. Nevertheless, it allows one to calibrate thenew measures relative to a reference FSC. Because we

Fig. 2. Model experiment testing the resolution measures NLOO-3Dand FSCe/o against FSCref with a simulated tomogram of the HBVcapsid. The tilts ranged from �68� to +68�. The angular steps were:(A) 1�; (B) 2�; and (C) 4�.


introduced some noise in our simulation, the FSC be-tween the generated tomograms and the referencemap, which is essentially noise-free, does not satisfyEq. (2). A reference measure FSCref that is consistentwith our definition of the ideal experiment (see Section2.2), and hence satisfies Eq. (2), is given by

FSCrefðkÞ ¼ FSC2ðkÞ; ð9Þwhere FSC is directly calculated between the tomogramand the reference map (see the Appendix of Stewart andGrigorieff, 2004).

The results are shown in Fig. 2 where, for each angu-lar increment, FSCref, NLOO-3D, and FSCe/o are com-pared. With an angular step of 1� (Fig. 2A), the threemeasures provide almost coincident curves. The resolu-tion is given as 11.7 A in each case. With an angular stepof 2 (Fig. 2B), there is still good agreement betweenNLOO-3D (14.2 A) and FSCref (13.9 A), while FSCe/o

falls off at higher frequencies, giving a resolution of14.7 A. Finally, with an angular step of 4�, (Fig. 2C),NLOO-3D and FSCe/o both underestimate the resolu-tion as given by FSCref, more severely so for FSCe/o.The corresponding figures are 22.6 A (FSCe/o), 17.5 A(NLOO-3D), and 16.2 A (FSCref).

We interpret these results as follows. At 1� tilt incre-ment, the major factor limiting the resolution of the sim-ulated tomogram is residual noise. Both NLOO-3D andFSCe/o give values close to FSCref, the highest achiev-able resolution, and sampling-limited resolution is nota factor (rh � 5.4 A). At 2� and 4�, NLOO-3D remainsclose to FSCref, although there is more noise in thetomograms since they are based on fewer projections:in contrast, the FSCe/o estimate of resolution is system-atically too low as sparse angular sampling becomesimportant (r# � 21.6 A at 4�). According to the aposteriori criterion (6), the resolution given by FSCe/o

is reliable only for the 1� tilt increment. Thus the expect-ed improvement of NLOO-3D over FSCe/o under theseconditions (see Section 2.4) is realized.

We performed variations of this experiment with dif-ferent SNRs but otherwise maintaining the same condi-tions (4� angular step)—Fig. 3. A consistent trend isobserved: NLOO-3D gives better results, i.e., ones thatare closer to FSCref, although the distinction betweenthe two measures is less pronounced at low values ofthe SNR, where noise becomes the dominant resolu-tion-limiting factor.

3.2. Results on a cryo-electron tomographic data set

We tested the new measures on experimental data fora field of isolated virions of herpes simplex virus type 1,HSV-1 (Grunewald et al., 2003b). The virion has anucleocapsid of �1250 A diameter, surrounded by ategument and envelope, giving a full diameter of�2200 A. The 150-A thick capsid shell is icosahedrally

symmetric but neither the DNA that it contains northe tegument or envelope share this symmetry. The cap-sid, which accounts for �20% of the virion mass (Lam-pert et al., 1969; Newcomb et al., 1989), has beenreconstructed by cryo-EM/SPA in numerous studies,at resolutions from �25 to 8.5 A (e.g., Cheng et al.,2002; Newcomb et al., 1993; Zhou et al., 1994, 2000).Fig. 4 shows the zero-tilt projection of one series collect-

Fig. 3. Resolution of the HBV capsid assessed for different signal-to-noise ratios (logarithmic scale) of the input projections. The recon-struction was obtained from 35 projection images, separated anangular increment of 4�. Resolution was determined with thresholdof 0.5.

Fig. 4. Zero-tilt image from a tilt series of isolated HSV particles bycryo-electron tomography. The image is oriented so that the tilt-axis isvertical. The white circle indicates the virion used to assess theresolution of a single particle. Resolution of each labeled capsid wasassessed separately by NLOO-3D, with the following results: (1) 68 A;(2) 67.5 A; (3) 69 A; and (4) 68 A. Bar, 1000 A.

Fig. 5. Resolution curves for FSCe/o and NLOO-3D as a function ofthe radial frequency for a cryo-tomogram of the HSV1 virion. (A) Fulltomogram; (B) single virion; (C) single capsid. For the single capsid,the FSCref is also plotted.


ed at 1.5� increments over the range �63� to +63�(Grunewald et al., 2003b). Projections and tomogramswere sampled at 7.4 A.

The resolution given by NLOO-3D for the entiretomogram (Fig. 5A) is 77 A, while FSCe/o gives a valueof 92 A. However, resolution is expected to vary in dif-ferent parts of the volume and, in particular, to be lowerin regions far from the tilt axis (see Section 3.1) or at the

periphery, where some features may not be present in allprojections. To assess resolution locally, we selected anintact virion near the center of the field (Fig. 4). The re-gion of interest was defined as a sphere of 1240 A radius.Its resolution according to NLOO-3D is �71 A, whereas

Fig. 6. Variation of resolution with tilt angle, as measured by NLOO-2D for a cryo-tomogram of HSV1 virions. Resolution at two differentcut-off thresholds is plotted. (A) Full tomogram; (B) single virion; and(C) single capsid.


FSCe/o gave 76 A. These resolutions are indeed higherthan the (average) resolutions obtained for the entiretomogram, and there is less discrepancy between thetwo measures—Fig. 5B. Taking the virion to be�2200 A thick, rh � 58 A.

Previously, these tomograms were assessed by com-paring capsids contained in them with a reference densi-ty map from SPA/cryo-EM (Grunewald et al., 2003b).In this study, we followed a similar procedure. Basically,we extracted the capsid from a virion by selecting aspherical shell between radii of 520 and 640 A, whichwe then aligned, translationally and orientationally,with the reference. We expected considerable discrepan-cy between the two capsid representations, since thedata were recorded at very different values of defocus(8 vs. 1–2 lm). For this reason and to maintain compa-rability with the earlier measurements (Grunewald et al.,2003b), we used a straightforward FSCref in this com-parison, i.e., without invoking Eq. (9).

The curves obtained by NLOO-3D and FSCe/o showsimilar behavior (Fig. 5C). Both give the resolution to bearound 68 A, slightly lower than the critical resolution,rcrit (65 A; see Eq. (6)), where rcrit was obtained usinga diameter D of 1280 A. These numbers are closely con-sistent with FSCref, which gave a value of 69 A,although we note that the latter curve shows systemati-cally lower correlation at low frequencies, reflecting theafore-mentioned disparity in defocus. Finally, resolutionanalysis for capsids in other virions in the same field (seeFig. 4) gave very similar results (67.5–69 A).

3.3. Using NLOO-2D to measure the variation of

resolution with tilt angle and to appraise a tilt series

The foregoing numbers represent averages over allviewing directions and resolution should be higher in-plane because of the missing wedge effect. The resultinganisotropy has been considered on theoretical grounds(e.g., Grunewald et al., 2003b; Unger et al., 1999): how-ever, in practice, resolution may fall off even faster athigh tilt-angles, on account of greater projected thick-ness (hence, multiple scattering) and focal gradients.NLOO-2D allows one to measure resolution empiricallyas a function of tilt angle. This dependence is plotted inFig. 6A for the whole cryo-tomogram containing HSV-1virions described above (results given in Fig. 5A). Thebest resolution, at low tilt angles, is 70–72 A, but fallsoff more rapidly at tilt-angles above 50� than geometri-cal considerations would anticipate (Grunewald et al.,2003b). Interestingly, there were several projections to-wards the end of the series, at 50�–55�, that were notice-ably poorer. In retrospect, this represented aninconsistency in focusing that was corrected again bythe time the tilt series reached 58�–60�. Such curves al-low one to check for radiation damage towards theend of a cryogenic tilt series (high positive tilt angles,

in this case). Thus, resolution is slightly but systemati-cally worse around +40� than it is around �40� whichprobably represents incipient radiation damage,although we would expect this effect to be more appar-ent in studies that extend to higher resolutions (Conwayet al., 1993).


We also used NLOO-2D to evaluate the resolutionfor a single virion near the center of the field (Fig. 6B,corresponding to results in Fig. 5B). Here the best reso-lution, at low tilt-angles, was �65 A, indicating that thefigure of 72 A for the whole tomogram was compro-mised by averaging in lower resolution data fromperipheral regions. The poorer resolution of the 50�–55� projections is again apparent.

Finally, we measured resolution by NLOO-2D for asingle capsid (Fig. 6C, corresponding to results givenin Fig. 5C). Resolution varies from �55 A for near-zerotilts to �100 A for the highest tilts (cf. 68 A for the over-all resolution from NLOO-3D). Thus, the best resolu-tion, in-plane and near the center of the field, is 55 A,and the deterioration in the perpendicular dimension ismore pronounced than previously estimated, whichwas calculated to be �90 A at 90� (Grunewald et al.,2003b).

4. Discussion

In this paper, we address the issue of resolution mea-surement in the context of electron tomography and de-rive two criteria, NLOO and FSCe/o, based on principlesof cross-validation. These criteria were evaluated inmodel experiments and with real cryo-ET data, wherebythe resolutions that they gave were compared with theresults of reference-FSC calculations. In the intuitivelyappealing reference-FSC approach, the calculated struc-ture (or a component thereof) is correlated with aknown reference structure. The new criteria observedclose consistency with FSCref.

4.1. Advantages and limitations of NLOO-3D and FSCe/o

In calculating FSCe/o, the projections are partitionedinto odd and even half-sets. The formula (5) used indefining this measure is quite well known in SPA (Gri-gorieff, 2000; Rosenthal and Henderson, 2003).NLOO-3D represents a further step in the same direc-tion. Here, the underlying idea is to estimate the qualityof a tomogram in terms of its ability to predict a missingprojection. In theory, this measure provides a goodapproximation to the ideal but unattainable resolutionof an FSC calculated between two tomograms derivedfrom full but independent, statistically equivalent, tiltseries of the same volume (Appendix B). A priori, weexpected NLOO-3D to give more accurate, and indeedhigher, resolutions than FSCe/o because more informa-tion goes into the tomograms that are evaluated. Suchturned out to be the case in the validation experiments(see Section 3), although the two measures are quite con-sistent if the resolution is lower than rcrit. In such a sit-uation, which reflects relatively high noise levels (fromany of several sources), the sparser sampling of Fourier

space in FSCe/o is of minimal importance. Finally,appraisal resolution over a tilt series of projections byNLOO-2D allows for a posteriori quality control ofthe data, e.g., screening out seriously deficient projec-tions, in addition to providing an assessment of the var-iation of resolution with viewing angle. This measurealso provides an estimate of the tomogram resolutionin the most favorable situation, which will usually bein-plane (near-zero tilt-angles).

4.2. Computational considerations in practice

FSCe/o is less demanding than NLOO-3D in terms ofcomputational cost, since it only requires the calculationof two tomograms. Accordingly, it may be economicalin practice to use FSCe/o first. If the resulting resolutionis lower than rcrit, it is likely that NLOO-3D will yield avery similar number, and little is to be gained by makingthe lengthier NLOO-3D calculation. Nevertheless, ashort-cut can be achieved in the NLOO-3D calculationby overall reducing the number of projections analyzedor by only reducing the number of tomograms to be gen-erated. In the first case, the estimate is restricted to com-parisons between every 2nd (3rd, 4th. . .) projection andthe corresponding reprojection from the tomogram cal-culated without that projection. In the second case a re-duced number of tomograms is generated by excludingfrom each of them multiple suitably spaced projection,instead of only one, and then all the reprojections corre-sponding to the excluded tilt angles are calculated. Inboth cases, we expect a little loss in accuracy whileobtaining a computational speed-up that is proportionalto the reduction factor.

NLOO-2D may be used to obtain information aboutthe anisotropy of resolution, which is an importantconsideration for thick specimens and likely to beunderestimated by simply geometrical considerations.

There are reasons (see above) to suppose that resolu-tion may vary somewhat within a tomographic volume,although the uniformity of resolutions that we obtainedfor different HSV capsids suggests that such variationsmay be encountered mainly around the edges (we notethat this result tends to validate the FSCref approach).However, if there should be a region that is particularlyimportant to the investigation in hand, it may be worth-while to calculate the resolution of this subvolume.ELECTRA is configured to effect calculations of allthese kinds.

4.3. Other measures of tomographic resolution

To our knowledge, only a few approaches to resolu-tion assessment have been proposed in electron tomog-raphy. A leave-one-out technique (not so named) wasone of several criteria tried by Taylor et al. (1997). Spe-cifically, the resolution achieved with a tomographic


reconstruction method suited for paracrystalline objectswas assessed by three criteria applied in the projectiondomain (FSC, DPR, and unweighted phase residuals).The reference image for each input projection was ob-tained from a tomogram calculated from the whole tiltseries, as well as from one calculated omitting the pro-jection in question (i.e., leave-one-out). Non-exclusiongreatly affects the outcome of this calculation (above,and unpublished results), as was reflected in the resultsobtained.

In this study, we have shown that an appropriatecombination of these quantities (Eqs. (7) and (8)) givesa measure that is more consistent with those used inSPA. More recently, application of the spectral signalto noise ratio (Unser et al., 1987) to tomographic recon-structions has been investigated in separate ways byPenczek (2002) and Unser et al. (2005). Penczek (2002)derived and tested a 3D formulation, but only for a par-ticular class of Fourier space interpolation-based recon-struction methods. Unser et al. (2005) have proposed aninteresting generalization of this approach but it has yetto be applied in practice.

4.4. Local variations in resolution

To complement measurements of the variation of res-olution with tilt-angle (see Section 3.3), we can also de-fine a measure describing the 3D distribution of NLOO,in the Fourier domain as well as in real space. In theFourier domain it would be similar to what has beenproposed for the SSNR (Penczek, 2002; Unser et al.,2005), and would be useful for assessing the distributionof the signal and to design anisotropic low-pass filtersoptimized for the tomogram. In our case, this measurewould be obtained by omitting the summation overR (k) in Eq. (8), then evaluating this modified expressionof NLOO-3D for each frequency voxel. An adequatemapping of the frequency 2D coordinates of the singleprojections to the three-dimensional Fourier domainwould be required, which could be derived via the Cen-tral Section Theorem. However, this kind of measure isexpected to give noisy results, because of the low num-ber of projections involved in the estimation, thusrequiring a proper regularization. As an alternative, vol-umetric variations of the resolution in the real space do-main may be assessed by calculating the resolutions ofsizeable subvolumes that, once combined, cover all thetomogram, or just selected parts of it (e.g., at the centeror the periphery of the field).

4.5. Density of angular sampling: implications

The resolution of a tomogram is ultimately limitedby the angular increment between projections. Interest-ingly, results obtained in the model experiment in Sec-tion 3.1 showed that tomographic resolution, as

assessed by NLOO-3D and FSCref, may exceed the lim-it, rh, given by Eq. (3). To explain this discrepancy, wenote that, as derived (Crowther et al., 1970), the rela-tionship was not as a strict equality, but was basedon an approximation—that the Bessel function Jn (x)should be �0 for 0 < x < n � 2. This considerationplaced a limit on the highest order, N, of Bessel func-tion that could be solved at a given radial frequencylimit for a particle of specified diameter. In fact, higherorder Bessel functions have small but non-zero valuesthat intrude into this domain. We infer that their con-tributions, in effect, underlie the detection of signal tothe observed resolution limits. We note that degree ofconsistency between rh and other measures should alsodepend, to some extent, on the threshold adopted forthe latter measures. Notwithstanding the observed dis-crepancy, this relationship Eq. (2) provides a usefulrule-of-thumb for resolution as constrained by the sizeof the angular increment.

4.6. Application to dual-tilt series-based tomograms

Currently, there is growing interest in the use of dual-axis tilt series (Penczek et al., 1995) to reduce the missingwedge to a missing pyramid. In this context, it is desir-able to have a method for quantifying the improvementin resolution that is achieved. The final tomogram isproduced by calculating and then merging two tomo-grams from the separate tilt series. Before merging, thetwo tomograms are processed by applying a transforma-tion to align them and to compensate for potential radi-ation-induced distortions of the volumes between therecording of the two tilt series (Mastronarde, 1997).Here, the NLOO approach is readily applicable, provid-ed that the experimental projections are compared withreprojections in the same frame of reference, i.e., with-out any relative transformation applied. In principle,the two sub-tomograms could be compared in an FSCcalculation although this approach is likely to suggesta lower resolution than would be obtained with a projec-tion partition that follows the order of recording be-cause, particularly with cryo-specimens, the second tiltseries depicts a specimen that has been subjected to moreradiation.

4.7. Resolution in tomograms of plastic sections

Although progress is being made with vitrified sec-tions (Al-Amoudi et al., 2004; Hsieh et al., 2002; Zhanget al., 2004), it is likely that most cellular tomographywill continue, for some time to come, to be done withplastic sections of freeze-substituted or conventionallyembedded material. Such tomograms may also be eval-uated by the present criteria. In preliminary estimatesfor tomograms of a �110 nm plastic section of a virus-infected cell (G.C. et al., work in progress), NLOO-3D


gave an overall resolution of 55 A, whereas analysis byNLOO-2D gave 40 A in-plane, increasing to 140 A at60�. With respect to the notably high in-plane resolu-tion, we note that this figure refers to internal consisten-cy of the representation given by the stained plasticsection. If it were possible to calculate a FSCref betweensuch a tomogram and the corresponding native hydrat-ed structure, it would surely yield a considerably lowerresolution. The notably low resolution at 60� representsanother example of its falling off faster than geometricalconsiderations would predict.

Acknowledgments

We thank David Belnap and Bernard Heymann forfruitful discussions, James Conway for useful sugges-tions and making available the HBV density map, BenesTrus for making available the HSV1 capsid densitymap, and Eric Freed, Vlad Speransky, Dennis Winklerfor collaboration on the tomography project on HIVbudding.

Appendix A. Derivation of the NLOO-2D formula

A.1. Relationship between Fourier ring correlation and

normalized square difference

Given two images F(i) and G(i) in the Fourier domain,we define the normalized square difference (NSD) for aregion R (k) centered around radial frequency k, as

NSDðiÞFGðkÞ ¼

Pm;n2RðkÞ F

ðiÞm;n � GðiÞ

m;n

�� 2Pm;n2RðkÞ F

ðiÞm;n

�� 2 þPm;n2RðkÞ G

ðiÞm;n

�� 2 ; ðA:1Þ

where the frequency components of the images are des-ignated by the index pair (m,n).

The NSD can assume values between 0 and 2, whichis evident after manipulating the expression (A.1), andrewriting it as

NSDðiÞFGðkÞ ¼ 1�

2P

m;n2RðkÞRe F ðiÞm;nG

ðiÞ�m;n

n oP


�� 2 þPm;n2RðkÞ G

ðiÞm;n

�� 2 .ðA:2Þ

If we normalize the images in the region R (k), to obtainF

ðiÞand G

ðiÞsuch that

FðiÞm;n ¼ F ðiÞ

m;n

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiXm;n2RðkÞ

F ðiÞm;n

�� 2r,

and

GðiÞm;n ¼ GðiÞ

m;n

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiXm;n2RðkÞ

GðiÞm;n

�� 2r,

; ðA:3Þ

the NSD between these two quantities is

NSDðiÞF GðkÞ¼1�

2P

m;n2RðkÞReF ðiÞm;nffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiP

m;n2RðkÞF ðiÞm;nj j2

q GðiÞ�m;nffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiP

m;n2RðkÞGðiÞm;nj j2

q8<:

9=;P

m;n2RðkÞF ðiÞm;nj j2ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiP

m;n2RðkÞF ðiÞm;nj j2

q 2þP

m;n2RðkÞGðiÞm;nj j2ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiP

m;n2RðkÞGðiÞm;nj j2

q 2

.

Since both terms in the denominator are equal to 1,NSD and FRC are related by

NSDðiÞF GðkÞ ¼ 1�

Pm;n2RðkÞRe F ðiÞ

m;nGðiÞ�m;n

n offiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiP


�� 2r ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiP

m;n2RðkÞ GðiÞm;n

�� 2r

¼ 1� FRCðiÞFGðkÞ. ðA:4Þ

A.2. Derivation of the noise-compensated leave-one-outfrom the normalized square difference formulation

The NSD Eq. (A.1) between the input projection X(i)

and the reprojection ~X�ðiÞ

is affected by the differentnoise statistics of the two images. Furthermore, the in-put projection may contain some additional signal thatis unwanted, since it is not present in the reprojection.We assume that a proper noise term N(i) can take intoaccount these two effects.

We define the noise-compensated normalized squaredifference (NNSD) as

NNSDðiÞX ðiÞ ~X

�ðiÞ ðkÞ

¼P

m;n2RðkÞ XðiÞm;n� ~X

�ðiÞm;n

�� 2�Pm;n2RðkÞ N

ðiÞm;n

�� 2Pm;n2RðkÞ X

ðiÞm;n

�� 2þPm;n2RðkÞ

~X�ðiÞm;n

�� 2�Pm;n2RðkÞ N

ðiÞm;n

�� 2 ;ðA:5Þ

where the noise term is introduced both at numeratorand denominator. The object of this modified formu-lation is to provide a new normalized measure byremoving from the squared difference any variancedue either to the differing noise statistics or to addi-tional signals that may be present in the input projec-tions. We then approximate the frequency componentsof the noise as jN ðiÞ

m;nj ffi jX ðiÞm;n � ~X

ðiÞm;nj. Assuming

that the discrepancy between an input projection andthe corresponding reprojection from the tomogramgenerated from all the input projections is always lessthan the discrepancy between the same input projec-tion and the reprojection from a tomogram recon-structed without that projection, this choice for N ðiÞ

m;n

guarantees that the NNSD can not become negative.It follows that


NNSDðiÞX ðiÞ ~X

�ðiÞ ðkÞ

¼1�2P

m;n2RðkÞRe X ðiÞm;n

~X�ðiÞ�

m;n

n oP

m;n2RðkÞ~X�ðiÞm;n

�� 2�Pm;n2RðkÞ

~XðiÞm;n

�� 2þ2P

m;n2RðkÞRe X ðiÞm;n

~XðiÞ�

m;n

n o.ðA:6Þ

The evaluation of the NNSD on the projections/repro-jections, after they are normalized in the region R (k),gives

NNSDðiÞ

XðiÞ ~X

�ðiÞ ðkÞ ¼ 1�

Pm;n2RðkÞ

Re X ðiÞm;n

~X�ðiÞ�m;n

� �P

m;n2RðkÞX ðiÞm;nj j2

Pm;n2RðkÞ

~X�ðiÞm;n

�� 2n o1=2

Pm;n2RðkÞ

Re X ðiÞm;n

~XðiÞ�m;n

� �P

m;n2RðkÞX ðiÞm;nj j2

Pm;n2RðkÞ

~XðiÞm;n

�� 2n o1=2

¼ 1�FRCðiÞ

X ~X�ðkÞ

FRCðiÞX ~X

ðkÞ. ðA:7Þ

By analogy with (A.4), the ratio between the two Fouri-er ring correlations in (A.7) represents a correlationmeasure, which we define as 2D noise-compensatedleave-one-out resolution estimate.

Appendix B. An approximated analysis of the NLOO-2D

formula

In the following we demonstrate how the NLOO-2Dapproximates the ideal experiment (see Section 2.2) inassessing the resolution. Precisely we show that theNLOO-2D gives a relationship near to the equivalent ofEq. (2) for the FRC. This result can then be extended tothe NLOO-3D by exploiting the Fourier Slice Theorem.

We consider the case when the reconstruction is per-formed by nearest-neighbor interpolation in the Fourierdomain, an approach already followed in (Penczek,2002) with regarding to the SSNR. Furthermore, inthe analysis we do not consider the presence in the inputprojections of signal components that can instead bemissing in the corresponding reprojections.

We assume that each input projection X(i) is corrupt-ed by additive random noise, uncorrelated between theprojections, such that each frequency component is

X ðiÞm;n ¼ Sm;n þ N ðiÞ

m;n; ðB:1Þ

where Sm, n is the signal to reconstruct, while N ðiÞm;n is a

zero-mean Gaussian noise component. The signal Sm, n

describes a component in a voxel of the three-dimen-sional Fourier space that is uniquely defined by thevalues of i, m, and n. Since we are focused only onthe signal in a defined slice, in the following we omitthe index i.

Assuming that the noise is independent both betweenfrequencies and projections, we have

E N ðiÞm;nN

ðjÞ�p;q

h i¼ dm;pdn;qdi;jr

2m;n; ðB:2Þ

where E[ ] indicates the expectation value, dx, y is theKronecker delta, and r2

m;n is the variance of the noisecomponent, which is assumed to be the same for eachprojection.

Each reprojection ~XðiÞ

from the reconstructed volumesimilarly represents, in the Fourier domain, a slice fromthe reconstructed object. When the reconstruction isperformed by nearest-neighbor interpolation, each com-ponent ~X

ðiÞm;n is obtained as an average from nm, n adjacent

projections

~XðiÞm;n ¼

1

nm;n

Xnm;nj¼1

X ðjÞm;n. ðB:3Þ

In this formulation the index j in the sum does not indi-cate the position of the input projection in the tilt series,but refers to an arbitrary sequence which includes onlythe input projections that contain the signal Sm, n to bereconstructed. Moreover, we assume that in the new se-quence the index i corresponds to i 0. We remark that thenumber nm, n of projections involved is different for eachspatial frequency, and it typically decreases with higherfrequency values.

Under these assumptions, from (B.1) and (B.3) it fol-lows that the signal-to-noise ratio of the i-th slice of thereconstruction, measured for a radial frequency k, is

SNRðiÞðkÞ ¼P

m;n2RðkÞjSm;nj2Pm;n2RðkÞ

r2m;nnm;n

. ðB:4Þ

Moreover, the frequency components of the reprojec-tion ~X

�ðiÞ, since obtained by excluding the correspond-

ing input projection from the reconstruction process,are given by

~X�ðiÞm;n ¼ 1

nm;n � 1

Xnm;nj¼1;j 6¼i0

X ðjÞm;n. ðB:5Þ

Under these conditions, we can then evaluate the expec-tation value of the NLOO-2D as

E½NLOO-2DðiÞðkÞ� ffiE FRCðiÞ

X ~X�ðkÞ

h iE FRCðiÞ

X ~XðkÞ

h i . ðB:6Þ

By making use of (1), (B.1), (B.2), and (B.5), the expec-tation value of the FRC at numerator is

E FRCðiÞX ~X

� ðkÞh i

ffiP

m;n2RðkÞjSm;nj2ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiPm;n2RðkÞjSm;nj2þ

Pm;n2RðkÞr

2m;n

q ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiPm;n2RðkÞjSm;nj2þ

Pm;n2RðkÞ

r2m;nnm;n�1

q .

ðB:7Þ

The expectation value of the FRC at the denominator isobtained in a similar way, just using (B.3) instead of (B.5)


E FRCðiÞX ~X

ðkÞh i

ffiP

m;n2RðkÞjSm;nj2þP

m;n2RðkÞr2m;nnm;nffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiP


m;n2RðkÞr2m;n

q ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiPm;n2RðkÞjSm;nj2þ

Pm;n2RðkÞ

r2m;nnm;n

q .

ðB:8Þ

Substitution of (B.7) and (B.8) in (B.6) gives

E NLOO-2DðiÞðkÞ� �

ffiP

m;n2RðkÞjSm;nj2ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiPm;n2RðkÞjSm;nj2þ

Pm;n2RðkÞ

r2m;nnm;n

q ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiPm;n2RðkÞjSm;nj2þ

Pm;n2RðkÞ

r2m;nnm;n�1

q ;

ðB:9Þ

which, according to (B.4), can also be written as

E NLOO-2DðiÞðkÞ� �

ffi SNRðiÞðkÞSNRðiÞðkÞþ1

Pm;n2RðkÞjSm;nj2þ

Pm;n2RðkÞ

r2m;nnm;nP


m;n2RðkÞr2m;n

nm;n�1

0@

1A

1=2

.

ðB:10Þ

We observe that the first ratio term on the right side of(B.10) gives the required relationship between correla-tion measure and signal-to-noise ratio. The second ratioterm under square root represents the underestimationfactor, which is generally close to one.

References

Al-Amoudi, A., Norlen, L.P., Dubochet, J., 2004. Cryo-electronmicroscopy of vitreous sections of native biological cells andtissues. J. Struct. Biol. 148, 131–135.

Bracewell, R.N., Riddle, A.C., 1967. Inversion of fan-beam scans inradio astronomy. Astrophys. J. 150, 427–434.

Brunger, A.T., 1992. Free R value: a novel statistical quantity forassessing the accuracy of crystal structures. Nature 355, 472–475.

Cheng, N., Trus, B.L., Belnap, D.M., Newcomb, W.W., Brown, J.C.,Steven, A.C., 2002. Handedness of the herpes simplex virus capsidand procapsid. J. Virol. 76, 7855–7859.

Conway, J.F., Trus, B.L., Booy, F.P., Newcomb, W.W., Brown, J.C.,Steven, A.C., 1993. The effects of radiation damage on the structureof frozen hydrated HSV-1 capsids. J. Struct. Biol. 111, 222–233.

Conway, J.F., Cheng, N., Zlotnick, A., Wingfield, P.T., Stahl, S.J.,Steven, A.C., 1997. Visualization of a 4-helix bundle in the hepatitisB virus capsid by cryo-electron microscopy. Nature 386, 91–94.

Crowther, R.A., DeRosier, D.J., Klug, A., 1970. The reconstruction ofa three-dimensional structure from projections and its applicationto electron microscopy. Proc. R. Soc. Lond. A 317, 319–340.

Efron, B., 1979. Computers and the theory of statistics: thinking theunthinkable. SIAM Rev. 21, 460–480.

Frank, J., Al-Ali, L., 1975. Signal-to-noise ratio of electron micro-graphs obtained by cross correlation. Nature 256, 376–379.

Frank, J., Verschoor, A., Boublik, M., 1981. Computer averaging ofelectron micrographs of 40S ribosomal subunits. Science 214,1353–1355.

Frank, J. (Ed.), 1992. Electron Tomography: Three-DimensionalImaging with the Transmission Electron Microscope. PlenumPress, New York.

Grigorieff, N., 2000. Resolution measurement in structures derivedfrom single particles. Acta Crystallogr. D Biol. Crystallogr. 56,1270–1277.

Grunewald, K., Medalia, O., Gross, A., Steven, A.C., Baumeister, W.,2003a. Prospects of electron cryotomography to visualize macro-molecular complexes inside cellular compartments: implications ofcrowding. Biophys. Chem. 100, 577–591.

Grunewald, K., Desai, P., Winkler, D.C., Heymann, J.B., Belnap,D.M., Baumeister, W., Steven, A.C., 2003b. Three-dimensionalstructure of herpes simplex virus from cryo-electron tomography.Science 302, 1396–1398.

Harauz, G., van Heel, M., 1986. Exact filters for general geometrythree-dimensional reconstruction. Optik 73, 146–156.

Heymann, J.B., 2001. Bsoft: image and molecular processing inelectron microscopy. J. Struct. Biol. 133, 156–169.

Hsieh, C.E., Marko, M., Frank, J., Mannella, C.A., 2002. Electrontomographic analysis of frozen-hydrated tissue sections. J. Struct.Biol. 138, 63–73.

Kak, A.C., Slaney, M., 1988. Principles of Computerized Tomograph-ic Imaging. IEEE Press, New York.

Kremer, J.R., Mastronarde, D.N., McIntosh, J.R., 1996. Computervisualization of three-dimensional image data using IMOD. J.Struct. Biol. 116, 71–76.

Lampert, F., Bahr, G.F., Rabson, A.S., 1969. Herpes simplex virus:dry mass. Science 166, 1163–1165.

Ludtke, S.J., Jakana, J., Song, J.L., Chuang, D.T., Chiu, W., 2001. A11.5 A single particle reconstruction of GroEL using EMAN. J.Mol. Biol. 314, 253–262.

McEwen, B.F., Koster, A.J., 2002. Academic colloquium on electrontomography. Amsterdam, the Netherlands, October 17–20, 2001. J.Struct. Biol. 138, 1–155.

Mastronarde, D.N., 1997. Dual-axis tomography: an approach withalignment methods that preserve resolution. J. Struct. Biol. 120,343–352.

Medalia, O., Weber, I., Frangakis, A.S., Nicastro, D., Gerisch, G.,Baumeister, W., 2002. Macromolecular architecture in eukaryoticcells visualized by cryoelectron tomography. Science 298, 1209–1213.

Newcomb, W.W., Brown, J.C., Booy, F.P., Steven, A.C., 1989.Nucleocapsid mass and capsomer protein stoichiometry in equineherpesvirus 1: scanning transmission electron microscopic study. J.Virol. 63, 3777–3783.

Newcomb, W.W., Trus, B.L., Booy, F.P., Steven, A.C., Wall, J.S.,Brown, J.C., 1993. Structure of the herpes simplex virus capsid.Molecular composition of the pentons and the triplexes. J. Mol.Biol. 232, 499–511.

Nitsch, M., Walz, J., Typke, D., Klumpp, M., Essen, L.O., Baumei-ster, W., 1998. Group II chaperonin in an open conformationexamined by electron tomography. Nat. Struct. Biol. 5, 855–857.

Penczek, P., Marko, M., Buttle, K., Frank, J., 1995. Double-tiltelectron tomography. Ultramicroscopy 60, 393–410.

Penczek, P.A., 2002. Three-dimensional spectral signal-to-noise ratiofor a class of reconstruction algorithms. J. Struct. Biol. 138, 34–46.

Perkins, G.A., Renken, C.W., van der Kley, I.J., Ellisman, M.H.,Neupert, W., Frey, T.G., 2001. Electron tomography of mitochon-dria after the arrest of protein import associated with Tom19depletion. Eur. J. Cell Biol. 80, 139–150.

Rosenthal, P.B., Henderson, R., 2003. Optimal determination ofparticle orientation, absolute hand, and contrast loss in single-particle electron cryomicroscopy. J. Mol. Biol. 333, 721–745.

San Martin, C., Burnett, R.M., de Haas, F., Heinkel, R., Rutten, T.,Fuller, S.D., Butcher, S.J., Bamford, D.H., 2001. Combined EM/X-ray imaging yields a quasi-atomic model of the adenovirus-related bacteriophage PRD1 and shows key capsid and membraneinteractions. Structure 9, 917–930.

Saxton, W.O., Baumeister, W., 1982. The correlation averaging of aregularly arranged bacterial cell envelope protein. J. Microsc. 127(Pt 2), 127–138.

Shaikh, T.R., Hegerl, R., Frank, J., 2003. An approach to examiningmodel dependence in EM reconstructions using cross-validation. J.Struct. Biol. 142, 301–310.


Stewart, A., Grigorieff, N., 2004. Noise bias in the refinement ofstructures derived from single particles.Ultramicroscopy 102, 67–84.

Taylor, K.A., Tang, J., Cheng, Y., Winkler, H., 1997. The use ofelectron tomography for structural analysis of disordered proteinarrays. J. Struct. Biol. 120, 372–386.

Unger, V.M., Kumar, N.M., Gilula, N.B., Yeager, M., 1999. Three-dimensional structure of a recombinant gap junction membranechannel. Science 283, 1176–1180.

Unser, M., Trus, B.L., Steven, A.C., 1987. A new resolution criterionbased on spectral signal-to-noise ratios. Ultramicroscopy 23, 39–51.

Unser, M., Sorzano, C.O., Thevenaz, P., Jonic, S., El-Bez, C., DeCarlo, S., Conway, J.F., Trus, B.L., 2005. Spectral signal-to-noise

ratio and resolution assessment of 3D reconstructions. J. Struct.Biol. 149, 243–255.

Zhang, P., Bos, E., Heymann, J., Gnaegi, H., Kessel, M., Peters, P.J.,Subramaniam, S., 2004. Direct visualization of receptor arrays infrozen-hydrated sections and plunge-frozen specimens of E. coliengineered to overproduce the chemotaxis receptor Tsr. J. Microsc.216, 76–83.

Zhou, Z.H., Prasad, B.V., Jakana, J., Rixon, F.J., Chiu, W., 1994.Protein subunit structures in the herpes simplex virus A-capsiddetermined from 400 kV spot-scan electron cryomicroscopy. J.Mol. Biol. 242, 456–469.

Zhou, Z.H., Dougherty, M., Jakana, J., He, J., Rixon, F.J., Chiu, W.,2000. Seeing the herpesvirus capsid at 8.5 A. Science 288, 877–880.

A resolution criterion for electron tomography based on...

Documents

Transcript of A resolution criterion for electron tomography based on...