An Evaluation of Image Quality Metrics for Scanning ...An Evaluation of Image Quality Metrics for...
1
An Evaluation of Image Quality Metrics for Scanning Electron Microscopy Figure 2. The CTF curves for the images shown in Figure 1a-f. As the CTF curve is unaffected by contrast and brightness, the “Reference”, “Brightness”, and “Contrast” curves trace back directly on top of one another. However, this is not the case when the reference image is distorted by blurring (Fig. 1d), vibrations (Fig. 1e) and Gaussian noise (Fig. 1f). In these cases, the curves are altered by changing slope of the curves in the midrange of feature sizes or by changing the perceived noise floor or by a combination thereof. 1 College of Nanoscale Science and Engineering, SUNY Polytechnic Institute, Albany, NY, USA 2 Nanojehm Inc., Albany, NY, USA 3 The Institute of Optics, University of Rochester, Rochester, New York, USA Matthew D. Zotta 1,2 , Yukun Han 2 , Matthew D. Bergkoetter 2,3 , and Eric Lifshin 1 Abstract Introduction Equations and Methods Conclusions Results References It has been shown that through the use of image restoration techniques, the quality and resolution of scanning electron microscope (SEM) image can be significantly improved [1]. The purpose of this research is to determine a suitable image quality metric for quantifying the extent of that improvement. Several common image quality metrics were applied to SEM images subjected to various perturbations such as contrast and brightness adjustments, blurring, vibration and statistical noise. Typically, the assessment of image quality involves the comparison of a figure of merit for an observed image against that of a reference image where the observed image is a representation of the true structure obtained under less than ideal circumstances and the reference image is a representation the true structure obtained under ideal or nearly ideal conditions. The figure of merit criteria discussed here are the mean squared error (MSE) [2], peak signal to noise ratio (PSNR) [2], the contrast transfer function (CTF) [3] and the mean structural similarity index (MSSIM) [4]. In addition to these more established criterion, Nanojehm Inc. has developed a novel and unbiased way to determine resolution. = 1 ’’ , − - , . /01 234 501 634 ℎ: , is the true image - , is the perturbed image = 10 ∗ log 14 ( 5EF G H 5IJ ) ℎ: MAX() is the peak signal in the true image The MSE is frequently used in signal processing as a way to quantify the error in an output signal versus the input signal. This can be applied to images as well. The PSNR is used often in conjunction with the MSE to give an indication of the impact of the noise on the reliability of the signal. Nanojehm’s Method Nanojehm’s technology combines knowledge of the SEM point spread function (PSF) with image restoration techniques to restore images degraded by the PSF. Restoring the image involves both de-blurring and noise regularization. In order to quantify the image quality improvement, Nanojehm has developed a process for determining the achievable image resolution for a given set of SEM operating conditions. The procedure is as follows: 1) Acquire an image of a field of particles of a known size and distribution 2) Select a plurality of particles in the field and combine them to form a single stacked image 3) Take a line profile through this stacked particle image 4) Copy the line profile and displace it from itself until the dip between the profiles reaches some minimum 5) Take either the percentage of the dip between fixed center-to-center distances or the center-to-center distance between profiles displaced to a set dip percentage. [1] Lifshin, E. et al, Microscopy and Microanalysis 20 (01) (2014), p. 78-89. [2] Gonzalez, R. C. in “Digital Image Processing”, (Addison-Wesley, New York), p. 354. [3] Joy, D. C. et al, Proc. SPIE 3998, Metrology, Inspection and Process Control for Microlithography XIV (2000) p. 108. [4] Wang, Z, IEEE Transactions on Image Processing, 13 (04) (2004), p. 600- 612. [5] The authors wish to thank Mr. Jeffrey Moskin, President of Nanojehm and NSF SBIR 1519678 for supporting this research. 1) While MSE, PSNR and MSSIM are useful metrics when applied correctly, they are particularly susceptible to contrast and brightness settings in the images. When it comes to determining SEM resolution , these parameters are not as important as blurring, vibration and noise. 2) The CTF provides particularly useful information independent of brightness and contrast settings while at the same time showing relatively unique indications of blurring, vibration and noise. However one drawback is that interpreting the curves is not necessarily intuitive and therefore requires experience with various images. 3) Nanojehm’s technique offers an unbiased attempt to quantify resolution. This technique is particularly suitable for quantifying the resolution improvement achieved by image restoration. [5] Figure 1. (a) Ideal reference image of Au-C Pella sample. (b-f) Reference image perturbed by: (b) varying brightness, (c) varying contrast range, (d) convolution with asymmetric PSF, (e) random line shifts to do vibrations with a given amplitude, (f) Gaussian noise. b. MSE = 416, PSNR = 21.97, MSSIM = 0.917 c. MSE = 490, PSNR = 21.26, MSSIM = 0.868 d. MSE = 431, PSNR = 21.82, MSSIM = 0.815 e. MSE = 400, PSNR = 22.15, MSSIM = 0.683 f. MSE = 460, PSNR = 21.53, MSSIM = 0.220 600 nm a. MSE = 0, PSNR = Inf, MSSIM = 1 Figure 4. (a) Observed Stacked Particle, (b) Restored Stacked Particle and (c) Reference Particle Signal BSE, 20kV, FOV = 200 nm, Ip = 5.86 pA , 164 particles combined. Sample: 19nm gold spheres on a TEM grid. (a) (b) (c) Figure 3. Observed image (left) and Restored image(right). Signal BSE, 20kV, FOV = 1.3 microns, Ip = 5.86 pA, Sample: 19nm gold spheres on a TEM grid. 90% max value 36% max value 20 nm 31 nm Figure 5. Center-to-center distance for the observed and restored particles displaced until a dip equal to 10% of the max intensity is reached. (left) Since the particle size is know to be 19 nm, an indication of the resolution can be obtained by setting the center-to-center distance to 19 nm for both profiles and measure the dips of each (right) The SSIM and MSSIM are metrics that attempt to make a closer comparison of structural components contained in the image rather than simple point by point pixel intensity variations. This particular method was developed to assess image degradation associated with compression methods such as JPEG. A key feature is that it recognizes that two images could have the same MSE, but may have totally different responses to the human visual system (HVS). Specifically, this method involves calculating the luminance, contrast and structural factors independently, and then combining them into a single value for simplified evaluation. The closer the MSSIM is to unity, the more structurally similar the two images are. The CFT utilizes the Fourier transform of an image to give an indication of the special frequencies present. This is useful not only for determining the relative quantity of feature sizes present but also for giving an indication of the level of noise. Statistical noise, along with vibrations and blurring, have relatively distinct effects on the shape of the CTF curves. While learning to interpret these curves takes significant experience, they can be quite useful for resolution determination.
Transcript of An Evaluation of Image Quality Metrics for Scanning ...An Evaluation of Image Quality Metrics for...
AnEvaluationofImageQualityMetricsforScanningElectronMicroscopy
Figure 2. The CTF curves for the images shown in Figure 1a-f. As theCTF curve is unaffected by contrast and brightness, the “Reference”,“Brightness”, and “Contrast” curves trace back directly on top ofone another. However, this is not the case when the referenceimage is distorted by blurring (Fig. 1d), vibrations (Fig. 1e) andGaussian noise (Fig. 1f). In these cases, the curves are altered bychanging slope of the curves in the midrange of feature sizes or bychanging the perceived noise floor or by a combination thereof.
1CollegeofNanoscaleScienceandEngineering,SUNYPolytechnicInstitute,Albany,NY,USA2NanojehmInc.,Albany,NY,USA
3TheInstituteofOptics,UniversityofRochester,Rochester,NewYork,USA
MatthewD.Zotta1,2,Yukun Han2,MatthewD.Bergkoetter2,3,andEricLifshin1
Abstract
Introduction
EquationsandMethods
Conclusions
Results
References
It has been shown that through the use of image restoration techniques,the quality and resolution of scanning electron microscope (SEM) imagecan be significantly improved [1]. The purpose of this research is todetermine a suitable image quality metric for quantifying the extent ofthat improvement.
Several common image quality metrics were applied to SEM imagessubjected to various perturbations such as contrast and brightnessadjustments, blurring, vibration and statistical noise. Typically, theassessment of image quality involves the comparison of a figure of meritfor an observed image against that of a reference image where theobserved image is a representation of the true structure obtained underless than ideal circumstances and the reference image is arepresentation the true structure obtained under ideal or nearly idealconditions. The figure of merit criteria discussed here are the meansquared error (MSE) [2], peak signal to noise ratio (PSNR) [2], thecontrast transfer function (CTF) [3] and the mean structural similarityindex (MSSIM) [4]. In addition to these more established criterion,Nanojehm Inc. has developed a novel and unbiased way to determineresolution.
𝑀𝑆𝐸 =1𝑀𝑁 ' ' 𝑓 𝑥, 𝑦 − 𝑓- 𝑥, 𝑦
./01
234
501
634𝑊ℎ𝑒𝑟𝑒:𝑓 𝑥, 𝑦 isthetrueimage𝑓- 𝑥, 𝑦 istheperturbedimage
𝑃𝑆𝑁𝑅 = 10 ∗ log14(5EF G H
5IJ)
𝑊ℎ𝑒𝑟𝑒:MAX(𝑓) isthepeaksignalinthetrueimage
The MSE is frequently used in signal processing as a way to quantify theerror in an output signal versus the input signal. This can be applied toimages as well.
The PSNR is used often in conjunction with the MSE to give an indicationof the impact of the noise on the reliability of the signal.
Nanojehm’s MethodNanojehm’s technology combines knowledge of the SEM point spread function (PSF) with image restoration techniquesto restore images degraded by the PSF. Restoring the image involves both de-blurring and noise regularization. In order toquantify the image quality improvement, Nanojehm has developed a process for determining the achievable imageresolution for a given set of SEM operating conditions.The procedure is as follows:1) Acquire an image of a field of particles of a known size and distribution2) Select a plurality of particles in the field and combine them to form a single stacked image3) Take a line profile through this stacked particle image4) Copy the line profile and displace it from itself until the dip between the profiles reaches some minimum5) Take either the percentage of the dip between fixed center-to-center distances or the center-to-center distance
between profiles displaced to a set dip percentage.
[1]Lifshin,E.etal,MicroscopyandMicroanalysis20(01)(2014),p.78-89.[2]Gonzalez,R.C.in“DigitalImageProcessing”,(Addison-Wesley,NewYork),p.354.[3]Joy,D.C.etal,Proc.SPIE3998,Metrology,InspectionandProcessControlforMicrolithographyXIV(2000)p.108.[4]Wang,Z,IEEETransactionsonImageProcessing,13(04)(2004),p.600-612.[5]TheauthorswishtothankMr.JeffreyMoskin,PresidentofNanojehmandNSFSBIR1519678forsupportingthisresearch.
1) While MSE, PSNR and MSSIM are useful metrics when applied correctly, they are particularly susceptibleto contrast and brightness settings in the images. When it comes to determining SEM resolution , theseparameters are not as important as blurring, vibration and noise.
2) The CTF provides particularly useful information independent of brightness and contrast settings while atthe same time showing relatively unique indications of blurring, vibration and noise. However onedrawback is that interpreting the curves is not necessarily intuitive and therefore requires experience withvarious images.
3) Nanojehm’s technique offers an unbiased attempt to quantify resolution. This technique is particularlysuitable for quantifying the resolution improvement achieved by image restoration. [5]
Figure1. (a)IdealreferenceimageofAu-CPellasample.(b-f)Referenceimageperturbedby:(b)varyingbrightness,(c)varyingcontrastrange,(d)convolutionwithasymmetricPSF,(e)randomlineshiftstodovibrationswithagivenamplitude,(f)Gaussiannoise.
b.MSE=416,PSNR=21.97,MSSIM=0.917
c.MSE=490,PSNR=21.26,MSSIM=0.868
d.MSE=431,PSNR=21.82,MSSIM=0.815
e.MSE=400,PSNR=22.15,MSSIM=0.683
f.MSE=460,PSNR=21.53,MSSIM=0.220
600 nma.MSE=0,PSNR=Inf,MSSIM=1
Figure4. (a)ObservedStackedParticle,(b)RestoredStackedParticleand(c)ReferenceParticleSignalBSE,20kV,FOV=200nm,Ip =5.86pA ,164particlescombined.Sample:19nmgoldspheresonaTEMgrid.
(a) (b)
(c)
Figure3. Observedimage(left) andRestoredimage(right).SignalBSE,20kV,FOV=1.3microns,Ip =5.86pA,Sample:19nmgoldspheresonaTEMgrid.
90%maxvalue
36%maxvalue
20nm31nm
Figure5. Center-to-centerdistancefortheobservedandrestoredparticlesdisplaceduntiladipequalto10%ofthemaxintensityisreached.(left)Sincetheparticlesizeisknowtobe19nm,anindicationoftheresolutioncanbeobtainedbysettingthecenter-to-centerdistanceto19nmforbothprofilesandmeasurethedipsofeach(right)
The SSIM and MSSIM are metrics that attempt to make a closercomparison of structural components contained in the image ratherthan simple point by point pixel intensity variations. This particularmethod was developed to assess image degradation associated withcompression methods such as JPEG. A key feature is that it recognizesthat two images could have the same MSE, but may have totallydifferent responses to the human visual system (HVS). Specifically, thismethod involves calculating the luminance, contrast and structuralfactors independently, and then combining them into a single value forsimplified evaluation. The closer the MSSIM is to unity, the morestructurally similar the two images are.The CFT utilizes the Fourier transform of an image to give an indicationof the special frequencies present. This is useful not only fordetermining the relative quantity of feature sizes present but also forgiving an indication of the level of noise. Statistical noise, along withvibrations and blurring, have relatively distinct effects on the shape ofthe CTF curves. While learning to interpret these curves takes significantexperience, they can be quite useful for resolution determination.
