(Aufrictig e Wilson, 1995)

7/29/2019 (Aufrictig e Wilson, 1995)

1/14

IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 14, NO. 4, DECEMBER 1995 733

X-Ray Fluoroscopy Spatio-TemporalFiltering with Object DetectionRichard Aufrichtig and David L. Wilson, Member, IEEE

Abstract-One potential way to reduce patient and staff x-ray fluoroscopy dose is to reduce the quantum exposure tothe detector and compensate the additional noise with digitalfiltering. A new filtering method, spatio-temporal filtering withobject detection, is described that reduces noise while minimizingmotion and spatial blur. As compared to some clonventionalmotion-detection filtering schemes, this object-detection methodincorporates additional a priori knowledge of image content;i.e. much of the motion occurs in isolated long thin objects(catheters, guide wires, etc.). We create object-likelihood imagesand use these to control spatial and recursive temporal filter-ing such as to reduce blurring the objects of interest. We useautomatically computed receiver operating characteristic (ROC)curves to optimize the object-likelihoodenhancement nethod anddetermine that oriented matched filter kernels with 4 orientationsare appropriate. The matched filter kernels are simplle projectedcylinders. We demonstrate the method on several representativex-ray fluoroscopy sequences to which noise is added to simulatevery low dose acquisitions. With processing, we find that noisevariance is significantly reduced with slightly less noise reductionnear moving objects. We estimate an effective exposure reductiongreater than 80%.

I. INTRODUCTIONSING X-RAY fluoroscopy, cardiologists and radiologistsU erform interventional procedures such as balloon angio-

plasty, atherectomy, blood vessel embolization, transjugularintrahepatic portosystemic shunts (TIPS), biliary drainage,catheter ablation, etc. As the number, complexity, and durationof interventional procedures increases, x-ray fluoroscopydosebecomes an issue. Procedures lasting 4 hours with Iluoroscopytimes over 100min are documented [l], [2], and occasionallyneuro-embolization procedures last 6-8 hours. Staff dose inangioplasty can far exceed that of a diagnostic exam [3], andin coronary angioplasty studies, 74% of the dose to patientsis obtained from fluoroscopy [4]. Fluoroscopy is also used fordiagnosis in such studies as gastrointestinal exams, where aparticularly large dose is used [5], and voiding cystograms ofManuscript received July 28, 1994; revised August 22, 1905. This work

was supported by the E. L. Lindseth Chair of Biomedical Iingineering atCase Western Reserve University, the Whitaker Foundation, and NIH GrantR29-HL48918. R. Aufrichtig was partially supported by fellowships from theDanish Research Academy and the Knud Hojgaard Foundation. The AssociateEditor responsible for coordinating the review of this paper and recommendingits publication was S. M . Piezer.R. Aufrichtig is with the Department of Biomedical Engimeering, CaseWestern Reserve University, Cleveland, OH 44106 USA.D. L. Wilson is with the Department of Biomedical Engineering, CaseWestern Reserve University, and the Department of Radiology, UniversityHospitals of Cleveland, Cleveland, OH 44106 USA.IEEE Log Number 9415803.

pediatric patients where dose to young patients is an issue.X-ray fluoroscopy accounts for approximately 112 of the U.S.population dose due to radiological procedures [6].Fluoroscopy images are nearly quantum limited. Given thenormal exposure at the input to an image intensifier, the sizeof a pixel, and the acquisition rate, the average number ofphotons captured in a pixel is ~ 3 57], [SI. Poisson statisticsare obeyed, giving a standard deviation of 6 , or 17% of themean. Simply reducing exposure increases the percent noise tounacceptable levels, particularly because interventional workcreates a need for improved image quality.There are several techniques for reducing patient and staffdose in interventional fluoroscopy imaging. One is the x-rayfovea where a normal exposure is applied in the center of thex-ray field and a reduced exposure is used in the periphery[9]. An image processing algorithm that accounts for beamhardening compensates the image brightness in the periphery.Area exposure to patients is reduced by M 70%, while scatteredexposure to operators is reduced by M 60%. A similar deviceis described by Rudin and Bednarek [lo].A second method under study for reducing dose is pulsedfluoroscopy where short x-ray pulses are used and imagesare acquired at reduced acquisition rates [8], [Ill-[13]. Fromhuman perception studies, we find average exposure savingsof 22, 38, and 49% for pulsed fluoroscopy at 15, 10, and 7.5acq/sec, respectively [8], [141. Similar exposure savings areobtained for detection of moving objects [15]. Another studyshows that hand-eye coordination is not greatly affected at highvelocities and/or low acquisition rates when motion appearschoppy [16]. During the cardiac cycle, the acquisition rate canbe varied in order to match cardiac dynamics [17]. Modelingindicates that pulsed fluoroscopy exposure savings arise fromthe interaction between the acquisition-ratenoise characteristicand the spatio-temporal characteristic of the human visualsystem [18].A third approach, and the subject of the present report, isto reduce exposure and use image processing to reduce theincreased noise. Image processing solutions on some existingmedical products consist of time-domain filtering, time-domainfiltering with motion detection to reduce motion blur, andspatial filtering for contrast enhancement of objects. In thecase of motion-detection filtering, motion is detected whenevera pixel intensity changes more than a prescribed limit betweenthe current and the previous frames, and pixels are temporallyfiltered only where there is no motion. Unfortunately, imagenoise often results in false motion. With an obese patient, the

0;!78-0062/95$04.00 0 1995 IEEE


2/14

734 IEEETRANSACTIONS ON MEDICAL IMAGING, VOL. 14, NO. 4, DECEMBER 1995

Noisy ImageSequence

Object Object LikelihoodImageLEnhancement rLUT Adjustmentof Filter WeightsSpatialFiltering

TemporalFiltering

EnhancedSequenceFig. 1. This is a flow diagram of the algorithm.The input consists of a noisyfluoroscopy image sequence. Each image in the sequence is processed to createan object-enhanced mage consistingof large pixel values where there is a highlikelihood of an object being present. These object-likelihood mages serveas inputs to look-up-tables (LUTs) which control both spatial and temporalfilter weights. Filtering produces a noise-suppressed, enhanced sequence withminimal motion and spatial blur.

skin dose maximum is reached; the images are noisy becauseof the decreased number of x-ray quanta detected; motion isdetected; and no smoothing is performed.There are some publications on image enhancement hard-ware for x-ray image sequences. Edmonds et al. have devel-oped a real-time system for fluoroscopic filtering and appliedit to gastrointestinal studies [5]. They create an output imagefrom a weighted combination of spatially and temporallyfiltered input images. Temporal filtering is favored where nomotion is detected and spatial filtering is favored where thereis motion. Few system details are published. A product fromDigivision includes a Wallis-type spatial filter for contrastenhancement as well as a recursive temporal filter [19].Riederer et al. developed a matched filter-type algorithm foroptimizing the signal-to-noise ratio of the contrast signal in asequence of angiographic images [ZO]. Kruger and coworkersdeveloped a hardware implementation for recursive temporalfiltering and applied it to angiographic images [21], [22].Dobbins e t al. developed a technique for real-time spatialfiltering of angiographic images [23].An advanced method for noise reduction in image-sequences is motion-compensated temporal smoothing[24]-[26]. Motion compensation requires that theinstantaneous optic-flow field be explicitly computed priorto filtering. Frames are warped such that object movementis reduced or eliminated. Temporal smoothing of motion-compensated sequences by a low-pass filter, such as a meanor median filter, reduces the noise variance, and motioncompensation prevents motion blur. Optical flow estimationin sequences of coronary angiography images has beenreported for image interpolation between image framesand for image coding [27], [28]. Due to the noise level,large interframe motion, and, in some images, the relativesparseness of recognizable features, optical flow methodsmay not provide accurate results in a wide range of studies.Further, the calculation is computationally demanding andmay not be suitable for real-time implementation.

Statistically based, spatial averaging filters have been de-veloped for signal-dependent Poisson noise, the case forquantum-limited fluoroscopy images [29], [3O]. This workhas recently been extended to time domain filtering appliedt5 x-ray images by Chan, Katsaggelos, and others [31]-[33].To provide motion-compensated temporal filtering, motion isestimated between two consecutive frames using a block-matching algorithm 1331.We propose a new filtering paradigm: object-guided, spatio-temporal filtering. Particularly in noncardiac angiography,most motion OCCLII-sn isolated, long, thin objects (catheters,guide wires, etc.). We demonstrate that such structures canbe detected and roughly segmented using a matched filteringapproach. We then locally adjust spatial and temporal filteringparameters such as to reduce object blurring. In contrast tomotion-detection filtering using image differences, his schemedistinguishes between soft tissue and catheter motion. Notethat motion bluning of soft tissues is generally not a problem.

In this paper, we describe the algorithm and optimize filterpahameters using a receiver operating characteristic (ROC)curve analysis. We report numerical results from the process-ing method and describe several qualitative observations.

II. ALGORITJXMA flow diagram is shown in Fig. 1.We spatially process eachinput image to create an object-likelihood image consistingof large pixel values where there is a high likelihood of anobject being present. The object-likelihood mages are controlimages. They serve as inputs to look-up-tables (LUTs) whichon a pixel-by-pixel basis adapt spatial and temporal filterweights. In the case of temporal filtering, we apply a recursivetemporal filter with spatially-dependent filter parameters. The

method reduces filtering if an object has moved to orfrom apixel. Similarly, we apply a spatially-varying noise reductionfilter that again uses the object-likelihood image as a controlimage. Filtering processes are implemented in a conservativemanner such as to produce an enhanced sequence with reducednoise without blurring objects of interest.A . Object Enhancement

Features of interest such as catheters, guide wires, arter-ies, and interventional devices, are all line-like structures.We wish to create an object-likelihood, background-removedimage consisting of large pixel values where there is a high1ikeILhood of such an object. Currently, we investigate twospatial processing methods: matched filtering and unsharp-mask processing. The methods rely on a finite base of supportand are amenable to hardware implementation.

1) Matched$ltenng: We wish to maximize the output dueto the object (signal) relative to the background noise. Thematched filter is the linear filter that maximizes the ratio ofthe peak signal to average noise power [34], [35], and, forthe case of white, additive noise, the matched filter kernelis a transposed version of the desired 2-D object. Althoughx-ray fluoroscopy noise follows a signal-dependent Poissondistribution, one can assume additive Gaussian noise for lowcontrast objects. X-ray system fluoroscopy noise is not exactly


3/14

AUFRICHTIG AND WILSON. X-RAY FLUOROSCOPY SPATIO-TENIPORALFILERING WITH OBJECT DETECTION 735

white [36]. Also, tissue spatial structure is not white and iscommonly called lumpy in the image perception literature.Nevertheless, the matched filter is attractive, and we developan enhancement scheme based upon the concept.We desire a convolution kernel which is a transposed versionof the 2-D object of interest [37], [38]. We often know theshape and size of an object but not the orientation.A solution isto use oriented matched filters and take a maximum of severalfilter outputs [38], [39].Our purpose is to detect objects like arteries ancl catheters,and a model consists of the x-ray projection of a cylinder. Thedimensionless x-ray path-length, pZ, for parallel rays througha cylinder is

p l ( z ) = 2 , 4 7 f o r l z l s (d - 1)/2 (1)where z is the distance from the center axis, d is the odd-valued cylinder diameter, I is the geometric distance throughthe cylinder, and p is an effective linear attenuation coefficientfor a cylinder containing contrast material [40].Using (l), we create a 2-D matched filter kernel

q z ,9)= k o ( z ,Y) - m ( 2 )where

y=(V-1)/2~ x=(W-1)/2

(4)y=-(V- 1)/2

and V and W are the odd-valued length and width of thekernel, respectively. Several issues arise. First, the minussign in IC , occurs because catheters and arteries are negativeexcursions. Second, after processing, objects are marked bypositive excursions in the output likelihood image. Third, amatched filter is indifferent to a constant multiplicative term[34], and we ignore the factor 2p in (1). Fourth, the generaldefinition of the matched filter contains a flat base region oneither side of the projected cylinder for (d - 1)/2 < 1x1 5( W - 1)/2. We investigate the noise reduction effect of thebase. Fifth, since we desire a background subtracted result,we create a high-pass kernel by subtracting the mean value,m [38]. A subtracted-mean kernel has a zero-frequency (DC)gain of zero and is an approximation to the matched filter.One can also divide by local variances to remove backgroundat the expense of additional computations [371.The kernel specified by (2)-(4) detects vertical cylinders.We create kernels oriented at different rotationad angles, 6 ,

Catheters and contrast-containing blood vessels have high x-ray ab-sorbance, are darker than the background, and give negative pixel-valueexcursion. When CO 2 is used as a contrast agent, the gas displaces the bloodand reduces absorbance [41].

Convolution Kernels

Fig. 2. In the oriented, matched-filter method, the image is processed withfour, 2-D matched filters consisting of projections of a cylinder. The 2-D filterkernels oriented at 0, 45, 90, 135 are calculated from (2)-(5). After matchedfiltering, a maximum operation is applied at each pixel position to give thefinal object-likelihood image, L ( s , ).

using the transformation below.ICs(z,y) = IC(z cos 6 - y sin 8, J: sin 6 -t y cos 0 ) . (5)

Bilinear interpolation is used to obtain k ( z ,y) at real-valuedarguments. By sufficiently zero-padding k ( x ,y) prior to rota-tion, we insure that the correct size of ICe(z,y) is achieved.An input image is filtered with each oriented matchedfilter, and a set of intermediate output images is created.The maximum at each pixel position (x,y) creates the finalmatched-filter enhanced object-likelihood image Lm;ttch(z,y),

where I (x,y) is the input image, * symbolizes convolution,and N is the number of oriented kernels. In our experiments,we most often use N = 4. The filtering scheme is illustratedin Fig. 2. 2 ) Unsharp-maskJiltering: Objects are also enhanced witha large-kernel unsharp-mask filter. In uwharp-mask process-ing, a smoothed version of the input image is subtracted fromthe original in order to create an edge-enhanced, filtered image[37], [42]. We create the high-pass filterLunsharp(2,y) = U) - 1(z, ) h(x, ) (7)

where h ( z , y ) is the blurring kernel. We investigate threeblurring kernel structures: (1) a flat kernel, hdat(z,y), ( 2 )a Gaussian kernel, hgau(z, ) , and (3) a cone-shaped kernel,hCone(z,y). he three kernels are:

(8)1hflat(Z,Y) =-exp (-- 2 2 + y2) )NU M 1

v (9)rP4hgau(z, Y) = LNUM


4/14

136 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 14, NO. 4, ECEMBER 1995

Input image LUT Argument LUT Filter Kernel Output Image

Fig. 3. The object-guided, spatial filtering process is illustrated for the case of a 3 x 3 kernel. For each element of the 3 x 3 window in the input image(left), the LUT argument is the absolute difference between the center pixel and the surround. Filter kernel weights are obtained from the LUT specified by(13) and shown in Fig. 4.The output center pixel becomes a linear sum of the input pixels in the window weighted by the kernel values, the a's.

LNU Mwhere 1x1 5 (p- 1)/2 and IyI 5 (p-1)/2, p is an odd number,and CNUM s a normalization factor consisting of the sumover the p x p numerator values.B. Spatial Filtering

The adaptive spatial filtering method is illustrated in Fig. 3.Adaptive spatial processing is done with a (2n+l) x (2m+l)kernel where the kernel elements are defined by (11) shownat the bottom of the page.The a's are obtained from a look-up-table ( L W ,

where L(z,,y3) are pixel values from the object-likelihoodimage and ysp and p S p re parameters which define the spatial-filter LUT function. The method is arranged to not filter acrossobject boundaries, and the pixel weights, the a's,depend uponthe absolute difference between center and surround object-likelihood pixels. When the absolute difference is large, aboundary is present, and those pixels across the boundary arenot heavily weighted. The LUT function is a variation of alogistic sigmoid function1

where y pecifies a gradient and ,8 determines the horizontalshift. The spatial filtering LUT function is plotted in Fig. 4with ys p = -0.2 and f lSp = 25. In our experiments we use a3 x 3 (n = m = I ) kernel.

Fig. 4. The LUT functions used to control spatial and temporal filteringare plotted. The LUT function, LUT (z,y,P) is a variation of a logisticsigmoid function (13). The spatial filtering weights, CY,,~'s, are obtained fromthe steeply decreasing dashed curve on the left. Parameters are ysp = -0.2and ,BSp 25 . The temporal filter parameter, IC , is obtained from the solidcurve on the right. Parameters are ytmp = 0.01 and Ptmp= 500. This curveis further modified to lie between rc,in = 0.50 and tcmax = 0909 (15). Theparameters y and p, control the slope and horizontal shift, respectively. Theslope at the center of the curve is y/4, as indicated by the thin lines. Thehalf maximum point (LUT ( a ,y,p) = 0.50), occurs at 2 = P , as shown bythe dotted l ies .

C . Temporal F ilteringA spatially-dependent, recursive temporal filter is applied.

h he temporal filter, the estimate of a pixel intensity I t ( z , ) ,at coordinates ( L C , ~ )nd time t , s given byf t ( ~) = ft-1(2, 9)+ ~ [ l t ( z ,) - t - ~ ( z ,)] (14)

where It s the measured intensity at the current time instant,and K. is a filter gain which depends upon space and time.The filter is applied independently to each pixel, I t ( z , ) , inthe image. When li = 1, there is no filtering (it I t ) ;when


5/14

AUFRICHTIG AND WILSON X-RAY FLUOROSCOPY SPATIO-TEMPORALFILTERING WITH OBJECT DETECTION 131

K: = 0, the current frame is ignored ( I t = It-l); nd when0 < K: < 1, the filtered noise variance is d ( 2 -6) of the inputnoise variance [43].We control the amount of temporal filtering using the object-likelihood images. A control image is obtained by subtractingthe present image, L t, from the last one, Lt-l. The value forK: comes from a LUT operation applied at each pixlel location(x, )K: =LUT (I & - Lt-ll,Ytmp,Ptmp) . Kmax - Krnin) + Kmin.

(15)Above, y t m p and Ptmpare temporal filter, LUT parameters ina logistic sigmoid function, and and K: are limits ofK :. Since the absolute value JL t- Lt- ll is used, a large valueof the argument indicates the presence of an objiect in thecurrent or previous frames, but not both. In this case, reducedfiltering is desired, and we use an upper limit, Kmax ( K : ~ ~ ~1). When the LUT argument is small, either there is no objectpresent, or an object is present in both frames, and a smallvalue &,in(&,in 2 ), is desired. The LUT function is plottedin Fig. 4 for ytmp = 0.01 and Ptmp= 500.

111. METHODSA . Test Images

Using real x-ray fluoroscopy mages, we optimizt: the objectenhancement filters and evaluate the filtering methods.We acquire images from a digital fluoroscopy x-ray unit. Avery low-exposure acquisition is simulated by crealing signal-dependent, Poisson-distributed noise. We assume the numberof detected x-rays, N , o be Poisson distributed around a meanof NO, N Poisson{No},and a linear conversion between Nand gray-scale ( N = X x gray-scale, where X is a constant).The noise distribution of the gray-scale values is

where f(z,y) and g(z,y), are the gray-scale values in theoriginal and noisy image, respectively [29]. The noisy imageis generated by multiplying the original pixel values by X andusing this as input to a random number generator which returnsPoisson-distributed values. The output is scaled by 1 /X andtruncated to the 8-bit gray-scale digitization range:. With thisapproach, we maintain a constant mean gray-scale value atdifferent noise levels. The amount of noise depends upon A.We use 6 image sequences consisting of between 62 and 124frames. Images are 8 bit, 512 x 512 pixels, and are acquiredfrom a digital cardiac cath lab (Hicor, Siemeiis MedicalSystem, Iselin, NJ). The cardiac fluoroscopy images are froma variety of views and include interventional objects such asmultiple interventional guide catheters (7-1 1 F), a diagnosticpigtail catheter (7 F), a guide wire for a guide catheter (0.38inches), interventional guide wires (0.14 inches), ,and inflatedballoons.We create noise using X = 2.50, 0.75, andl 0.50, andinvestigate the simulated exposure levels. We recognize thatthe output noise distribution is not easily characterized because

TABLE ITHISTABLE ISTS HE FRACTIONF THE STANDARD EXPOSURE, F Q , im , FORTHE 6 TESTSEQUENCESN THE STUDY. HE TABLE IVESMEANS

AND STANDARD EVIATIONSALCULATEDVER40 FRAMESN EACHSEQUENCE.HREENOISEEVELS, X = 2.50, 0.75, AND 0.50, ARE U S E D .F Q , sim IS CALCULATEDSING 17), AND VARIANCESRE ESTIMATEDFROM RELATIVELY FLAT, BACKGROUND-SUBTRACTED00 x 100 PIXELREGIONS OF INTERESTS (ROIS) CONTAINING NO OBJECTS AND M OTION.VALUES INTHE BOTTOMOW AR E AVERAGEDVE R AL L EQUENCES

Fraction of simulated exposureover standard exposure,80,2.50 0.75 0.50sequence x1 0.17 f .05 0.04 f .02 0.04 f .022 0.14 f .03 0.05 f .01 0.03 f .013 0.15 f .03 0.05 f .01 0.03 f .014 0.16 f .04 0.05 f .02 0.04 f .025 0.13f .02 0.04 f .01 0.03 f .00

6 0.13 f .03 0.04 f .01 0.03 f .01average 0.15 0.05 0.03

the original input image is not noise free and because thenoise level depends upon gray-scale variations in the image.Hence, we elect to measure a change in effective exposure.Assuming inverse proportionality between image intensifierexposure, Q, and noise variance, u2 ,we calculate the fractionof the simulated exposure over the standard exposure in theoriginal sequence,

where the subscripts f and g refer to original and noisysequence, respectively. Within each frame of the sequence,variances are calculated from a 100 x 100-pixel region ofinterest (ROI) containing relatively smooth background tissueand no visible motion. Forty images from each sequence areaveraged to estimate the local mean, p f ( z ,y), and the resultis subtracted from each single frame. In this way, the effect oftissue-structure noise is minimized, and the variance, U; , canbe estimated from the background-subtracted rames.A similarcalculation is done for the noisy sequences to obtain 0:.Table I lists the fraction of the standard exposure for the 6test sequences used in the study. The x-ray unit is calibratedto 40 pR/sec (or 1.3 pR/acq at 30 acqkec) at the inputto the image intensifier. Using (17) and values from Table I,it follows that with X = 2.50, 0.75, and 0.50, we simulateexposure levels corresponding to x6, , and 1 pR/sec.B. Object Likelihood Processing

For the case of the matched filter, several filter parametersare optimized. They are: (1) the length of the projectedcylinder, V, 2) the diameter, d , and (3) the width of the kernel,W ,or the size of the base regions, (W - 12 . For the case ofthe unsharp-mask filters, we wish to determine the best shape(flat, Gaussian, or cone) as well as the optimal kernel size, p .Finally, the two methods are compared.A visual inspection of object likelihood images is notsatisfying because small variations in the output are difficultto detect. Instead, we use a quantitative measure that borrowsfrom receiver operating characteristic (ROC) curve analysis.


6/14

738 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 14, NO. 4, DECEMBER 1995

I ) ROC analysis: The object-likelihood process should en-hance objects of interest while not enhancing other structures.Assume that we know object and background pixels froma reliable, independent segmentation, a gold standard. Ifwe simply threshold an object-likelihood image, we createa segmentation where pixels above and below threshold aremarked as object and background, respectively. In general,pixels are correctly identified as object (true-positive), incor-rectly identified as object (false-positive), correctly identifiedas background (true-negative), or incorrectly identified asbackground (false-negative). We call these TP, TN, FP, andFN, respectively. A true-positive fraction (TPF) and a false-positive fraction (FPF) are computedTPF = number of pixels correctly determined as objectto tal number of object pixels

(181TPTP +-F Nnumber of pixels incorrectly determined as objecttotal number of background pixels

-FPF =

(1%FPT N + FP-By decreasing the threshold, one increases both TPF and FPF.Such analyses arise in psychophysical signal-detection ihe-ory. A plot of TPF versus FPF for various threshold levelsis called a receiver operating characteristic (ROC) curve[44]-[46]. ROC curve analysis is an accepted method forevaluating observer performance, and the technique has beenwidely applied in radiology to evaluate imaging modalities[46]-[48] and image processing techniques [49], [50].We ap-ply this methodology to analyze the object likelihood process.In this case, the analysis has nothing to do with humanobserver performance.

The more closely an ROC curve approaches the upper leftcorner ([FPF, TPF] = [0, l]), the better is the discriminationbetween object and background. Higher ROC curves are bettersince a greater TPF is recorded for each value of FPF. Astraight line with TPF = FPF indicates guessing, and we desirean ROC curve well above this. Different methods exist forevaluating ROC curves [46]-[48], and we adopt the frequentlyused, area under the ROC curve, A , . A chance performancegives A, = 0.5, while a higher value indicates improvedperformance.2) Calculationof ROC curves: To calculate the ROCcurve, one must identify object and background pixels. Thisis done by manually segmenting magnified images using animage processing and image analysis program (DIPStationTM;

Hayden Image Processing Group; Boulder, CO). The outputof this process is a binary image of the objects of interest.Following manual segmentation, we apply the object-likelihood enhancement methods to the original gray-scaleimages. Each object-likelihood image is thresholded at 256levels. At each threshold, we obtain the number of object andbackground pixels and calculate FPF and TPF. We excludecollimated areas. In human observer studies, an ROC curveoften consists of ~5 data points [45] and a continuous curve,often a binomial, is fitted to the data [45] in order to calculatethe area, A,. In the case of our computerized segmentation,

we compute FPF and TPF from a large number of samples andwith small increments in threshold. We get relatively smoothcurves and compute the integral numerically using trapezoidalintegration. For a variety of experimental conditions, the ROCcurve is plotted and A, is estimated.3) Experiments: We compute ROC curves for 15 differentpameter combinations of the matched filter kernel. Wechoose (d ,V,W ) = [(1,7,7); (1,9,15); (1,11,15); (3,7,7);

(3,9,9); (3,9,15); ( 3 J L W ; (3,13,15); (3,15,15); (5,7,7);(5,7,15); (5,11,13); (5,11,15); (5,13,15); (5,15,15)]. Likewise,we compute ROC curves for various unsharp-mask blurringkernels. We examine 3 shapes (flat, cone, and Gaussian), eachat 3 sizes (p = [7, 11, 151). For each image, at a given noiselevel we obtain 24 (15 from the matched filter and 9 fromu n s h q masking) ROC curves and A, values. Three imagesfrom different sequences are used, and noise is added withX = 0.75 and 0.50. The total number of tests is 216.C. Spatio-temporal Filtering and Evaluation

All image sequences are filtered with the LUT parametersfixed to the levels specified in Fig. 4. In addition, the temporalfilter gain, K , is constrained between &,in = 0.5 and &max =0.9.For spatial filtering only, the effective exposure reductioncan vary from 0 to 8 . Zero reduction occurs for the casewhere an object pixel is surrounded by eight backgroundpixels, or vice versa. In this case, arguments to the LUT arehigh, and no filtering is done. Maximum effective exposurereduction occurs when all nine pixels belong to the same state(background or object), and the arguments to the LUT aresmall. For the case of temporal filtering, exposure reductioncan vary from (1 - K ~ ~ J ( ~ - K ~ ~ J )o (1 -nmax/(2 - K ~ ~ ~ ) ) ,depending upon the arguments to the LUT.One measure of filter performance is the signal-to-noiseratio improvement calculated in dBs over each frame [31],Wl.

/

Above, ft s the original low-noise sequence, gt is the noisysequence, and f t is the filter output. An average improvement,I S N R ,s obtained by averaging values of I s N R ( ~ )fter therecursive filter has reached a steady state, typically 10 frames.If it is assumed that the original sequence, f t , is noise-free(o$ G O ) , I ~ N Refore taking dBs can be interpreted asthe ratio of the noise sequence variance, a i , to the filteredsequence variance, d .

One can find a new, effective x-ray exposure level that givesa noise reduction equivalent to filtering. We assume inverseproportionality between exposure and variance and computean effective exposure reduction, RQ, i l t , due to filtering

f t


7/14

AUFRICHTIG AN D WILSON: X-RAY ELUOROSCOPY SPATIO-TEMPORAL FILTERING WITH OBJECT DETECTION 139

Above, the cr2s are noise variances and the Q s are imageintensifier exposure values. This expression is evaluated intwo ways. First, we estimate backgrounds via temporal andspatial processing and create background-subtracted ROIs,as described in Section 1II.A. Second, we use I ~ N Reforetaking dBs (20) averaged spatially over an enitire imageor an ROI. A drawback of the second approach is that itassumes the noise variance of the original input sequence tobe zero. However, because subtracting the original sequenceremoves background, it can be used on ROIs that containmoving, as well as stationary, structures. The first methodcannot be used in regions with moving structures. We computean effective fraction of the standard exposure obtained afterfiltering, F ~ , f i l ~ .rom (17) and (21), we get

We calculate performance measures for all six image se-quences. To investigate effects separately, we evaluate theSNR improvement for spatial filtering only, tempoiral filteringonly, and combined spatio-temporal filtering.Filtering methods are compared to the 2-D Kuan spatialfilter [29]. The Kuan filter is a local, linear minimum meansquare error (LLMMSE) noise-smoothing filter which adaptsto local changes in image statistics based on a non-stationarymean and non-stationary variance Poisson-noise image model[29].This filter was previously applied to fluoroscopy images[ 3 2 ] , 33] and shown to perform better than the 2-1) Kak filter[51], which is optimized for Gaussian noise.A critical issue is the amount of image blur introduced byprocessing. Often a numerical value for noise ricduction isinsensitive to edge blurring. In preliminary experiments, weevaluated images visually and adjusted spatial and temporalLUT parameters so as to minimize motion and spatial blur. Onselected ROls containing structures of interest, we examinespatial blur in great detail. We compare the LUT-controlledspatial filter, an averaging filter, and the Kuan filter. Wevisually evaluate the ROls before and after filtering andinvestigate cross sectional profiles through a guide-wire.

IV. RESULTSA. Object-Likelihood Processing

Fig. 5 illustrates object-likelihood processing. Fig. 5(a) isthe original image while Fig. 5(b) has added noise with X =0.5. Fig. 5(c) and (d) are the matched filter outputs of Fig.5(a) and (b), respectively. Similarly, Fig. 5(e) and (f) arethe unsharp-mask outputs. The image contains a 10-F guidecatheter with a 0.014-inch guide wire, typical image structuresin cardiac interventional imaging. Improved performance isobtained with the matched filter, especially in the case of thenoisy frame.Fig. 6 shows ROC curves obtained from the images inFig. 5.Fig. 6(a) shows results from the original image, whileFig. 6(b) is for X = 0.5. The low-noise ROC curves [Fig.6(a)] are much higher than the high-noise curves [Fig. 6(b)].Considerable improvement in ROC curves is obtained as

Fig. 5. Object-likelihood enhancement processing is illustrated. The original512 x 512 image (a) and the noisy (A = 0.50) image (b) contain a 10-F guidecatheter with a 0,014-inch guide wire. Matched-filter output images are shownin (c) and (d), for the original and the noisy image, respectively. Similarly,(e) and (0 are unsharp-mask outputs. Matched filtering results in an improvedenhancement, especially for the noisy image. In the unsharp-mask processing,a 15 x 15 cone-shaped blurring kemel is used. Matched filter parameters are:15 x 15 kemel, 4 orientations, projected cylinder diameter of d = 5 pixels,and a length of V = 15 pixels.one optimizes the filter parameters shown in the figure. Theoptimum curve is obtained with (d ,V,W )= ( 5 , 15 , 15);A, is0.91 and 0.83 for the original and noisy frames, respectively.Areas, A,, under ROC curves are given in Tables I1and 111. Tables include results at different noise levels andimage frames from two other sequences. The matched filteris in all cases superior to unshaq-mask processing. Improvedperformance (larger A,) is achieved as the width and lengthof the mask increases. Saturation of A , occurs approximatelyfor V = 15 and W = 15. The best blurring kernel for theunsharp-mask processing is the 15x 15 cone shaped kernel.The worst kernel is the Gaussian, and virtually no enhancementis achieved with this filter, as the performance is close tochance (A , z 0.5). As noise increases (decreasing A), theperformance of unsharp-mask processing decreases drastically,while the matched filter is less sensitive to noise. Based onthese results, we chose the matched filter with (d ,V, W )= ( 5 ,15, 15) for the purpose of object enhancement.


8/14

740 El33 TRANSACTIONS ON MEDICAL IMAGING, VOL. 14, NO. 4, DECEMBER 1995

g: 15xl58alh: 3 x 7 conei: 15xl5gaussj: 7 x 7 g m

0.25 0.5 0.75False Positlve Fraction(a)

I f

g : 1 5 x l ! 5 t W th: 7 x 7 wneI: 15xl5gausoj: 7 x 7 gauss

"0 0.25 0.5 0.75 1False Positive Fraction(h)

Fig. 6. These are computed RO C curves after object-likelihood enhancementprocessing of the original image (a) and the noisy image (b) from Fig. 5.Matched filtering (solid lines) is always superior to uns hq -m as k processing(dashed lines). Parameters for the two enhancement methods are given in thefigure. For the case of the matched filter, parameters are listed as triplets,( d ,V, ) ' s , where the parameters represent cylinder diameter, cylinderlength, and kernel width, respectively. For the case of the unshaq-mask,the kemel size and type are listed. Best performance is obtained with the (5 ,15, 15) matched filter, and, in this case, A, = 0.91 and 0.83 for the originaland noisy image, respectively. The largest area under the ROC curve for theunshq-mask is obtained with a 15 x 15 cone-shaped kemel, which givesAZ = 0.79 and A , = 0.59 for the two images.

We can also examine the quality of object-likelihood pro-cessing by examining images (Fig. 7). In Fig. 7(a), a manual

Fig. 7. This figure also illustrates object-likelihood, enhancement processingof images from Fig. 5. The original, low-noise image is manually seg-mented (a) in order to identify true positives and true negatives. Withoutobject-likelihood enhancement, it is not possible to threshold the originalimage to create a reasonable binary representation of the catheters (b). Amuch better thresholding result is obtained after image enhancement with the(5, 15, 15) matched filter kernels (c and d). In c, thresholding gives a FPFof 255% and a TPF of ~ 7 5 %or the original, low-noise image. Similarly,the threshold level in (d) gives a FPF of %5 % and a TPF of ~ 6 0 %or thenoisy image.

TABLE 11INTEGRATEDREAS, , , UNDERR O C CURVESRE OBTAINEDOLLOWINGMATCHEDFILTERING.LER PARAMETERSRE:CYLINDER IAMETER,

THREEDIFFERENTMAGESRE SH O WN .OC CALCULATIONSREd , C Y L N E R LENGTH,, ND KEFNEL WIDTH, w.ESLLTSROMDONE FOR ORIGINAL, LOW-NOISEMAGES (bDICATED BY -) AND2 NOISE LEVELS(A = 0.75 AN D 0.50). h'ALL CASES, THEHIGHEST ALUEOF A, IS O B T A ~ DITH THE (5, 15, 15) K E ~ L

I Area under RO C curvtimage 2mage 1matched filterparametersd

11- v w7 79 15

1 11 153 7 73 9 93 9 153 11 153 13 153 15 155 7 75 7 155 11 135 11 155 13 155 15 15

0.83 0.67 0.640.72 0.58 0.560.78 0.64 0.620.84 0.72 0.710.84 0.73 0.710.85 0.74 0.720.85 0.74 0.730.74 0.59 0.570.84 0.74 0.720.84 0.73 0.720.85 0.75 0.740.85 0.76 0.76

x- 0.75 0.50

0.76 0.62 0.600.87 0.74 0.710.87 0.75 0.720.77 0.64 0.630.83 0.70 0.680.87 0.79 0.760.88 0.80 0.770.90 0.82 0.800.78 0.64 0.620.86 0.79 0.750.88 0.80 0.770.88 0.82 0.790.90 0.83 0.810.91 0.84 0.83

0.89 0.81 0.79

A , image 3x- 0.75 0.50

0.86 0.61 0.580.93 0.79 0.760.94 0.80 0.770.88 0.71 0.660.92 0.80 0.760.94 0.86 0.840.94 0.87 0.850.95 0.88 0.860.95 0.89 0.860.90 0.72 0.680.94 0.87 0.860.95 0.88 0.860.95 0.89 0.870.95 0.91 0.890.95 0.90 0.88

segmentation of the catheter and guide wire from Fig. 5(a) isshown. Without preprocessing it is no t possible to thresholdand get a reasonable binary representation [Fig. 7(b)]. Aftermatched filtering, we get a very good representation of the


9/14

AUFRICHTIG AND WILSON X-RAY FLUOROSCOPY SPATIO-TEMPORALFILTERING WITH OBJECT DETECTION

unsharp masktype P7hj,.t 11157h,,, 1115

7ha,, 1115

filter parameters

741

Area under RO C curve, A ,image 1 image 2 image 3x x x

- 0.75 0.50 - 0.75 0.50 - 0.75 0.500.63 0.54 0.53 0.65 0.55 0.54 0.77 0.59 0.580.71 0.57 0.56 0.72 0.58 0.57 0.84 0.64 0.620.77 0.60 0.58 0.77 0.60 0.59 0.87 0.67 0.650.60 0.53 0.52 0.63 0.54 0.53 0.73 0.56 0.550.61 0.53 0.52 0.64 0.54 0.53 0.75 0.57 0.560.61 0.53 0.52 0.64 0.54 0.54 0.75 0.57 0.560.64 0.54 0.53 0.66 0.56 0.55 0.78 0.60 0.580.73 0.58 0.56 0.74 0.58 0.57 0.86 0.65 0.630.79 0.61 0.59 0.79 0.61 0.59 0.88 0.68 0.66

Fig. 8.using (16) with X = 0.75.Three frames from a 60 frame noisy fluoroscopy sequence are shown. The frame numbers are: (a) 5 , (b) 30, and (c) 60. Noise is added

Fig. 9.represent a high likelihood of an object being present. These object-likelihood images are used to control the amount of spatial and temporal filtering.Object likelihood images from the 3 input frames in Fig. 8.The matched filter is used with parameters ( 5 , 15, 15). Large pixel values (white pixels)

catheter in the original image [Fig. 7(c)]. A good binary imageis obtained even in the case of added noise (A = 0.50) asshown in Fig. 7(d).

B. Spatio-Temporal FilteringWe illustrate the processing on three frames from a 60

frame sequence. Noisy input images are created with X = 0.75[Fig. 8(a)-(c)]. The same frames are shown following object-likelihood enhancement [Fig. 9(a)-(c)] and spatio-temporalnoise reduction [Fig. lO(a)-(c)]. Background noise is greatlyreduced without blurring catheters, which move significantlyduring the 60 frame sequence.

Using the image sequences from Figs. 8and 10, we plot theSN R improvement, I ~ N R ( ~ ) ,n a frame-by-frame basis (Fig.11). We examine the cases: 3 x 3 Kuan, 5 x 5 Kuan, object-guided 3 x 3 spatial filtering, object-guided temporal filtering,and combined spatio-temporal filtering. For this 60-framesequence, I s N R ( ~ )s lowest for the 3 x 3 Kuan filter. Nextcomes object-guided temporal filtering which is less effectivethan the object-guided spatial filter. The 5 x 5 Kuan and theobject-guided 3 x 3 spatial filter give similar improvements(257.3 and 7.9 dB, respectively). The highest I ~ N Ralue isachieved by combining object-guided, spatial and temporalprocessing; it is m9.7 dB.In Table IV we list means and standard deviations of theSN R improvement calculated for 6 image sequences at 3 noiselevels. The object-guided spatial filter performs better thanthe 3 x 3 Kuan filter and compares well to the 5 x 5 Kuan.In general, SNR improvement increases with increasing noiselevels (decreasing As). This is observed for all filter typesbut is most prominent for the spatio-temporal combination.Because the degree of temporal filtering varies with motionbetween successive frames, I s N R ( t ) fluctuates somewhat asa function of time. In the table, this is observed as higherstandard deviations for cases with temporal filtering.Tables V and VI give effective exposure savings, R ~ , f i l ~(21), obtained with the spatio-temporal filter. Values in Ta-ble V are calculated from I ~ N Rsing the entire image asdescribed in Section 1II.C. Values in Table VI are obtainedfrom 100x 100 pixel, background-subtracted ROIs (Section1II.A.). The average effective exposure savings in Table VI arehigh (293%) at all three noise levels, and there is a tendency


10/14

142

sequence X2.501 0.750.502 0.75

2.503 0.752.50

2.505 0.75

2.500.50

0.504 0.75

0.50

IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 14, NO. 4, DECEMBER 1995

SN R Improvement, ISNR (dB)spatial temp. spatia-temp. Kuan-3 Kuar-55.8i .1 3.0 f .5 6.1 i .6 3.1f .1 5. 1 zt 0.27.2f .1 3.7 f 0.2 8.9 f .4 3.2 f .3 7.3 f 0.27.5i .1 3.9 f .1 9.9i .4 3.2 i .3 7.6 i .27.3f .1 4.1f .1 9.7 5 0.4 3.6 * 0.1 7.9 f 0.24.7f .2 3.4 * 0.3 5.3f .4 3.0 i .1 5.0i .26.0 i .1 4 .1 i .1 8.5f .5 3.2f .2 7.2 zt 0.25.9f .1 3.1 f .4 6.2 =t0.5 3.4i .3 5.5 + 0.26.2 f .0 3.5 * 0.4 6.5 i .6 3.4 0.1 5.8f .18.1i .3 4.0 5 0.1 9.7 f .5 3.7 f .0 7.6i .1

6.2i .1 3.5f .6 6.3f .8 3.4 f .1 5.S f .37.5f .1 4.1 f .1 10.4 & 0.6 3.5 f .2 8.2 f .1

6.3 f .1 4.2 * 0.0 9.5 f .3 3.4 f .2 7.8 ?C 0.17.6f .2 3.7 f .2 9.3 f 0.3 3.6 f .2 7.0 f .18.0i0.1 4.2*0.1 1 0 . 0 f 0 . 3 3 .7 i O . l 7 .6 i 0 .1

Fig 10reduced as a result of the filtering process. There is no noticeable blur of the catheters, w h x h are in motion dunng the cardiac cycleOutputs of the object-gulded spatio-temporal filter are shown Respective input image frames are from Flg 8 Background noise 1s significantly

3.9 9.34.1 10.2 3.5 7.43.5 7.9

0.50 1 8 . 6 f .0 4.0 k .1 10.9 f .4 [ 3.7 + 0.0 8.0 f .02.50 16.0f .0 3.3* 0.3 6.4 f .6 13.3f .2 5.6 i .26 0.75 1 7.9 * 0.2 3.9i .1 9.4f .4 1 3. 6 f 0.1 7.4 i .20.50 [ 8.3 f .1 4.1f .1 10.6 f .3 13.7 f .1 7.9 f .12.50 I 5.8 3.3 6.1 1 3.3 5.5

toward higher savings with the most noisy sequences (A =0.50). Values calculated from the ROIs (Table VI) are alwayslarger than those obtained from the entire image (Table V) .This is expected since there are no objects and no noticeablemotion in the ROIs, and this results in maximal filtering. Thelowest value in Table V is obtained at the lowest noise level(A = 2.5). In this case, the assumption of negligible noise inthe original sequence may be invalid.

1 s ~ ~s also calculated over 100 x 100 pixel ROIs. In flatregions with relatively little motion, 1 s ~ ~s z dB higherthan obtained over the entire image. However, in a regioncontaining a moving object or lumpy background, I S N Rsdecreased by as much as 2 dB. In such regions, effectiveexposure savings decrease to ~ 8 0 %or X = O S and X = 0.75.As given in (22), the fraction of standard exposure afterfiltering, F Q , i l t , is the ratio between F Q , im and (1 -RQ, i l t ) .For 100x 100 pixel ROIs, average F Q , ~ ~ ~nd (1 - R ~ , f i l t )values are obtained from Tables I and VI, respectively. Using40 pR/sec as the standard exposure, we estimate effective

SNR mprovement121 I I I I I I

8

- patio-temporal- x 5 Kuan- patial-emporal- x 3 KuanFrame Time(1/30 ec)

Fig 11 The improvement in SNR, I ~ N R ( ~ ) ,or the sequence from Fig 8is plotted as a function of frame number Five filtering methods are appliedas indicated In the figure inset The highest SNR improvement is obtained bythe spatio-temporal filter combination

exposure levels after filtering to be ~ 2 6 ,4, and 78 pFUsecfor X = 0.50, 0.75, and 2.50, respectively.We have chosen LUT parameters that minimize spatialand temporal motion blur. Fig. 12 illustrates performance ofthe object-guided spatial filter near catheter edges. Fig. 12(a)shows part of an image from the original sequence and a cross-section. The two valleys in the cross-section clearly indicatethe presence of the guide catheter and guide wire. In Fig.12(b), we show the image and profile after noise addition withA = 2.50. Images and profiles in Fig. 12(c)-(f) are obtained


11/14

743UFRICHTIG AN D WILSON: X-RAY FLUOROSCOPY SPATIO-TEMPORAL FILTERING WITH OBJECT DETECTION

TABLE VSEQUENCES OBTAINEDITH THE SPATIO-TEMPORALILTER. ALUESRE

IMAGEUSING HE METHODEFINEDN (20) BEFOREAKEG dB's

THISTABLE IVES STIMATEDXPOSURE SAVINGS, RQ, ilt, FOR THE 6 IMAGECALCULATED USING (21). VARIANCES ARE CALCULATED FROM THE ENTIRE

Estimated exposure savingsfor entire image, RQ, i t tsequence x

2.50 0.75 0.501 0.75 f .04 0.87 f .01 0.90 f .012 0.77 f .05 0.89 f .01 0.91 f .013 0.70 f .03 0.86 f .02 0.89 f .014 0.76 f .03 0.88 f .01 0.90 f .015 0.78 f .03 0.89 f .01 0.92 f .016 0.77 f .03 0.89 f .01 0.91 f .01average 0.76 0.88 0.91

TABLE VITHISTABLEGIVESEFFECTIVEXPOSUREAVINGS, RQ f i l t ,FOR THE SPATIO-TEMPORALILXER VALUES RE CALCULATEDUSING (21) AND THE SAME 00 x 100 PIXEL ROI's USED OESTIMATE Q , ~ ~ ~OR THE NOISYEQUENCES IN TABLE

Estimated exposure savings for 100 x 10OFbackground subtracted ROI, RQ,2.50 0.75 0.50 -

sequence x1 0.92 f .01 0.95 f .02 0.95 f .002 0.92 f .02 0.94 f .01 0.94 f .013 0.94 f .00 0.95 f .00 0.95 f .004 0.92 f .01 0.94 f .01 0.95 f .015 0.93 f .02 0.95 f .01 0.95 f .006 0.93 f .02 0.95 f .01 0.95 f .01 -0.95 --verage 0.93 0.95

after filtering with the 3 x 3 object-guided spatial filter, a 3 x 3averager, a 3 x 3 Kuan, and a 5 x 5 Kuan, respectively. Fromthe figure, it is seen that the averager and the 5 x 5 Kuan resultin significant spatial blur; that is, their catheter profiles areshort and wide. Both the 3 x 3 object-guided spatial filter andthe 3 x 3 Kuan give minimal blur, but more noise reductionis obtained with the object-guided filter.

V. DISCUSSIONA. Algorithm Assessment

As compared to motion-detection filtering using imagedifferences [52], [ 5 3 ] , he present method incorporates morea priori knowledge. The object-likelihood images indicatethe presence of objects of interest-long, thin features suchas catheters, guide wires, and arteries. The power of themethod is that the likelihood images control both spatialand temporal filtering such as to reduce potential spatial andtemporal motion blur. The object-guided spatial filter performswell when compared to other adaptive methods. With regardto SN R improvement, the 3 x 3 object-guided spatial filterperforms better than the 3 x 3 Kuan, and similar to the 5 x 5Kuan (Fig. 11 and Table IV). However, blurring of edges issignificantly reduced with the 3 x 3 object-guided spatial filteras compared to the 5 x 5 Kuan (Fig. 12). Obje~ct-likelihoodimage differences are used in the temporal control function(14)-(15), and temporal filtering is reduced whenever objectsof interest movefrom or to a spatial location. A desirable effect

8060

40h 20 40 So 80 ,bo

40: 20 40 60 a0 ,do

40A 20 40 50 80 1kpixelFig. 12. A guide catheter and guide wire are cropped from an image andfiltered by various methods to produce the ima ges on the le ft. Cross-sectional,pixel -valu e profiles are extracted along the dashed line s and plotted on theright. (a) is from an original image, w hile (b) is no ise added at X = 2.50. (c )is obtained after filtering the noisy image with the 3 x 3 object-guided spatialfilter. (d) is processed with a 3 x 3 averager. A 3 x 3 and a 5 x 5 Kuan areused in (e) and (0, espectively. Most spatial blur occurs in (d) and (0.The3 x 3 objec t-guid ed spatial filter has minim al blur, and more noise reductionthan the 3 x 3 Kuan filter.

is that inappropriately enhanced, stationary features such asbone edges are temporally filtered.Object-likelihood enhancement is a critical step. Smallkernel, edge-enhancing filters such as a Sobel are inappropriatebecause they enhance edges, and not the entire object. Further,the performance of small-kernel operators is highly degradedin the presence of fluoroscopic noise levels. All currentlyevaluated methods take advantage of the spatial averagingcharacteristic of a large kernel. We find these methods superiorto a morphological filter investigated earlier [54]. This mor-phological filter, like others, tends to perform poorly in thepresence of large, Gaussian noise variances [55] , [56] . It iswell-known that a matched filter has a relatively broad spatialoutput, and we use this feature to create good, conservativecontrol images. As compared to simple image differences,the matched filter approach basically maximizes the signal-to-noise ratio in the control image.The advantage of unsharp-mask filters is computationalspeed. The flat kernel is fastest because it is separable andimplementable with running averages. The Gaussian kernel islikewise separable, but virtually no enhancement is obtained


12/14

144 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 14,NO. 4, DECEMBER 1995

with this shape and kernel orientation. The best performanceis obtained with the cone-shaped blurring kernel. This kernel,with a negative excursion, better matches the intensity profileof the artery, and gives improved enhancement following thesubtraction operation2We find ROC analysis to be a useful technique for filteroptimization, and it indicates significantly better enhancementwith the oriented matched filters (Tables H and HI). We usecylinders in four orientations. With more orientations (6 ,8,and

12), the TPF typically increases at the expense of an increasein the FPF, and there is no increase in the area under the ROCcurve, A, (not shown). This is obtained on images with noobvious object orientation and hence is not due to preferentialhorizontal and vertical orientation. With only two orientations,smaller A,s are obtained, and four seems to be a reasonablecompromise.The algorithm uses LUTs to control both spatial and tem-poral filtering. In earlier implementations, we thresholded thelikelihood image and used binary decisions to regulate spatialand temporal filtering [57]. In general, an LUT approachblends the amount of processing from one region to the next.Without such blending, one sometimes detects differences intexture. We adjust LUT parameters to create conservativefiltering. Variations in object-likelihood values are obtainedfrom one image sequence to the next due to differences inobject contrast, object size, etc. Currently, LUT parameters areadjusted so as to reduce filtering of even the most low-contrastobjects. We have even more confidence with high-contrastobjects, and they are filtered slightly less. In general, onecan adjust the filter parameters such as to create more noisereduction at the expense of possible spatio-temporal blumng.Interestingly, with the reported LUTs, we find that noisereduction from object-guided spatial filtering is greater thanfrom object-guided temporal filtering (Fig. 11 and Table IV).B. Image Sequence Evaluation and Exposure Savings

With regard to SN R improvement, our spatio-temporalmethod compares favorably to some previously reported re-sults. Since image sequences vary greatly, we compare resultsusing a reference method, the 5 x 5 Kuan. With X = 0.75,the object-guided 3 x 3 spatio-temporal filter performs 2 dBhigher than the 5 x 5 Kuan (Table IV). For other fluoroscopysequences with X = 0.75, Chan et al. similarly report animprovement of 2 dB over the 5 x 5 Kuan when they usea spatio-temporal LLMMSE filter (5 x 5 window) combinedwith block matching for motion estimation C321, [33] .Similarabsolute SN R improvements are reported by Katsaggelos etal. [31], but there is no common reference method for directcomparison to our results.Although numerical evaluation is attractive, it does notaccount for small defects such as occasional edge blumngin a single image or ghosting that occurs when residualgray-values remain. Hence, in addition to numerical eval-uation, we perform visual inspection and optimization. Wetry to ensure that filtered images resemble the low-noise

We anticipate improved performance with the Gaussian if its orientationis changed to be similar to the cone (flipped peak down).

input images as closely as possible. We single-step imagesequences in order to examine for ghosting. As might beexpected, we find that ~~i~ and K~~~ can affect ghosting.With our conservative choices, ghosting is not found in anyimage sequence and is relatively unaffected by changes in theother temporal filter parameters ( T ~ ~ ~nd Ptmp). o optimizespatial processing, we view single frames. Parameters (yspand&) are optimized so as to reduce noise with no noticeableobject blurring. Less-sharp, soft-tissue edges are smoothed,but this does not appear to be a problem in angiography.In other fluoroscopy procedures such as in gastro-intestinalradiology, this will have to be evaluated. In angiographyfluoroscopy, it is important to faithfully reproduce arteriesvisualized with test injections of contrast material. We evaluatethe method with some contrast-containing artery images andfind reasonable results despite rapid motion of the contrastbolus and matched-filter kernels that do not ideally matchthe range of artery sizes. The issue of non-ideal kernel sizesalso arises when one zooms the image intensifier, but this issimply solved by changing kernels at different zoom factors.An interesting concept is to use different kernel parameters forcatheter manipulations and contrast test-injections. In order toprove parameter optimization, an exhaustive clinical trial isrequired.Despite the shortcomings of a numerical variance evalu-ation, it is of interest to predict potential effective exposuresavings from image noise reduction. When variance ratios arecalculated over the entire image, we estimate exposure savingsbetween 75 and 91%for X from 2.50-0.50 (Table V). In ROIscontaining no moving objects, effective exposure savings are~ 9 5 % .Table VI). In regions with object movement, this valuedecreases to 80% for X = 0.50 (Section B).We relate results with no motion to absolute image inten-sifier exposure values. A standard image intensifier exposureis 40 pWsec. In our simulations, we use X = 0.50, 0.75, and2.50, correspondingto exposures of zl, 2, and 6 @/sec. Afterspatio-temporal processing, the noise reduction results in anequivalent exposure of 26, 34, and 78 pWsec, respectively.For X = 0.50, 0.75, we do not recreate a variance equivalentto the 40 pR/sec of the original sequence. However, with X =2.50, we create a sequence that contains even less noise thanthe original. These effective exposure values represent best-case scenarios in which maximum spatio-temporal filtering isachieved. However, even in the case of motion where valuesare reduced, the effective dose savings are larger than thoseachieved using low-frame-rate pulsed fluoroscopy [8], [141 orthe x-ray fovea [9].

There are limitations to the above analysis. A fundamentallimitation of using variance reduction to assess filtering isthat it does not account for the interaction of spatio-temporalfiltering with the human observer. It is well known that thespatial frequency characteristic of noise affects detectability.With regard to various image reconstruction problems, Wagnerand Brown review theoretical and experimental investigationsof spatially correlated noise in single images and find signifi-cant deviation from the white noise case [ S I . We previouslyreported a model for the human observer that cascades thenoise from a sequence of image frames with a spatio-temporal


13/14

AUFRICHTIG AND WILSON X-RAY FLUOROSCOPY SPATIO-TEMPORAL FILTERING WITH OBJECT DETECTION 745

filter function for the observer [18]. Experimentally and theo-retically, we are currently investigating the interaction betweentemporal digital filtering and human observer detectability.Both spatial and temporal noise correlations from digitalfiltering will act to lower our dose reduction calculations.There are other limitations. In pixels nearby moving objects,less variance reduction is obtained. We do not Sti& with anoiseless input; hence, sequence noise distributions are notexactly Poisson. Variance calculations are approximate. Elec-tronic noise may be appreciable [59]. While these limitationsmight limit the applicability of using variance reduction topredict dose reduction, the use of variance to compare similarfiltering techniques is probably justified.C. Potential Improvements and Future Work

Within the object-likelihood framework, there are otherprocessing possibilities. Earlier, we investigated a Kalman-type filter that could be modified. The advantage of tlhe Kalmanapproach is that it provides optimal temporal filtering givenan appropriate system and measurement model. Likewise, onecan probably modify the LLMMSE filter [29], [32], [33] toinclude object likelihood.Our analysis suggests using large filter kernels, that nec-essarily require a large number of computations. However,the finite base of support allows parallel processing andcomputation with fast neighborhood processors. Computationsof multiple orientations can be performed in parallel, the 0 and90 kernels are separable, and various reduced-computationapproximations to the oriented matched filter are possible.Current 2-D convolver chips approach the demands of thealgorithm [60].There are other possibilities for improving the object-likelihood images. Model-based approaches similar to thoseused in edge detection [37] are possible. Dallas et al . presentan interesting approach based on dynamic programming[611. Many describe vessel tracking algorithms that maybe applicable, but most methods require user interaction tostart [49]. Earlier, we investigated hysteresis thresholding[57]. This method results in a superior object-likelihoodimage by reducing false positives; however, it necessarilyrequires serial processing. In summary, some of these methodsare promising but add computational complexity Since ourtargeted application is filtering of real-time x-ray ,fluoroscopyimages, we are obligated to consider algorithms which runin parallel.

There are a host of other potential modifications. LUTscan be adjusted to match organ studies andlor a physiciansindividual preference. The use of multiple LU?s is com-mon practice in digital radiography systems. Although de-tectability increases with unsharp-mask processing [42], wecurrently reduce noise but do not change gray-scale valuesof detected objects. Contrast-enhancement is an interestingpossibility. Image perception experiments have been used tostudy detectability in pulsed fluoroscopy at reduced acquisi-tion rates [8], [14], [15], [40]. Similar experiments shouldbe applied to the study of image enhancement of clinicalimages.

ACKNOWLEDGMENTImages were kindly provided by Dr. T. Vrobel, Cardiac CathLab, Metro Health Medical Center, Cleveland, Ohio. D. Geigerand A. Singh of Siemens Corporate Research, Princeton, NJ,provided inspiration and help in the early stages of algorithmdevelopment.

REFERENCES[1] B. Berthelsen and W. Cederblad, Radiation dose to patients andpersonnel involved in embolization of intercerebral arteriovenous mal-formations, Acta Radiol., vol. 32, pp. 492497, 1991.[2] B. D. Lindsey, J. 0. Eichling, D. Ambos, and M. E. Cain, Radiation ex-posure to patients and medical personnel during radiofrequency catheterablation for supraventricular tachycardia, Am . J. Cardiol. , vol. 70, pp.[3] S.P. Jeans and K. Faulkner, An investigation of the radiation dose tostaff during cardiac radiological studies, Br . J. of Radiol., vol. 58, pp.419-428, 1985.[4] P. N. Cascade, L. E. Peterson, W . J. Wajszczuk, and J. Mantel, Radia-tion exposure to patients undergoing transluminal coronary angioplasty,Am . J. Cardiol., vol. 59, pp. 996-997, 1987.[5] E. W. Edmonds, J. A. Rowlands, D. M. Hynes, B. D. Toth, andA. 5. Porter, Digital methods for reducing exposure during medicalfluoroscopy, in Proc. SPIE Med. Imag. IV , Bellingham, WA, pp.406-411, 1990.[6] National Council on Radiation Protection and Measurement, Bethesda,MD, Exposure of the U S . Population from Diagnostic Medical Radi-ation, Rep. 100, 1989.[7] A. Macovski, Medical Imaging. Englewood Cliffs, NJ: Prentice-Hall,1983.[8] R. Aufrichtig, P. Xue, C. W. Thomas, G. C. Gilmore, and D. L. Wilson,Perceptual comparison of pulsed and continuous fluoroscopy, Med.[9] M. S.Labbe, M.-Y. Chiu, M. S. Rzeszotarski, and D. L. Wilson, Thex-ray fovea, a device for reducing x-ray dose in fluoroscopy, Med.Phys., vol. 21, pp. 471481, 1994.[lo] S.Rudin and D. R. Bednarek, Region of interest fluoroscopy, Med.Phys., vol. 19, pp. 1183-1189, 1992.[11] J. H. Grollman, Jr., Radiation reduction by means of low pulse-ratefluoroscopy during cardiac catheterization and coronary arteriography,

Am . J. Roentgenol., vol. 121, pp. 636-641, 1974.[12] M. A. Wondrow, A. A. Bove, D. R. Holmes, J. E. Gray, and P. R.Julsrud, Technical consideration for a new x-ray video progressivescanning system for cardiac catheterization, Catheter. Cardiovasc.Diagn., vol. 14, pp. 126-134, 1988.[13] D. R. Holmes, M. A. Wondrow, J. E. Gray, R. J. Vetter, J. L. Fellows,and P. R. Julsrud, Effect of pulsed progressive fluoroscopy on reductionof radiation dose in the cardiac catheterization laboratory, J. Am . Coll.

218-223, 1992.

P h y ~ . , ol. 21, pp. 245-256, 1994.

Cardiol., vol. 15, pp. 159-162, 1990.1141 R. Aufrichtig, C. W. Thomas, G. C. Gilmore, and D. L. Wilson,Comparison-of detectability in pulsed versus continuous fluoroscopy:A simulation study, in Proc. SPIE Med. Imag. VI, Y. Kim, Ed.,Bellingham, WA, 1992, vol. 1653, pp. 374-378.1151 P. Xue, R. Aufrichtig, C. W. Thomas, G. C. Gilmore, and D. L. Wilson,The perception of moving objects in reduced acquisition rate pulsedfluoroscopy, in 5th Far West Image Perception Con$. Newport, RI,1993.[16] S.L. Fritz, S.E. Mirvis, S.0. Pais, and S.Roys, Phantom eval-uation of angiographer performance using low frame rate acquisitionfluoroscopy, Med. Phys., vol. 15, pp. 600-603, 1988.[17] A. Abdel-Malek, F. Yassa, and J. Bloomer, An adaptive gating ap-proach for x-ray dose reduction during interventional procedures, IEEETrans. Med. Im ag., vol. 13, pp. 2-12, 1994.[18] R. Aufrichtig, C. Thomas, P. Xue, and D. L. Wilson, A model forperception of pulsed fluoroscopy image sequences, J. Opt. Soc. Am.,A, vol. 11, pp. 3167-3176, 1994.

[19] G. W. Schmidt, R. G. Hier, S. Benyamin, and S. E. DeForest, Real-timeadaptive contrast enhancement of medical images by an inexpensivemodular image processor, in Proc. SPIE Med. Imag. , 1988.[20] S. . Riederer, A. L. Hall, J. K. Maier, N. J . Pelc, and D. R. Enzmann,The technical characteristics of matched filtering in digital subtractionangiography, Med. Phys. , vol. 10, pp. 209-217, 1983.[21] R. A. Kruger, A method for time-domain filtering using computerizedfluoroscopy, Med. Phys. , vol. 8, pp. 466470, 1981.


14/14

746 JEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 14, NO . 4, DECEMBER 1995

1221 C. W. Hardin, R. A. Kruger, F. L. Anderson, B. E. Bray, and J. A.Nelson, Real-time digital angiocardiograhy using a temporal high-passfilter, Radiol., vol. 151, pp. 517-520, 1984.[23] J. T. Dobbins, M. S. V. Lysel, B. H. Hasegawa, W. W. Peppier, and C. A.Mistretta, Spatial frequency filtering in digital subtraction angiography(DSA) by real-time digital video convolution, in Proc. SPIE: Applicuf.Opt. Instrum. Me d X I , G. D. Fullerton, Ed., Bellingham, WA, vol. 419,pp. 111-121, 1983.[24] E. Dubois and S. Sabri, Noise reduction in image sequences usingmotion-compensated temporal filtering, IEEE Trans. Commun., vol.[25] T. S. Huang and Y. P. Hsu, Image sequence enhancement, in ImageSequence Analysis, T. S. Huang, Ed. Berlin: Springer Verlag, 1981,[26] A. Singh, Optic Flow Computation: A Unijied Perspective. Los Alami-tos, CA: IEEE Computer Society Press, 1992.[27] A. Abdel-Malek, 0. Hasekioglu, and J. Bloomer, Image segmentationvia motion vector estimates, in Proc. S H E M ed . f m g .N , Bellingham,WA, 1990, pp. 366-371.[28] P. Roos and M. A. Viergever, Registration and reversible compression

of angiographic image sequences, in Proc. SPIE Med Imag. HI,Bellingham, WA, 1989, pp. 383-391.[29] D. T. Kuan, A. A. Sawchuk, T. C. Strand, and P. Chavel, Adaptivenoise smoothing filter for images with signal-dependent noise, IEEETrans. Pattem Anal. Mach. Intell., vol. PAM-7, pp. 165-177, 1985.[30] S.-S. Jiang and A. A. Sawchuk, Noise updating repeated Wiener filterand other adaptive noise smoothing filters using local image statistics,Appl. Opt., vol. 25, pp. 2326-2331, 1986.[31] A. K. Katsaggelos, J. N. Driessen, S. N. Efstratiadis, and R. L.Lagendijk, Spatio-temporal motion compensated noise filtering ofimage sequences, in Proc. SPIE Esual Commun. I m g e Process ing N,Bellingham, WA, pp. 61-70, 1989.1321 C. L. Chan, B. J. Sullivan, A. V. Sahakian,A. K. Katsaggelos, T.Frohlich, and E. Byrom, Spatio-temporal filtering of digital angio-graphic image sequences corrupted by quantum mottle, in Proc. SPIEMed. Imag. V, Bellingham, WA, vol. 1450, pp. 208-217, 1991.[33] C. L. Chan, A. K. Katsaggelos, and A. V. Sahakian, Tmage sequencesfiltering in quantum-limited noise with applications to low-dose fluo-roscopy, IEEE Trans. Med. Imag., vol. 12, pp. 610-621, 1993.[34] J. B. Thomas, Statistical Communication Theory. New York, Ny:Wiley, 1969.[35] P. Z. Peebles, Communication System Principles. Reading, IvlA:Addison-Wesley, 1976.[36] M. L. Giger, K. Doi, and C. E. Metz, Investigation of basic imagingproperties in digital radiography. 2. Noise Wiener spectrum, M e dPhys., vol. 11, pp. 797-805, 1984.[37] W. K. Pratt, Digital Image Processing, 2nd ed. New York Wiley,1991.[38] S. Chaudhuri, S. Chatterjee, N. Katz, M. Nelson, and M. Goldbaum,Detection of blood vessels in retinal images using two-dimensionalmatched filters, IEEE Trans. Med. I m g . , vol. 8 , pp. 263-269, 1989.[39] L. Zhou, M. S. Rzeszotarski,L. J. Singeman, and J. M. Chokreff, Thedetection and quantification of retinopathy using digital angiogram,IEEE Trans. Med. Imag., vol. 13, pp. 619 426 , 1994.[40] D. L. Wilson, P. Xue, and R. Aufrichtig, Perception of fluoroscopy

COM-32, pp. 826-831, 1984.

pp. 290-310.

last-image-hold, Med. Phys., vol. 21, pp. 1875-1883, 1994.[41] I. F. Hawkins, Carbon dioxide digital subtraction angiography,Am . J.Ruenfgerwl., vol. 139, pp. 19-24, 1982.1421 L. D. Loo, K. Doi, and C. E. Metz, Investigation of basic imagingproperties in digital radiography. 4. Effect of unsharp masking on thedetectabilityof simple pattems, Med. Phys., vol. 12,no. 2, pp. 209-214,1985.[43] J. W. Candy, Signal Processing. The Modem Approach. New York,NY:McGraw-Hill, 1988.[44] C. E. Metz, Basic principles of ROC analysis, Sem. Nucl. Med., vol.[45] C. E. Metz, ROC methodology in radiologic imaging , Invest. Radiol.,vol. 21, pp. 720-733, 1986.[46] I.A. Swets, ROC analysis applied to the evaluation of medical imagingtechniques, Invest. Radiol., vol. 14, pp. 109-121, 1979.[47] C. E. Metz, D. J. Goodenough, and K. Rossmann, Evaluation of re-ceiver operating characteristic curve data in terms of information theory,with applications in radiography,Radiol., vol. 109, pp. 297-303, 1973.[48] K. Myers, H. Barrett, M. Borgstrom, D. Patton, and G. Seeley, Effectof noise. on detectability of disk signals in medical imaging, J. Opt.S oc . A m A , vol. 2, pp. 1752-1759, 1985.[49] K. R. Moffmann, K. Doi, S. Chen, and H. Chan, Automated trackingand computer reproduction of vessels in DSA images, Invest. Radiol.,vol. 25, pp. 1069-1075, 1990.[50] H. Chan, K. Doi, C. J. Vyborny, K. Lam, and R. A. Schmidt, Computer-aided detection in mammograms: Methodology and preliminary clinicalstudy, Invest. Radiol., vol. 23, pp. 66Lc671, 1988.[51] A. Rosenfeld and A. C. Kak, igital Picture Processing. 2nd ed., vol.1 and 2. Ne w York: Academic, 1982.[52] E. Krestel, Ed., Imaging Systems for Medical Diagnostics. Berlin:Siemens AG, 1991.[53] A. Singh, D. L. Wilson, and R. Aufrichtig, Enhancement of x-rayfluoroscopy images, in Proc. SPIE Med. Imag., Bellingham, WA, vol.1898, pp. 3W 31 0, 1993.[54] R. Aufrichtig and D. L. Wilson, Marking arteries and catheters in x-ray fluoroscopy using morphological filtering, n Proc. Con$ IEEE Eng.M ed Biol.Soc., Piscataway, NJ, IEEE, vol. 14, pp. 1 8541 855, 1992.[55] I. Pitas and A. N. Venetsanopoulos,Nonline ar Digital F ilters: P rinciplesand Applications.[56] _ _ , Order statistics in digital image processing, Proc. IEEE, vol.[57] R. Aufiichtig, D. Geiger, A. Singh, and D. L. Wilson, Spatia-temporalx-ray fluoroscopy filtering using object detection, in Computers inCardiology. Washington, D.C.: IEEE Computer Society Press, 1993,pp. 587-590.[58] R. F. Wagner and D. G. Brown, Unified SNR analysis of medicalimaging systems, Phys. Med. Biol., vol. 30, pp. 489-518, 1985.[59] H. Luijendijk, Practical experiments on noise perception in noisyimages, in Proc. SPIE Med. Imag. I994: Image Perception, H. L.Kundel, Ed., Bellingham, WA, vol. 2166, pp. 2-8, 1994.[60] PDS Pl64 88: Single chip 2-D c onvolver with integral line delays, PlesseySemiconductors, Scotts Valley, CA, 1989.[61] W. J. Dallas and H. Roehrig, A dynamically programmed computeralgorithm for assisting the radiologist in locating vessels, in Proc. SPlEMed. I m g . , Y. Kim, Ed., Bellingham, WA, vol. 767, pp. 4 62 47 0, 1987.

8, pp. 283-298, 1978.

Boston, MA: Kluwer, 1990.80, pp. 1893-1921, 1992.

(Aufrictig e Wilson, 1995)

Documents

Transcript of (Aufrictig e Wilson, 1995)