DIFFERENTIAL STRUCTURE OF IMAGES: ACCURACY OF REPRESENTATION

Bart M. ter Huar Romeny, Wiro J. Niessen, Janita Wilting and Luc M.J. Florack

3D Computer Vision Research Group, Utrecht University Hospital, Room E.02.222,

Heidelberglaan 100, 3584 CX Utrecht, The Netherlands email: bart@cv.ruu.nl

ABSTRACT

Differentiation is known to be ill-posed in the sense of Hadamard. The theory of regular tempered dist,ribu- tions and the concept of Gaussian convolution filters open the way to a well-posed differentiation process, thereby introducing the notion of scale (or: inverse resolution). There is no a priori fundamental limit to the order of differentiation of images provided they are calculated on a sufficiently high scale (relative to pixel scale and noise correlation width), and provided we have a sufficient dynamic range of intensity values. Constraints in resolution (both in the spatial and in the intensity domain) enforce a scale-dependent restriction to the accuracy with which Gaussian kernels G,(z; CT) can be representled in a physical sense. So at a given scale c (e.g. in units of the sampling scale) and a given measure of inaccuracy a there is a maximal order n above which the margin a is exceeded. In this paper we quantify this relation.

1. INTRODUCTION

Scale-space is a necessary paradigln when we consider a physical observation by some linear visual system, that has total ignorance of the outside world. For, when there is no a. priori preference for locat,ion, orient,a.tion and size of the involved aperture, we end up with a system that has receptive fields for all locations, ori- entations and aperture sizes (scales) [ I , 2, 3, 4, 51. It is this last parameter that creates t,he one-para.meter family of scaled images that we call a scale-space [6]. It has been shown in a number of ways that the Gaussian kernel G(5; U ) is the unique kernel in this case [T, 81:

in D dimensions. The generating equation f )r linear scale-space is the isotropic diffusion equation

aL at - = AL

where AL denotes the Laplacean. The Gaussian ker- nel is the Green's kernel of the diffusion equation and convolving the image with a Gaussian yields a higher level in scale-space. It is the only kernel that does not generate new detail when applied (causality).

Interestingly, extracting the differential structure of observed data ( L ) with such a front-end visual sys- t,em h r n s out to be equiyalent t_o convolution of the 'real outside world data' ( L ; L = L * G) with 3aussian derivative kernels because differentiation and convolu- tion commute:

al; ~ ( L * G ) - aG - _ - - = L * - 8 X d X d X

The Gaussian derivative kernels form a complete set of scaled dzflerentzal operators [l]. The zeroth cnder ele- ment can be seen as the tdentzty opemtors, pcbrforming just a scaling.

This set has highly attractive properties, its strong regularization [9] , rotation invariance, separa )ility and cascading to form higher order operators.

2. ACCURACY, SCALE ANI! DIFFERENTIAL ORDER.

Now being able to extract higher order differerrtial struc- ture from discrete data in a regularized way, a number of questions arise regarding the accuracy of 1 he result. The number of zero-crossings of the spatial represen- tation of the kernels is equal to the order, and it is clear that increasing the order with a given spatial sam- pling grid leads to representation problems. This can be counteracted by increasing the scale of the operator.

21 0-8186-6950-0194 $4800 0 1994 IEEE

There must be a fundamental relation between the scale of the opemtor, the order of differentiation, the spatial sampling distance and the error allowed. This paper addresses this relation, which can be expressed in the following two basic questions:

4 0 0 0 0 .

30000

20000

1 0 0 0 0

1. Given an image, with specified sampling distance (resolution) and size, to what order can differential structure be extracted given a level of accuracy? E.g. what order information can faithfully be extracted at a given scale from an MR image of 256 x 256 pixels?

4 0 0 0 0 .

3 0 0 0 0 .

2 0 0 0 0 .

1 0 0 0 0 .

2. If we want to extract differential st,ructure at a cer- tain order and a specified scale, how good should we measure? E.g. if we want to ext,ract corners (requiring second order kernels) a t some scale, what resolution of acquisition is required?

The finite intensity discretization of measured data is another serious limit on the accuracy of extraction of differential structure. This will be treated in a forth- coming paper. An example where very high order im- age derivative kernels appear will be presented in the second part of this paper.

3. THEORY

There is a direct relation between scale of an opera- tor and resolution: We define the natural scale U as the scale parameter of the kernel U,, expressed in the intrin- sic distance, the pixel-or sampling distance 6 : U = u ~ / E .

Convolution with Gaussian derivatsive kernels in the spatial domain is equivalent to a product of the Fourier transform of the image with the Fourier transform of the kernel in the Fourier domain. The (1-dimensional) nth order Gaussian derivative kernel with scale U has the following representation in the Fourier domain:

&(U; U) = ( i ~ ) ~ exp(-u2w2/2) (1)

order n- 2 Order n- 1

Figure 1: Fourier spectrum of Gaussian derivative ker- nel for order 1 (left) and 2 (right) as a function of spa- tial scale.

Figure 1 shows the first and second order function as a function of scale 6. The discrete Fourier transform is intrinsically periodic, with a period of 2 r . Figure 1 shows just this period.

For smaller scale U the Fourier transform of the ker- nel increases in width, a t a certain scale giyring rise to aliasing. In theory this occurs at all scales due to the infinite extent of the exponential function, but it be- comes apparent at smaller scales. The 'le;.ked' infor- mation is folded back, in theory even from all further periods as well, so the amplitude no longer represents the accurate value of 'pure' differentiation.

amp1 .

~ 0 .5 1 1 . 5 2 2 .5 3 ( 9

Figure 2: Aliasing error of a 24th order Gaussian derivative kernel at scale 1.9 pixels in the I'burier do- main. Only positive frequencies are show n. Lower curve: Fourier spectrum without aliasin];. Upper dashed curve: Resulting spectrum with aliaing.

The moment this aliasing starts to be significant gives us the relation between scale, order of lifferentia- tion and error we look for. Note the shape of the equa- tion (1 ) at, U = 0. The term (iw)" amplifies the higher frequencies in the spectrum strongly for im:reasing n. The exponential weighting of the Gaussian ilways (for any n ) presses the function t,o zero finally, i i the limit w + +20.

As an example, in figure 2 the aliasin); is shown (dashed) for a 24th order kernel at scale U = 1.9 pixels.

We consider the powerspectrum, i.e. tht: square of the signal. The error can be defined as t i e relative energy of the aliased frequencies over the total energy:

22

where r (n ) is the Euler gammafunction, and r(n, Z O , Z I )

is the generalized incomplete gamma function r ( n , z0)-

r(n, 21). Figure 3 shows the aliasing error as a function of scale U and order n for orders up to 100. Note that indeed for higher order a larger scale must be selected.

Figure 3: Error of aliasing as a function of differential order ( n ) and scale (s) of the Gaussian derivative oper- ator, expressed in pixels. Upper graph: orders ( n ) up to 100, scales (s) up to 5 pixels. Lower graph: same function, orders up to 10, scales up to 2 pixels. Both graphs: top level is equal to 5% error.

The error allowed is a matter of choice, dependent on the application. In figure 4 the lines for error(n , U ) = 0.05 (5%) and error(n , U ) = 0.01 (1%) are plotted. Be- cause the lower order derivatives are computed most frequently, a graph is given for orders up to 10. The error as a function of U and n is rather steep, indicating

that the maximum allowed order for a given r (or the reverse) is rather independent of the particular choice of the errorlevel. This is clear from the c1osene:a of the lines in figure 4.

4 1

0 1 0 20 4 0 60 80 1 O C

Figure 4: Maximum differential order that can be extracted with Gaussian derivative operator:; without aliasing as a function of scale of the Gaussian Iierivative kernels expressed in pixels. Upper graph: hc rizontal - orders up to 100, vertical - scales up to 5 pixtls. Lower graph: same function, orders up to 10, scales up to 2 pixels. Allowed error: 5% (upper line in each plot) ~ 1% (lower line in each plot).

This error occurs due to limits in the inner scale, i.e. the kernelsize approaches the resolution of the mea- surement. In the frequency domazn this lec to a too wide spectrum. A similar bound occurs a t the outer scale, i.e. the field of view of the measuremer t (the im- age). This means that if the kernel gets too wide in the spataal domain, we can define a similar error by taking the relative energy of the 'leaked' kernel ovel- the total

23

energy of the kernel. This clearly reveals the duality of the spatial and frequency domain. If we consider both domains, an upper bound for small scales is expected due to aliasing in the Fourier domain, while for larger scales the upperbound will be due to aliasing in the spatial domain. The intersection of both curves then implies a resolution dependent maximurn in the graph of maximum differential order as a function of scale, given a level of inaccuracy.

4. EXAMPLE: DEBLURRING GAUSSIAN BLUR.

5. DISCUSSION

We estimated errors in representation, whcn extract- ing high order structure. The scale of thz operator has to be large enough to prevent aliasing to disrupt the accuracy of the measurement. The iniplementa- tion of convolution with Gaussian derivat ve kernels can be done both in the spatial as in the lburier do- main. Both approaches are in principle equilalent. The Gaussian derivative operators as presented in this pa- per are linear differential operators. Recent1.r there has been much interest in non-linear scale-spaces [13].

he reconstructed to a delta-spike when applying t,he in- finite series. Clearly the approximate square-root rela.tion bet.ween order, scale of operator and error is seen in this appli- cation. More details of the deblurring technique can be found in [lo].

'This required the calculation of 1128 spatial partial deriva- tives. Implementation was done with eq. ( 1 ) and an automated parserjcode generator.

Figure 5 : Deblurring a Gaussian blob to ! )2 th order wi th different operator wiclt hs and intensity liscretiza- tion. Intensity representation in doubles. Top row: blurred spike, c = 10 pixels. Second to bottom row: Deblurring from 2"' to 92t" order in steps ( f 6 orders of different8iation. Left to right, column: Standard de- viation of Gaussian differential operator of l , 3, 5 , 7, 9, 11, 13, 15, 17 and 19 pixels. Image resolut on 64x64 pixels. All linages individually normalized.

24

In non-linear geometric scale-spaces (like afine or projective scale-space) [14, 15, 161 they are no longer correct. Approximation with linear operators in an it- erative scheme is feasible, but only at small scale. This study shows that some care must then be taken.

The authors thank Alfons Salden, Arnold Harbers, Ruud Geraets and Max Viergever for their valuable contributions. They are grateful to NSF and Esprit for funding the collaboration on ” Geometry-Driven Diffu- sion” in which context this research was carried out.

All graphs were produced with MatheinaticaTM.

6. REFERENCES

[I] L. M. J. Florack, B. M. ter Haar Romeny, J . J. Koen- derink, and M. A. Viergever, “Scale and the differential structure of images,” Image and Vision Computing,

[2] T. Lindeberg, Scale-Space Theory in Computer Vi- sion. The Kluwer International Series in Engineering and Computer Science, Dordrecht, the Nehherlands: Kluwer Academic Publishers, 1994.

[3] J . J. Koenderink, “The structure of images,” Biol. Cy-

[4] B. M. ter Haar Romeny and L. M. J. Florack, “A mul- tiscale geometric model of human vision,” in Percep- tion of Visual Information (W. R. Hendee and P. N. T. Wells, eds.), ch. 4, pp. 73-114, Berlin: Springer-Verlag, 1993.

[5] T. Lindeberg and B. M. ter Haar Romeny, “Linear scale-space: I. basic theory. ii. early visual opera- tions.,” in Geometry- Driven Diffusion in Computer Vision (B. M. ter Haar Romeny, ed.), Computational Imaging and Vision, pp. 1-38,39-72, Dordrecht, t,he Netherlands: Kluwer Academic Publishers, 1994.

[6] A. P. Witkin, “Scale space filtering,” in Proc. Inter- national Joint Conference on Artificial Intellagerace, (Karlsruhe, Germany), pp. 1019-1023. 1983.

vol. 10, pp. 376-388, July/August 1992.

bern., vol. 50, pp. 363-370, 1984.

[7] L. M. J. Florack, The Syntactical Structtrl-e of Scolrrr Images. PhD thesis, University of titredit,, Iltrecht, The Netherlands, November 1993. Cum Laude.

[8] J. Babaud, A. P. Witkin, M. Baudin, and R. 0. Duda, “Uniqueness of the Gaussian kernel for scale-space fil- tering,” IEEE Trans. Pattern Analysis and h-lnchine Intelligence, vol. 8, no. 1, pp. 26-33, 1986.

[9] L. M. J. Florack, B. M. ter Haar Romeny, J. J. Koenderink, and M. A. Viergever, “lmages: Regular tempered distributions,” in Proc. of the NATO 4d- vanced Research Workshop Shape in Picture - Math- ematical description of shape in greylevel images (Y.- L. 0, A. Toet, H. J. A. M. Heijmans, D. H. Foster, and P. Meer, eds.), vol. 136 of NATO .4SI Series F. pp. 651-660, Springer Verlag, Berlin, 1994.

[IO] B. M. ter Haar Romeny, L. M. J. Florack, M. de Swart, J . Wilting, and M. A. Viergever, “Deblurring Gaussian blur,” in Proc. Mathematical Methods in Medrcal Imag- ing 11, vol. 2299, (San Diego, CA), SPIE, Jidy, 25-26 1994.

[I11 R. A. Hummel, B. B. Kimia, and S. W. Zucker, “De- blurring Gaussian blur,” Computer Vision, Graphics, and Ima,ge Processing, vol. 38, pp. 66-80, 19;%7.

[I?] B. B. Kimia and S. W. Zucker, “Analytic nverse of discrete Gaussian blur,” Optical Engineerin, 1, vol. 32, no. 1, pp. 166-176, 1986.

[13] B. M. ter Haar Romeny, ed., Geometry-Drillen Diffu- sion in Compciter Vision. Computational Im iging and Vision, Dordrecht: Kluwer Academic Publishers, 1994.

[I41 G. Sapiro and A. Tannenbaum, “Affine invariant scale- space,” Intevnational Joirrnal of Compute r Vision, vol. 11, pp. 25-44, 1993.

[15] T. Cohignac. F. Eve, F. Guicharcl, and (?. Lopez, “Affine morphological scale space: Numerical analysis of its fundamental equation,” tech. rep., CER EMADE, IJ1iiversit.e Paris Dauphine, 1993. unpublished.

[16] 0. Faugeras, “Carhn’s moving frame methc’d and its applications to the geometry and evolution of curves in the Eiiclidean, affine and projective plana:s,” Tech. Rep. TR-2053, INRIA, 1993.

25