IEEE International Symposium on Industrial Electronics (ISlE 2009)Seoul Olympic Parktel, Seoul, Korea July 5-8, 2009--

Fast Image Blurring Using Lookup Table for RealTime Feature Extraction

Jegoon RYUGraduate School of Information,

Production and SystemsWaseda University

Fukuoka, Japan 808-0135Email: jkryu@ruri.waseda.jp

Abstract- In this paper, a novel fast Gaussian blurring methodthat uses Look-Up Table (LUT) is presented for creating areal time Difference of Gaussian (DoG) pyramid. The LUT forfast and accurate Gaussian blur is obtained from the valuescalculated from one-dimensional(lD) Gaussian elements andpixel values of image. To evaluate the method, computation timeand similarity with a conventional two-dimensional(2D) Gaussianblur are measured, and compared to other blurring methods.Based on the computational results, the proposed method showsgood performance than other blurring methods such as box filter,Stack-blur, Romain Guy's method, and recursive Gaussian blur.The proposed method could be effectively applied to create theDoG pyramid for feature extraction.

I. INTRODUCTION

Image blurring (or smoothing) is one of the most funda­mentally used operations in image processing. There are well­known linear and non-linear blurring methods such as conven­tional 2D Gaussian filter and median filter. They are used asparts of more complex algorithms including feature extraction,noise reduction, and segmentation. Lowe [1] applied theconventional Gaussian filter for creating DoG pyramid in scaleinvariant feature transform (SIFT). DoG kernel (D(x, y, u))and Gaussian filter (G(x, y, u)) are represented as follows :

D(x, y, o ) == L(x, y, ku) - L(x, y, o ) (1)

L(x, y, o ) == G(x, y, o ) * I(x, y) (2)

In equation 1, sigma value is increased by a certain stepto obtain the DoG pyramid. Also, for calculating accuratelocal extrema, a floating-point computation is performed inGaussian convolution. This process requires relatively muchcomputation cost. Bay [2] applied the box filter for fastimplementation of feature extraction, called Speed Up RobustFeature (SURF). SURF approximates second order Gaussianderivatives with box filters. Although the method has fastperformance, the accuracy of feature point is that SIFT issuperior to SURF, because of a floating-point computation ofGaussian filter.

There are several approaches for fast blurring. Young [3]and van Vliet [4] have designed computationally efficientmethods for approximating Gaussian-based convolutions by

978-1-4244-4349-9/09/$25.00 ©2009 IEEE

1864

Toshi Hiro NISHIMURAGraduate School of Information,

Production and SystemsWaseda University

Fukuoka, Japan 808-0135Email: tosi-hiro@waseda.jp

running a recursive infinite-impulse-response (IIR) filter for­ward over the input signal, then running a second IIR filterbackward over the first filter's output. To transition betweenthe two filters, they use a suboptimal heuristic that producessignificant amplitude and phase distortion for all points withinabout three standard deviations of the right-hand boundary.Perri [5] introduced a SIMD 2-D convolver for fast FPGA­based image and video processor. The authors utilized there-configurability nature of FPGA to introduce a new SIMDimage and video processor. This processor can dynamicallyadapt itself to perform 2-D convolutions between variable bitresolution. The results showed efficient solution for image con­volution when compared to traditional DSP. Alex [6] presenteda few tricks which can help make real-time blurring possible.Alex used the box filter for implementation of fast blurring.The computation cost of the method is independent of kernelsize, meaning that it takes the same time to blur the image nomatter how much blurring is required. Mario Klingemann [7]used a compromise method between Gaussian blur and boxblur. The method is called Stack-blur, since it uses a kind ofmoving stack of colors whilst scanning through the image.Romain Guy [8] and Durand Emmanuel [9] introduced fastmotion blurring for motion blur on web.

In this paper, we propose the fast Gaussian blurring methodusing look-up table. The purpose of this paper is to highlightmore fast and accurate optimizations of blurring algorithms.Also, this fast and accurate blurring method is required for realtime feature extraction. Look-Up Table technique has utilizedfor fast computation, and leads to low computation cost forimage processing [10]-[14]. In the design of LUT, ID Gaus­sian coefficients and pixel value of image are used to obtainthe LUT. And, for performing two directional convolutions,the method uses integer coefficients LUT instead of floating­point. This approach achieves low computational cost, and cansignificantly outperform straightforward implementation byusing a Very Large Scale Integrated circuit (VLSI). Evaluationof the method is measured by computation time and similaritywith a conventional 2D Gaussian blur, and compared to otherblurring methods such as box filter, Stack-blur, Romain Guy'smethod, and recursive Gaussian blur. Based on the results, theproposed method has good performance at computation cost

(a) (b) (c) (d)

Fig. I. Two-dimensional convolution filters for blurring image: (a) Box filter, (b) Tent filter, (c) Quadratic filter, and (d) Gaussian filter.

(5)

(4)

+rQ[y ,x] = L C[y,x + k]

k = - r

where,

bo 1.57825 + 2.44413q + 1.4281q2 + 0.422205 q3

b1 2.44413q + 2.85619q2 + 1.26661q3

b2 - 1.4281q2 - 1.26661q3

b3 0.422205l

and the relation between the desired a and q is given by:

{0.98711a - 0.96330, a ?: 2.5

q = 3.97156 - 4.14554V1 - 0.26891a, 0.5 :::; a :::; 2.5.

A relative accuracy of this approximation increases as filterradius a increases, but even with small a the accuracy isgood [3].

3) Box blur: A box blur, also known as moving average,is an simple linear filter in which each pixel in the resultingimage has a value equal to the average value of its neighboringpixels in the input image. The property of using all equalcoefficients allows implementation with a simple accumulationalgorithm . It is efficiently faster than using a sliding windowalgorithm, but it lacks smoothness of a Gaussian blur. Thealgorithm is based on a fact that sum Q of elements inthe rectangular window can be decomposed into sums C ofcolumns of this window:

The one-dimensional recursive filter has the normalizationconstant B, and the filter coefficients can be calculated asfollows:

Thus, the per-pixel complexity of Gaussian blur becomesO(logr). However, the constant is quite large, and for manypractical purposes Gaussian blur can be successfully approxi­mated with simpler filters.

2) Recursive Gaussian filter: Using higher orders of recur­sive filter allows a good approximation of Gaussian filtering.This filter should be computed in two passes : forward andbackward directions, and it has 12 multiplications per pixel intwo-dimensional case, independently of the value of a in theGaussian kernel. The following one-dimensional forward (orbackward) difference equation takes an input (or intermediateresult) x[n] and produces an intermediate output result (or theoutput) y[n] [3] :

y[n] = Bx [n]+ (b1y [n- 1] + b2y[n- 2]+ b3y[n- 3])jbo (3)

A. Linear blurring

J) Gaussian blur: A Gaussian blur is created by applyinga Gaussian function, which is used in statistical for normaldistribution. The Gaussian blur enables a wide range ofphotorealistic effects. It is also used as pre-processing stepof reducing image noise, reducing detail, and scale-spaceconstruction.

A Gaussian blur effect is mathematically generated by con­volving an image with a kernel of Gaussian values. Gaussianfilter on a square support is linearly separable. That is, two­dimensional Gaussian filter can be decomposed into a seriesof one-dimensional filtering for rows and columns. Whenthe filter radius is relatively small (less than few dozens),the fastest way to calculate the filtering result is direct one­dimensional convolution. When a filter radius is large, di­rect convolution becomes expensive, and FFT-based overlap­add(OLA) convolution is often used to perform an efficientGaussian blur for large blur radius . A common mistake hereis to transform the whole image row (or column) with FFT,do the same with a zero-padded Gaussian kernel, multiplycomplex spectra and do the inverse transform. First of all, itshould be considered that the result of convolution has a lengthN + M - 1, where N is the signal size and M is a filter kernelsize (equal to 2r + 1), i.e. the output signal is longer thanthe input signal. Without proper padding (extension) of data,regular convolution will turn to a circular convolution , leadingto problems near image boundaries. Secondly, calculating FFTof the complete image row is not optimal , since the complexityof FFT is O(N log N) . The complexity can be reduced bybreaking the kernel into sections with an approximate lengthM and performing OLA convolution section-wise . The FFTsize should be selected so that circular convolution is excluded.Usually optimal performance is achieved when FFT size F isselected as the smallest power of 2 larger than 2M, and signalsection size is selected as F - M +1 for full utilization of FFTblock. This reduces the overall complexity of one-dimensionalconvolution to O(N log M).

and similarity compared to other blurring methods.This paper is organized as follows. Section II shows

the blurring methodology. In Section III, the proposed fastGaussian is represented in detail. Computational results aresummarized in Section IV. Finally, the conclusion is presentedin Section V.

II. BLURRING METHODOLOGY

1865

where C[y,x] is sum of (2r + I)-pixel columns centered at[y, x], and Q[y, x] is sum of (2r + 1)(2r + I)-pixel squarewindow centered at [y, x]. This allows a simple update rulewhen window Q is moving from left to right:

Q[y, x + 1] == Q[y, x] + Cry, x + r + 1] - Cry, x - r] (6)

Column sums C can be, in turn, effectively updated whenwindow Q is moving down to the next row:

Cry + 1,x] == C[y,x] + I[y + r + 1,x] - I[y - r,x] (7)

where I [y, x] is image pixel value. The result of the box filteris equal to

complexity on i.i.d. data) is also introduced in [16]. It hasa complexity of O(logr). This running max algorithm canbe used for adding pixels to a 2D window of a 2D minimaxfilter with a worst-case complexity of 0 (r log r) operationsper pixel.

3) Kuwahara filter: A Kuwahara filter a edge-preservingsmoothing filter, which smoothes . It takes a kernel aroundthe objective pixel the input image of size J == K == 4L + 1,where L is a integer. This kernel is split into four overlappingregions. In each of the four regions, the mean brightness andvariance are measured. The output value is the mean value ofthat region has the smallest variance.

(12)_l-exp-(~)~a

oa.; [n] = {

The number of samples (N) for Gaussian convolution usingone-dimensional kernel is finite. It is common to choose N ==3u or Su.

g2D (x, y)

Table 1 and 2 represent the coefficients of one-dimensionalGaussian kernel from sigma 1 to sigma 2)2, and the samplenumber of Gaussian kernel is 1 x 7 and 1 x 9.

III. THE PROPOSED METHODOLOGY

A Gaussian blurring effect on image is typically generatedby convolving an image with Gaussian filter as changingsigma. In practice, it is best to take advantage of the GaussianBlur's linearly separable property by dividing the process intotwo passes. In the first pass, a one-dimensional kernel isused to convolve the image in only the horizontal or verticaldirection. In the second pass, another one-dimensional kernelis used to convolve in the remaining direction. The result hasthe same effect as convolving with a two-dimensional kernelin a single pass, but requires fewer computations.

where a is referred to as both the standard deviation and scale.The multi -dimensional Gaussian filter can be separated intoone-dimensional Gaussian filters along its main axes. Equation10 can be rewritten as follows:

A. Separable Gaussian filter

The use of the Gaussian kernel for blurring has become ex­tremely popular. The point spread function (PSF) and transferfunction for the space Gaussian are given by:

(8)

(9)Ixl < T

else{1- E1

T 'oBartlett(x, T) ==

1B[y, x] = (2r + 1)2 Q[y, x]

and it requires one multiplication per pixel after Q [y, x] iscalculated. So, the overall per-pixel complexity of the box filteris 4 additions and 1 multiplication per pixel. Some additionalover-head is required to calculate initial values of C and Q atthe image boundaries.

4) Triangular blur: The blurring effect of a box filter isoften not sufficient. An iteration of box filter can improve theblurring effect. That is, as the number of iterations increases,the superposition filter approaches a Gaussian blur. Triangularfilter is simply a Bartlett function ( or tent function ). it canbe constructed as constructed as a 2 box filters with the sameradius. The computational complexity of the filter is twice ofa box filter, and visual effect is similar to Gaussian blur.

B. Non-Linear blurring

1) Median filter: Median is a non-linear spatial filter forthe smoothing of signals. The output value is the middlevalue of neighboring pixel values which is calculated by firstsorting all the pixel from the surrounding neighborhood intonumerical order. The kernel is usually square but can be anyshape. The filter is often used to remove noise from imageor other signals, and is particulary useful for salt and peppernoise, speckle noise, and impulse noise. But, One of the mainproblems with the median filter is relatively expensive andcomplex to compute. The straightforward implementation ofmedian filter requires 0 (r2 log r) operations per pixel to sortthe array of (2r + 1)(2r + 1) pixels in a window. Fortunately,a fast algorithm is presented, which reduces the complexity toO(r) [15].

2) Min/Max filters: A max(or min) filter outputs a max­imal(or minimal) pixel value from its rectangular window.A straightforward implementation requires O(r2 ) operationsper pixel. Fast implementations of ID running max filterare described in [16]. A simple and fast algorithm calledMAXLINE2 is using a circular buffer of delayed input ele­ments. This algorithm works very fast on i.i.d. (independentidentically distributed) data, but has a worst-case complexityof O(r) for a monotonically decreasing data. An algorithmwith a better worst-case complexity (although with a worse

1866

TABLE I

GAUSSIAN COEFFICIENTS OF a == 1 AND a == V2.

sigma1D Gaussian coefficients

Caeffl Caef f2 Caeff3 Caeff4 Caeff5 CaefIe Caeff70- = 1 0.004468 0.054028 0.242011 0.398985 0.242011 0.054028 0.004468

0- = V2 0.031373 0.105418 0.221339 0.283739 0.221339 0.105418 0.031373

TABLE II

GAUSSIAN COEFFICIENTS OF a == 2AND a == 2V2.

sigma1D Gaussian coefficients

Caeffl Caef f2 Coe j j-; Caeff4 Caeff5 Caef [e Caeff7 Caeff8 Caeff9

0-=2 0.029548 0.067312 0.123539 0.178587 0.202026 0.178587 0.123539 0.067312 0.029548

0- = 2V2 0.064081 0.092560 0.122042 0.144696 0.153242 0.144696 0.122042 0.092560 0.064081

Using equation 15 and 16, Gaussian coefficients of equa­tion 14 is corrected. One-dimensional Gaussian kernel hasa symmetric property. Thus, in according to the order ofEx..Coe]f s( a, k), integer coefficients can be normalized byadding 1 at each term in the kernel symmetrically until eo- isO. Equation 17 represents a look-up table for convolution oftwo passes using a coefficient values.

B. Design of Look-Up Table

Gaussian blur are used in the computation of high-pass andband-pass filters. High-pass filters are obtained by subtractinga blurred result from the original image, and band-pass filtersare obtained by subtracting the blurred images of two differentsigma. We can obtain the Difference of Gaussian as calculatingthese process. Lowe [1] in equation 1 and 2 introducedthat DoG pyramid was obtained by the difference betweenGaussian convolution with initial sigma value and Gaussianconvolution with k scaled sigma. Gaussian blur computation isslowed depending on the sigma's growing. Thus, for reducingcomputation time, equation 13 is written as follows:

N

eo- ==p- LLuTCaeffs(eY,k)k=O

(16)

When eYk is increased like Sk={ -j2, 2, 2-j2, 4, 4-j2...}, eYewill be like Se={l, -j2, 2, 2-j2, ...}. In this study, we createthe coefficients LUT using Se.

where coef f .peakoalue denotes the max integer coefficientafter scaling, pixelcpeaknoluc is the max pixel value of im­age, and p is a scaling factor for converting floating coefficientsto integer value. Two step passes are required for applying theproposed method using two LUTs. Two LUTs are used in thehorizontal and vertical direction as follows:

G(eY~) == G(eY~) * G(eY~_l)

G(eY~) *G(eY~_l) == G(eY~ + eY~-l)

eY~ == eY~ + eY~-l

(To = J(T~ - (TLI (13)

LuTcanv(n, m) n x m (17)

{0::; n < coef f .peakcolueo ::; m ::; pixei.peakualsu: x p

Ex.Coej j sio, k) == LuTcaeffs(eY, k) - p X g~D [k] (15)

(19)

(18)

L LuTcanv (LuTcaef fs (eYk' p), I(p, j))p=-r

r

( L LuTc anv(LuTc aef fs( eYk, p), B x ( i, p))) / p2p=-r

B~t(i,j) ==

r

where B x and B x y denote the results of applying the LUTin the horizontal and vertical direction, respectively. Usingequation 18 and 19, floating-point computation for imageblur is not needed. These computation process could be alsousefully applied to the VLSI architecture for image bluralgorithm.

(14)LuTCaeffs(eY, k) round(p x g~D [k])

{Ikl::; Noa == {1,-j2,2, ...}

where a denotes the standard deviation of one-dimensionalGaussian filter, and (2No+1) denotes the sampling number. p

is the scale factor to convert gaussian coefficients to integervalues. When converting the scaled Gaussian's floating-pointvalues into the integer values needed for the LUT, the sumof the coefficients will be different from 1 x p. This causes adarkening or brightening of the image. To remedy this, thenormalization process is required. Equation 15 denotes thedifference between floating-point coefficient and integer valuescaled, and equation 16 is the difference between 1 x p andsum of integer coefficients scaled.

1867

(a) (b) (c)

(d) (e) (f)

(g) (h) (i)

Fig. 2. Blurring experiment using conventional two-dimensional Gaussian filter and the proposed method: (a) Original image. (b) shows the result byconventional two-dimensional Gaussian filter with (J" = I. (c) the proposed method with (J" = I. (d) conventional two-dimensional Gaussian filter with (J" = V2.(e) the proposed method with (J" = V2. (f) conventional two-dimensional Gaussian filter with (J" = 2. (g) the proposed method with (J" = 2. (h) conventionaltwo-dimensional Gaussian filter with (J" = 2V2. and (i) the proposed method with (J" = 2V2.

TABLE III

EVALUATION OF BL URR ING MET HODS ACCORD ING TO COMPUTING TIME(t m s ) AND ERROR RATE (e) .

where k is the standard deviation. M and N denote a imagesize, and (i, j) is the pixel coordinate in a image. 'Pij and 'r]ij

denote the pixel values of the blurred image by conventionaltwo-dimensional Gaussian blur and other blurring methods,respectively.

(20)MxN

Blurring 0- =1 0- = V2 0- =2 0- = 2V2methods t-«, e t -«, e t-«, e t rn s e

Conventional 20 Gaussian filter 47 0 78 0 172 0 297 0

Recursive Gaussian filter 12 14.9 12 13.1 12 11.08 12 9.0

Stack-blur 16(r = 3) 0.81 18(r = 4) 1.02 21(r = 6) 0.75 31(r = 9) 0.65

Romain Guy's Method 20(r = 1, x2) 1.5 30(r = V2, x 3) 1.04 20(r = 2, x2) 0.92 30(r = 2V2, x 3) 1.49

Box filter (x 6) 42(r = 3) 0.9 46(r = 5) 0.9 52(r = 7) 1.0 60(r = 9) 1.2

The proposed method 7 0.2 7 0.3 8 0.5 8 0.8

IV. C OM PUTATI ONAL R ESULTS AND DISCUSSION

The approach of the proposed method were tested using320 x240 gray images on a Workstatio n computer of a 3.2GhXeon(TM) CPU and IGBytes main memory. The evaluation ofblurring methods is performed according to the computationtime and similarity with the blurred image by conventionaltwo-dimensional Gaussian filter.

1868

In the Figure 2, we show the blurred image by conventionalGaussian filter and the proposed method as changing thesigma from 1 to 2-j2. And, table 3 represents the comparisonof the computing time and similarity using other blurringmethods. As expected, implementation the conventional two­dimensional Gaussian filter was relatively slow. RecursiveGaussian filter was about 2 or 3 times as fast. Computationtime of Stack-blur and box filter is independent of filterradius(r). Romain Guy's method and box filter also requirerepetition accomplishment for similarity with Gaussian blur.The best results were obtained with the proposed result ap­proach, which was almost 7 or 30 times as fast as conventionalGaussian blur. Also, according to the similarity of the blurredimage, the proposed method is superior to other methods. It isclear that the proposed method performs very well comparedto other methods.

v. CONCLUSION

In this paper, we have introduced a fast and accurate methodfor image blurring. The method used a look-up table whichwas created from coefficients one-dimensional Gaussian fortwo-dimensional Gaussian convolution. The LUT played keyrole to omit the floating- point computation for gaussianconvolution. We demonstrated that the proposed method had agood performance for image blurring according to computingtime and similarity as comparing to other blurring methodssuch as conventional Gaussian filter, recursive gaussian filter,Stack-blur method, box filter, and Romain Guy's Method.Based on these computation results, the proposed method canbe also effectively utilized to create the DoG pyramid forfeature extraction or other image processing techniques.

REFERENCES

[1] D. G. Lowe, "Distinctive image features from scale invariant features,"International Journal of Computer Vision, vol. 60, no. 2, pp. 91-110,2004.

[2] Herbert Bay, Andreas Ess, Tinne Tuytelaars, and Luc Van Gool, "SURF:speeded up robust features," Computer Vision and Image Understanding,vol. 110, no. 3, pp. 346-359, 2008.

[3] I. T. Young and L. J. van Vliet, "Recursive implementation of the gaussianfilter," Signal Processing, vol. 44, pp. 139-151, 1995.

[4] L. J. van Vliet, I. T. Young and P. W. Verbeek, "Recursive gaussianderivative filters," Proc. of the Int. Conf. on Pattern Recognition, vol. 1,pp. 509-514, 1998.

[5] S. Perri, M. Lanuzza, P. Corsonello and G. Cocornullo, "SIMD 2-Dconvolver for fast FPGA-based image and processors," Proc. of theMilitary and Aerospace Programmable Logic Devices (MAPLD) Int.Conf., Washington, 2003.

[6] E. Alex, "Four tricks for fast blurring in software and hardware", [Online]Available : http://www.gamasutra.com/features/20010209/evans_01.htm.2001.

[7] M. Klingemann, Stack-blur, [Online] Available: http://incubator.quasimondo.com, 2004.

[8] Romain Guy, [Online] Available: http://www.progx.org, 2005.[9] Duran Emmanuel, [Online] Available: http://www.koders.com/delphi/

fid5AC677DA9310032F9D061DC4D148CFAC158EB37D.aspx?s=algorithm, 2004.

[10] P. L. Chu, "Fast Gaussian Noise Generator," IEEE Trans. on Acoustics.Speech. and Signal Processing, vol. 37, no. 10, pp. 1593-1597, 1989.

[11] M. S. Mohammed, S. L. Matilia, L. Nozal, "Fast 2D convolution filterbased on look up table FFT," Proceedings of the IEEE Int. Symp. onIndustrial Electronics, pp. 446-449, 1992.

[12] M. Mese, P. P. Vaidyanathan, "Look up table (LUT) inverse halftoning,"IEEE Trans. on Image Processing, vol. 10, pp. 1566-1578, 2001.

1869

[13] X. P. Zhang, K. Li, and X. F. W, "A Novel Look-Up Table DesignMethod for Data Hiding With Reduction Distortion," IEEE Trans. onCurcuits and Systems for Video Technology, vol. 18, no. 6, pp. 769-776,2008.

[14] Y. M. Koo, B. S. Shin, "An Efficient Point Rendering Using Octree andTexture Lookup," Lecture Note on Computer Science(LNCS 3482), vol.3482, pp. 1187-1196, 2005.

[15] B. Weiss, "Fast Median and Bilateral Filtering," ACM Transactions ofGraphics (proceedings of the ACM SIG-GRAPH'06 Conference), 2006.

[16] M. Brookes, "Algorithms for max and min filters with improved worst­case performance," IEEE Trans. on Circuits and Systems, vol. 47, no. 9,pp. 930-935, Sep 2000.

[17] R. Deriche, "Fast algorithms for low-level vision," IEEE Trans. PatternAnalysis and Machine Intelligence, vol. 12, no. 1, pp 78-87, 1990.