The 19th Korea -Japan Joint Workshop on Frontiers of Computer Vision

Single Image Enlargement Based on KernelEstimation and Linear Weighting

I Komang Somawirata, Keiichi Uchimura and Gou KoutakiComputer Science and Electrical Engineering

Graduate School of Science and Technology, Kumamoto University2-39-1 Kurokami Kumamoto, 860-8555 Japan

Email: kmgsomawirata@navi.cs.kumamoto-u.ac.jp

Abstract-This paper proposes a method for single image en­largement with linear weighting techniques and kernel estimation.The aims of our technique are to reduce the distance of thepixel value too far especially for interpolation pixel. Contributionfrom the closest pixels to the interpolation point is unchangedmeanwhile the farthest pixel contribution will be estimated. Thereare four pixel contributions as a determinant of the interpolationpixel value. The four pixels are placed in a 2x2 kernel matrix.Each pixel in the kernel has a weighting value. The weight value isorganized in the separate places that are placed on the weightingmatrix with 2 x 2 sizes. Weight values obtained from the linearcurve based on the position of the pixel interpolation. Improvingthe image quality is performed only on the interpolation pixels.Experimental results show our new method can produce a betterenlargement result, especially in the edge image regions comparedto the comparison methods that used in this paper.

I. INTRODUCTION

Technique of image upscaling to resulting larger imagesizes or a technique that produces a high-resolution image(HR) is called image enlargement. Image enlargement can bedone in two ways, namely enlargement of the image using thehardware and software. Enlarged image using the hardwareneeds high cost compared to the enlargement of the imageusing the software.

Many methods have been developed to produce a largerimage size or high-resolution images. Enlarged image grad­ually done to produce a better image quality, this techniqueis known as the Pyramid Steps (PS) [1]. The PS techniquerequires a long process, because a magnification of the imageis conducted gradually using weighting scaling technique.Weighting scales technique using four reference pixels to getone pixel interpolation. The four of the reference pixels are ob­tained from the low-resolution (LR) images. Weighting scalestechnique was then developed in the window kernel section.The fourth element of the window is filled with a combinationof LR image pixels and interpolation pixel. The combinationof LR image pixels and interpolation pixel methods is calledby the manipulation adaptive interpolation kernel (AMIK) [2].AMIK method provides three kinds of kernel combination.The first combination kernel is to interpolation pixel betweentwo pixels in horizontal direction. The second combinationkernel is to interpolation pixel between two pixels in verticaldirection and the third combination kernel is to interpolationpixel among four pixels.

Yunsang Han [3] proposes the fast and effective imagesuper-resolution algorithm. Its algorithm consists of three

phases. The first phase is to estimate the parameters of theinput image and the enlarged image that comes from downsampling the input image. The second phase is to calculate theparameter adjustment. The third phase images enhancement forreduce the artifact. This method has a good computing time butstill has a lower image quality in the edge region of the image.Reverse Diffusion Interpolation (RDI) [4] is introduced byOlivier Salvado. Partial Differential equation is used to obtainthe value of the highest gradient direction to produce the Ri­verse Diffusion process. RDI method is no filtering in homoge­neous regions, and no extra parameter is needed to characterizean edge. Takeshi Aso [5] proposes image enlargement usinga weighted sum of linear extrapolation (WLE) of the pixelneighboring to the pixel to be estimated. This algorithm is verysimple and suitable for real time processing. WLE algorithmis almost same accuracy compared with classical methodssuch as nearest neighbor, bilinear interpolation and cubicinterpolation. Weisheng Dong [6] proposes an image super­resolution algorithm based on learning various sets. A patch isprocessed and then one set of bases is adaptively selected tocharacterize the local sparse domain. This algorithm is calledby adaptive sparse domain selection (ADS). ADS algorithmuses two adaptive regularization into the sparse representationframework. The first is a set of auto-regressive (AR) modelsand the second is the image non-local self-similarity. Manyiteration processes cause longer computation time. PatrickVandewalle [7] proposes a frequency-domain technique basedon their low-frequency, aliasing-free part. This enlargementtechnique is called frequency-domain approach (FDA). FDAuses bi-cubic interpolation for image reconstruction. The low­frequency aliasing-free part of the image is used to estimateplanar rotation and translation parameters.

From some image enlargement methods that have beendescribed in the previous section. There have classic problemssuch as edge image areas are not good quality, still has therough contour. Generally, these problems arise if the enlarge­ment is more than two times. Therefore, in this paper proposeda new enlargement image method to reduce the damage imageof the edge area. This method uses kernel estimation andweight kernel. After that is continued by image enhancement.

II. PROPOSED METHOD

A. Scaling Technique with Kernel Estimaton

The Scaling technique in this method uses the weightedinterpolation technique. This technique uses four samples pixelthat used to determine an interpolation pixel. The four samples

978-1 -4673 -5621 -3/13/$31.00 02013 IEEE 174

The 19th Korea -Japan Joint Workshop on Frontiers of Computer Vision

up and rounding down of "Px". As well for ayl and ay2 isobtained from rounding up and rounding down of "py". x andy is the coordinate value in the high-resolution (RR) imagefo (x, y), and S is the scale factor of image enlargement.

In this research, we renew element values of i, (i, }) to thefs(i,}). The renewal value of the fs(i,}) elements is adjustedwith the pixel interpolation position. The pixel element in thekernel i, (i,}) which the farthest from the point of interpolationis estimated. Kernel estimation aims to reduce discontinuityedges of the image area. Pixel estimation applied to a pixelthat is located farthest from the interpolation point. Estimationof Kernel element fs(i,}) can be written as on (4) to (7).

Figure 1 shows four of the pixel region position frominterpolation pixel and the position of the kernel i, (i,}) ele­ment that has been changed. The region 1 is for the value(ax2 - Px) ::; 0.5 and ay2 - Py ::; 0.5. The region 2 is for thevalue (ax2 - Px) ::; 0.5 and ay2 - Py > 0.5. The region 3 isfor the value (ax2 - Px) > 0.5 and ay2 - Py ::; 0.5. The region4 is for the value (ax2 - Px) > 0.5 and ay2 - Py > 0.5.

If the fs(i,}) matrix that have been renewed are fs(i,}), thenthe matrix fs(i,}) in each region are as follows:

Fig. 1. Four possibilities for the kernel estimation element1. If interpolation is in the region 1 such as shown in

Fig.1 (a), then fs(i,}) can be written as (4).

A •• Ifs(l,l) fs(1,2) 1

f s (1,J) = f s (2, 1) Us(2,1 )~ is (1,2) ) (4)

2 If interpolation is in the region 2 such as shown inFig.1 (b), then fs(i,}) can be written as (5).

(5)

Fig. 2. Linear weighting curve

3 If interpolation is in the region 3 such as shown inFig.1 (c), then fs(i,})fs(i,}) can be written as (6).

fA (i .) == Ifs(l, 1) (fs ( I ,I )t

f s( I ,2)) 1

s ,J f s (2, 1) f s (2, 2) (6

)

4. If interpolation is in the region 2 such as shown inFig.1 (d), then fs(i,}) can be written as (7).

pixel is placed in the sample pixel kernel fs(i,}) with the size2 x 2.

Ai· _I (fs (2,I )t f s ( I ,2))

f s ( ,J) - f s (2, 1) (7)

yPy == S (3)

Where, the element matrix of fs(i,}) is obtained by aij ==f( axi, ayj), with i=}= 1,2. f( nl, n2) is the low-resolution (LR)image. axl and ax2 is obtained respectively from rounding

The matrix element of sample pixel kernel fs(i,}) is ob­tained from a Low Resolution image through the reversetransformation [1] as in (1).

fs(i,j) == lall al21 (1)a21 a22

xPx == S (2)

Whereas for the weighting matrix is obtained by reversetransformation that is gotten by dividing the "x" and "y"coordinates in the fo(x,y) matrix by the scale factor (S) asin (2) and (3).

Figure 2 shows the weighting value for "x" and "y"coordinates in the interpolation pixels. The determination ofweighting value for "x" and "y" coordinates uses linear curve.Each coordinates "x" and "y" has two opposing linear curvesamong others are uphill curve and downhill curve. By refer­ence in the Fig. 2, then a weighting matrix on the interpolationpixel can be calculated by (8). The values of wti.j) matrixelements always changes, that depending on the position ofinterpolation pixel.

175

The 19th Korea -Japan Joint Workshop on Frontiers of Computer Vision

ax 2 - Px(9)/-LxI == ax 2 - ax l

Px - ax l (10)/-Lx2 == ax 2 - ax l

Wherein:fe(x, y) = The image enhancementN = The number of pixel elementsfI = H-elements without the highest and the lowest element.

The image enlargement is obtained by summing of all themultiplication matrix between fs(i,}) and wti.j), as in (11).

2 2

fo(X,Y) == ~~fs(i,j) x w(i,j) (11)i=1 j=1

w(i,j) == IW

l l W121 (8)W21 W22

Where, Wij == /-Lxi X /-Lyj with i=}=1,2. The weight value of/-LxI is obtained by linear equation as in (9) and /-Lx2 is obtainedby (10). By referring on linear equation in (9) and (10) then/-Lyl and /-Ly2 is obtained.

Method Image Samples (PSNR in dB)Lena Clown Baboon

FDA [7] 19.03 16.00 16.95RDI [4] 23.41 19.29 18.31WLE [5] 24.47 19.67 18.40

Pyramid Step [1] 26.77 22.58 20.13AMIK [2] 27.29 22.71 20.18

Proposed Method 27.35 22.73 20.22

IV. CONCLUSION

III. SIMULATION RESULT AND ANALYSIS

TABLE I. PSNR RESULT ON IMAGE ENLARGEMENT BY S=4

In our simulation, we use the standard image, there areLena, Clown and Baboon. Previously, the simulation wasconducted by changing the sizes of the standard Lena imagesize (512 x 512) be scaled down the four times smaller size(128 x 128). It also applies similarly for the Baboon andClown images. After that the smaller image size is enlargedby a scale factor equal to four. The enlargement image resultsmust be in the same size with the original image size.

The image enlargement result is analysed by Peak Signal­to-Noise Ratio (PSNR). Table 1 shows the comparison ofPSNR results for the scale factor (S) equal to four. We compareour proposed methods with several comparison methods suchas FDA, RDI, WLE, Pyramid Step (PS) and AMIK. Ourproposed method has PSNR value higher than the comparisonmethod for Lena, Clown and Baboon images.

Figure 3 shows an enlarged image result for Lena imagewith four-time scale factor by several comparison methods. OnWLE method, the enlargement image result has very roughtexture compared with other methods. Let's see on the Fig. 3especially the image in the frame on the hat of image Lena.The FDA, RDI and WLE methods have damage on the edgeof the hat Lena image. The edge of the hat image looks roughand there is an excess of high frequency. The excess of highfrequency is marked by white pixels in the image. The PSmethod has smooth image enlargement result compared withFDA, RDI and WLE. However, if the PS method is comparedwith AMIK and our proposed method, then the PS methodmore contrast than AMIK and our proposed method. By visualobservation, our proposed method has slightly better comparedwith AMIK method.

The simulation results on the Clown image in Fig.4 showFDA, RDI and WLE methods did not have the smooth degra­dation pixels, so that the image looks grainy. However, theresults of the Pyramid Step, AMIK and our proposed methodhave the quality image more refined and have a good continuityin edges image. This condition also applies equally for sampleexperiment using image baboon as shown in Fig. 5.

In general, our proposed method has a good result. Imageenlargement on AMIK method has a little more blurred thanthe proposed method. Meanwhile, the enlargement image re­sult for Pyramid steps has little more contrast than the proposedmethod.

This paper presents the new techniques for image enlarge­ment using a linear weighting curves and kernel estimation.The experimental result shows our proposed method has betterresults compared to the comparison methods. The measure­ment of image quality analysis by PSNR shows our proposed

(12)if Px and Py is integerif Px or Py is fractions

fo(x, y) == {fo(x, y)fe(x, y)

B. Image Enhancement

fo(x, y) contain the original pixel value from low­resolution (LR) image and the interpolated pixel. Mathematicalformulation of selection original pixel or an enhancement pixelfor interpolation pixel can be written in (12). In the Equation(12), original pixels from the LR image on fo(x, y) is retained,meanwhile the enhance process is only on interpolation pixels.

The original pixel position in the fo(x, y) matrix can beknown by dividing the value of "x" and "y" coordinates withthe scale factor "S" or by Px and Py. If the result of "Px"

and "py" both are integer, then the fo(x, y) coordinates is theposition of the original pixels. Otherwise, if the result of "Px"

and "py" both or one of them are a fraction, then the fo(x, y)coordinates are the interpolation pixel position.

We use sliding window filter for image enhancement(fe(x, y) ). Window (H) was used 3 x 3 sizes. The matrixelements of the window H are shown on the (13). To reducethe distance of the pixel values are too far, then element of thewindow H that has the highest value and the lowest must beeliminated. Equation (14) shows the highest and lowest valuesof the element window H that has been eliminated.

Image output (fo(x,y)) in (12) is the selection result be­tween the original pixel and the enhancement pixel (fe ( x, y)).

fo(x - 1, y - 1) fo(x - 1, y) fo(x - 1, y + 1)

H == fo(x, y - 1) fo(x, y) fo(x, y + 1)

fo(x + 1, y - 1) fo(x + 1, y) fo(x + 1, y + 1)(13)

fI == H ~ (max(H) and min(H)) (14)

1 N A

fe(x, y) = N ~ H(i) (15)i=1

176

The 19th Korea -Japan Joint Workshop on Frontiers of Computer Vision

Fig. 3. Comparison of Lena image enlargement by S=4. The frame in the image is the sample of damage edge area for comparison.

Fig. 4. Comparison of Clown image enlargement by S=4

177

The 19th Korea -Japan Joint Workshop on Frontiers of Computer Vision

Fig. 5. Comparison of Baboon image enlargement by S=4

method has the greatest value compared with comparisonmethods.

Our next research will develop a weighting model forscaling techniques using non-linear curve such as the S curveor Gaussian curve.

ACKNOWLEDGMENT

The author would like to thank the Directorate General ofHigher Education Ministry of National Education (DGHE) ofIndonesia wich providing scholarship.

REFERENCES

[1] I Komang Somawirata, Keiichi Uchimura, and Gou Koutaki, Enlarge­ment Digital Image by Pyramid Steps Algorithm With Local Image Data,Proceeding of the Eighteenth Korea-Japan Join Workshop on Frontiersof Computer Vision, pp. 153 - 159, 2012.

[2] I Komang Somawirata, Keiichi Uchimura, and Gou Koutaki, ImageEnlargement Using Adaptive Manipulation Interpolation Kernel Basedon Local Image Data, IEEE ICSPCC Hong Kong , pp. 474 - 478,2012.

[3] Yunsang Han and Sangkeun Lee, Single Image Super Resolution basedon Parameter Estimation, Proceeding of the Eighteenth Korea-JapanJjoin Workshop on Frontiers of Computer Vision, pp. 302 - 305, 2012.

[4] Olivier Salvado, Claudia M Hillenbrand and David L Wilson, PartialVolume Reduction by Interpolation with Reverse Diffusion, InternationalJournal of Biomedical Imaging Vol.2006, pp. 1 - 13, 2006.

[5] Takashi Aso, Noriaki Sutake and Takeshi Yamakau, A Fast and AccurateImage Enlargement Algorithm Employing a Weighted Sum of LinearExtrapolations, IEEE Automation Congress, Proceedings. Vol. 18,pp. 251 - 258, 2004.

178

[6] Weisheng Dong, Lei Zhang, Guangming Shi and Xiaolin Wu, ImageDeblurring and Super-resolution by Adaptive Sparse Domain Selectionand Adaptive Regularization, IEEE Trans. on Image Processing, Vol20, pp. 1838 - 1857,2011.

[7] Patrick Vandewalle, Sabine S usstrunk, and Martin Vetterli, A FrequencyDomain Approach to Registration ofAliased Images with Application toSuper-resolution, EURASIP Journal on Applied Signal Processing,Article ID 71459, pp. 1 - 14, 2006.