Image Enlargement Using Adaptive Manipulation Interpolation Kernel Based on Local Image Data

I Komang Somawirata, Keiichi Uchimura, Gou Koutaki Human and Environmental Informatics Department,

Computer Science and Electrical Engineering, Kumamoto University Kumamoto,Japan

e-mail: kmgsomawirata@navi.cs.kumamoto-u.ac.jp

Abstract—This paper proposed Adaptive Manipulation Interpolation Kernel (AMIK) based on local image data. The adaptive interpolation kernel used closed loop system. The previous output result fo’(i,j) also be used as reference for the next interpolation calculation. Closed loop applied in the system, if the interpolation system performed between two pixels for horizontal or vertical directions. Furthermore, fo’(i,j) filtered by Three Mean Median Filter (TMMF). That method used to increase image quality. TMMF is the average value of three median nearest pixels in the 3x3 window size. The test results shown good results compared to the Bilinear, Bicubic, Lanczos 2, Lanczos 3 and Pyramid step. PSNR is the quantitative analysis, which used to measure image performance. Our method has the highest PSNR value in the gray and RGB image, which compared to the comparison methods.

Keywords- resolution; enlargement; interpolation; kernel

I. INTRODUCTION The single-frame of image enlargement called image

scaling, image interpolation, image zooming, super-resolution, and so on. Image scaling has been widely researched and developed in various fields such as medical image processing, military, and consumer electronics [1,2,3].

Our research paper is organized as follows: Section 1 introduction. In the introduction contains about magnification objective and a brief description of our methods. Section 2 presents the proposed method, procedure for image enlargement described in detail in this chapter. Section 3 shows the simulation results, and then the conclusion described in section 4.

Image scaling has been developed to produce high resolution images. This research has been done by Jianping Xiao [2]. Their research presented the Adaptive Interpolation Algorithm with implementing the Newton Polynomial. Once the relativity is judged, at the same time each of order difference values provided to the second-order Newton interpolation function. Thus, the adaptive function for image interpolation is concluded according to both relativistic and classical Newton polynomials. In their research [2] is not used close-loop process to maintain stability of interpolation process.

Y. Lin [4] propose the interpolation filter using truncated raised cosine pulses for image resizing. They set the rectangular-truncated raised cosine pulse with roll off factor and truncation length to a good interpolation value result. K. Jensen [5] presented a nonlinear interpolation scheme for still image resolution enhancement. Their algorithm combined from novel edge fitting operator and edges detect based on a source model emphasizing the visual integrity.

My previous research is pyramid step [6]. This technique is to image enlargement in stages until the desired scale value. Pyramid Step having several layers of images. The last layer is the result of the pyramid. Pyramid exists if enlargement values are more than two (S>2), and If S is more than 2, the number of hidden layer image is H=S-2 and enlargement on each layer is S’=S/(S-1)+S’ per layer.

Yun Liang [7] proposes patch wise scaling methods to resize an image to emphasize the important areas and preserve the global visual effect. Their method for rescaling image is based on optimizing the image distance. The image distance is defined based on so-called local bidirectional similarity measurement and smoothness measurement to quantify the quality of resizing outputs.

Zooming image based on Fuzzy-adapted linear interpolation (FALI) algorithm proposed by Hsiang-Chieh Chen [8]. Their fuzzy used to calculate the weight and also considering the local gradient. FALI algorithm used to improve on three scaling methods such as bilinear, bicubic and bicubic splines. The quality image using PSNR measurement has a greater value compared to the three methods.

Barker[9] propose an algorithm to learn a recognition-based prior which can be used to improve the performance of super-resolution. They use bayesian MAP formulation and recognition-based priors. Recognition algorithm is also added to the image reconstruction to improve the performance of the HR algorithm. The algorithm has been tested on two types of image. That is the image of a face image and text.

Problems that often occur in the image scaling technique are the image scaling results less smooth, an increase in the value of the edge and increase the contrast and blurring.

978-1-4673-2193-8/12/$31.00 ©2012 IEEE

474

Adaptive interpolation gives an important role in the process of interpolation. In this research, to solve these problems by combining the adaptive manipulation interpolation based on bilinear kernels and spatial filter. Our method used closed-loop process. In spatial filter, we combine mean filter and median filter. The objective of the enlargement is only obtained the smooth enlargement image without reducing the quality of image enlargement. To determine the performance of adaptive interpolation kernel, we compared our method with common interpolation methods such as Bilinear, Bicubic, Lanczos and Pyramid which using PSNR measurement.

II. THE PROPOSED METHOD In this research, we proposed a method of digital image

magnification, which combined from the Adaptive Manipulation Interpolation Kernel and the Three Mean Median Filter (TMMF)

A. Interpolation Process In this research, the renewing of conventional bilinear

interpolation is presented. In conventional bilinear using the four reference points, which sourced from the pixels of the image source fs(x,y). We combine the pixels image source fs(x,y) with data interpolation results in the previous coordinates. Also, we provide a window fw(n1,n2) with size of 2x2, as illustrated on Fig.1.

fw(n1,n2) window has three conditions, as shown in Fig.1. Figure 1, the point “P” is the pixel to be interpolated. The first condition as illustrated in Fig. 1(a), if the point “P” is close to the four pixels in the sources image fs(x,y), then the fourth pixels are to be filled in the window fw(n1,n2), as in the Fig.1(a).

The second condition on Fig.1(b), If “P” is between of the two pixels in horizontal direction, then the window fw(n1,n2) filled with the two values from the source image pixels and two pixels from the previous interpolation results.

The third conditions in Fig.1(c), If “P” is between of the two pixels in vertical direction, then the window fw(n1,n2) filled by the two pixels of the source image and two pixels from the interpolation results which is nearest to “P” point. After, we got the matrix window fw(n1,n2). Then, the process continued with the calculation of the weights. Wherein, eq. (1) is the image size of the enlargement result.

Axy=R.S (1)

Denotes: “Axy” is the new image size “R” is height and width of the source image “S” is the scale factor.

The interpolation results fo’(i,j) have two conditions. The first condition, if the “A” divided to “S” produces an integer value, then on this coordinate filled with original pixels. The second condition, if the “A” divided to “S”, the result is fractional value. Then, the value of pixel “P” calculated from the Adaptive Manipulation Interpolation Kernel. The mathematical formulation in this process as shown in eq.(2).

(a) P interpolated in center

(b) P interpolated in horizontal direction

(c) P interpolated in horizontal direction

Figure 1. The Filling Process of the Window fw(n1,n2)

In the eq.(3) is used to find the value of “P”. Wherein, window fw(n1,n2) is one of three conditions on Fig.1.

⎪⎩

⎪⎨

⎧

= valuefracional is

integer is ),(),('

SAifP

SAifyxfs

jifo (2)

∑∑= =

=2

121

2

121

2 1

),(),(n n

nnnnnfwP μ (3)

2221

121121 ),(

nnnn

nnnμμμμ

μ = (4)

The first condition uses the fw(n1,n2) data which obtained from Fig.1(a). Meanwhile, the calculation of μn(n1,n2) matrix elements using eqs.(5) to (8). In this case μn(n1,n2) is equal to μa(n1,n2). While, the symbol ↑ is mean rounding up.

))((11 SA

SA

SA

SA

a yyxx −↑−↑=μ (5)

475

)1)((12 SA

SA

SA

SA

a yyxx −↑−−↑=μ (6)

))(1(21 S

A

S

A

SA

SA

a yyxx −↑−↑−=μ (7)

)1)(1(22 S

A

S

A

SA

SA

a yyxx −↑−−↑−=μ (8)

Then,

∑∑= =

=2

121

2

121

2 1

),(),(),('n n

nnannfwyxfo μ (9)

The second condition uses the fw(n1,n2) data which obtained from the Fig.1(b). Meanwhile, the calculation of μn(n1,n2) matrix elements using eqs. (10) to (13). In this case μn(n1,n2) is equal to μb(n1,n2).

))((11 S

A

S

A

SA

SA

b yyxx −↑−↑=μ (10)

)1)((12 S

A

S

A

SA

SA

b yyxx −↑−−↑=μ (11)

5.021 =bμ (12)

5.022 =bμ (13) Then,

∑∑= =

=2

121

2

121

2 1

),(),(),('n n

nnbnnfwyxfo μ (14)

The third condition uses the fw(n1,n2) data which obtained from the Fig.1(c). Meanwhile, the calculation of μn(n1,n2) matrix elements using eqs. (15) to (18). In this case μn(n1,n2) is equal to μc(n1,n2).

))((11 S

A

S

A

SA

SA

c yyxx −↑−↑=μ 15)

5.012 =cμ (16)

))(1(21 S

A

S

A

SA

SA

c yyxx −↑−↑−=μ (17)

5.022 =cμ (18)

Then,

∑∑= =

=2

121

2

121

2 1

),(),(),('n n

nncnnfwyxfo μ (19)

B. Three Mean Median Filter (TMFF) The pixel interpolation filtered by Three Mean Median

Filter (TMMF). We use 3x3 window size in the TMMF. The TMMF is the average value of three median nearest pixels in the 3x3 window size. That method can be formulated as in eq.(20).

Figure 2. Flowchart System

TMMF = Mean([ws(c-1) ws(c) ws(c+1)]) (20)

Where,

w = [fo’(x-k,y-k), fo’(x-k,y) ,…,fo’(x+k,y+k)] (21)

wws gh ↓= (22)

The matrix “w” is the data sample with the M×M window size. The value of “M” is 2k+1 with “k” equal to 1 in this paper. The “c” value is the rounding up from the division

results of length matrix “w” divided by 2. Notation of wgh ↓ is

the sequencing element value of matrix w. “g” is the minimum value of the element matrix “w”. “h” is the maximum value of the element matrix “w”.

Finally, we write the fo(i,j) output equation. That completed with TMMF and illustrated as on the eq.(23).

476

(a) Low Resolution Lena

(b) Bilinear Interpolation

(c) Bicubic Interpolation

(d) Lanczos 2

(e) Lanczos 3

(f) Pyramid Step

(g) Adaptive Manipulation Interpolation Kernel

Figure 3. The Comparison Result of Image Enlargement on 8 bits Lena by S=4

Figure 4. PSNR values for Lena gray image compared to the proposed method and the other methods.

Figure 5. PSNR values for Boat gray image compared to the proposed method and the other methods.

Figure 6. PSNR values for Lena RGB image compared to the proposed method and the other methods.

Figure 7. PSNR values for Pepper RGB image compared to the proposed method and the other methods.

477

⎪⎩

⎪⎨

⎧

= real is ))((

integer is ),(),('

SAifi,jfo'TMMFSAifyxfs

jifo (23)

C. Flowchart The flowchart in Fig.2 illustrates the process of image

enlargement. The input image is Low Resolution. The image enlargement uses the scale factor greater than 1 (S>1). After, we gotten the value of “S”, then the interpolation process continued by the Adaptive Manipulation Interpolation Kernel method. The quality improvement of image enlargement by TMMF method only used for interpolation pixels.

III. EXPERIMENTAL RESULT Figure 3 is the cropped image magnification of the Lena 8

bit by “S” equal to 4. Lanczos method in Figs.3(d) and (e) respectively the rough result and has a steep edge.

The Bilinear and the Bicubic on Figs.3 (b) and (c) has finer result. If the both methods compared to the Lanczos method. Our method in Fig.3(g) has enlargement image result more subtle, which has a contrast value not excessive.

Analyzing the enlarged image by observation must be done thoroughly and carefully. Image observation analysis is very difficult to do, so the quantitative analysis is necessary to be conducted. Furthermore, we use the quantitative analysis for several images such as 8 bit Boat image, 8-bit and 24-bit Lena image and 24-bit Peppers image. The PSNR result presented on the bar graph, which shown on Fig.4 until Fig.7. The bar graph makes easier to analyze the enlargement image quality on different scale factors “S” ranging from 2 to 5.

PSNR is the quantitative analysis, which used to measure image performance. Equation (24) used to calculate PSNR especially for gray image, and eq. (25) for RGB image. For the analysis using the PSNR, the size of image matrix reference and the enlargement image size must be in the same dimensions and the same matrix size. So that, we change the size of the image reference fR(i,j) to be a smaller size, with the size of ½, 1/3 , ¼ and 1/5. After that, we change the size of a small image to be the same size of the image reference fR(i,j) by a scale factor equal to 2,3, 4 and 5.

MSE

jifMaxPSNR R ),((

log10 10= (24)

RGB

RRGB MSE

jifMaxPSNR

),((log10 10= (25)

On the PSNR analysis results, our method has good results in the gray image and RGB image. Bar graph in Fig.4 to Fig.7 shows the greater the value of “S” affected the decrease of

PSNR value. It applies for all methods. Our proposed method and pyramid method have a higher PSNR value for each value of the scale factor “S”, compared to the Bilinear, Bicubic and Lanczos.

IV. CONCLUSION The Adaptive Manipulation Interpolation Kernel methods

combined with TMMF. It is capable to improving the result of enlargement image quality. On the bargraph, our method has the largest PSNR value on each scale factor S compared to the comparison methods, either on the 8-bit gray colour image and 24-bits RGB images.

For the next research, we will improve the performance of our method by including the learning base, so that becomes to the excellent algorithm for image enlargement.

ACKNOWLEDGMENT The author would like to thank the Directorate General of

Higher Education Ministry of National Education (DGHE) of Indonesia which providing scholarship.

REFERENCES [1] [1] P. Thevenaz, T. Blu, and M. Unser, “Interpolation r evisited”,

IEEE Trans. Medical Imaging, U.S.A, pp. 739 – 758, July 2000 [2] [2] Jianping Xiao, XuechengZou, Zhenglin Liu, and XuGuo,”

Adaptive Interpolation Algorithm for Real-time Image Resizing”, Proceedings of the First International Conference on Innovative Computing, Information and Control (ICICIC'06), 2006.

[3] [3] Hideaki Kawano, NoriakiSuetake, Byungki Cha and Takashi Aso, “Sharpness preserving image enlargement by using self-decomposed codebook and Mahalanobis distance” Image and Vision Computing 27,pp684–693, 2009

[4] [4] Y. Lin, H. H. Chen, and Z. H. Jiang, etc., “Image Resizing with Raised Cosine Pulses”, Proceedings of 2004 International Symposium on Intelligent Signal Processing and Communication Systems, U.S.A, pp. 581-585, November 2004.

[5] [5] K. Jensen, and D. Anastassiou, “Subpixel edge localization and the interpolation of still images”, IEEE Trans. Image Processing, U.S.A, pp. 285-295, March 1995

[6] [6] I Komang Somawirata, Keiichi Uchimura, and Gou Koutaki: “Enlargement Digital Image by Pyramid Steps Algorithm With Local Image Data” FCV2012 Proceeding of the Eighteenth Korea-Japan Jjoin Workshop on Frontiers of Computer Vision, pp. 153-159, 2012.

[7] [7] Yun Liang, ZhuoSu, and XiaonanLuo : “Patchwise scaling method for content-aware image resizing ”, Signal Processing, Vol.92, pp.1243-1257, 2012

[8] [8] Hsiang-ChiehChen,and Wen-JuneWang: "Fuzzy-adapted linear interpolation algorithm for image zooming", Signal Processing, Vol.89, pp.2490-2502, 2009

[9] Simon Baker and Takeo Kanade: “Limits on Super-Resolution and How to Break Them”, IEEE Conference On Computer Vision and Pattern Recognition, 2000.

478