786 IEEE TRANSACTIONS ON PATTERN ANALYSIS AND...

Projection Operators and MomentInvariants to Image Blurring

Jan Flusser, Senior Member, IEEE, Tom�a�s Suk, Ji�r�ı Boldy�s, and Barbara Zitov�a

Abstract—In this paper we introduce a new theory of blur invariants. Blur invariants are image features which preserve their values if

the image is convolved by a point-spread function (PSF) of a certain class. We present the invariants to convolution with an arbitrary

N-fold symmetric PSF, both in Fourier and image domain. We introduce a notion of a primordial image as a canonical form of all

blur-equivalent images. It is defined in spectral domain by means of projection operators. We prove that the moments of the primordial

image are invariant to blur and we derive recursive formulae for their direct computation without actually constructing the primordial

image. We further prove they form a complete set of invariants and show how to extent their invariance also to translation, rotation and

scaling. We illustrate by simulated and real-data experiments their invariance and recognition power. Potential applications of this

method are wherever one wants to recognize objects on blurred images.

Index Terms—Blurred image, N-fold rotation symmetry, projection operators, image moments, moment invariants, blur invariants, object

recognition

Ç

1 INTRODUCTION

AUTOMATIC object recognition, which is based on invari-ant features, has become an established discipline in

image analysis. Among numerous descriptors used for thispurpose, moments and moment invariants play a veryimportant role and often serve as a reference state-of-the-artmethod for performance evaluation (interested readers canfind a comprehensive survey of moment invariants in [1]).

1.1 Brief History of Moment Invariants

In the long history of moment invariants, one can identify afew milestones that substantially influenced further devel-opment. The first one was in 1962, when Hu [2] employedthe results of the theory of algebraic invariants, which wasthoroughly studied in 19th century by Hilbert [3], andderived his seven famous invariants to rotation of 2Dobjects. This was the date when moment invariants wereintroduced to broader pattern recognition and image proc-essing community. The second landmark dates in 1991when Reiss [4] and Flusser and Suk [5] independently dis-covered and corrected a mistake in so-called FundamentalTheorem and derived first correct sets of moment invariantsto general affine transformation. The third turning pointwas in 1996-98 when Flusser and Suk [6], [7] introduced anew class of moment-based image descriptors which areinvariant to convolution of an image with an arbitrary cen-trosymmetric kernel. They offered another theoretical viewon moment invariants and also opened the door to newapplication areas. For the first time, moment invariants

were able to handle not only geometric distortions of theimages as before but also blurring and filtering in intensitydomain.

1.2 Motivation to Blur Invariants

Assuming the image acquisition time is so short that theblurring factors do not change during the image formationand also assuming that the blurring is of the same kind forall pixels and all colors/gray-levels, we can describe theobserved blurred image gðx; yÞ of a scene fðx; yÞ as a convo-lution

gðx; yÞ ¼ ðf � hÞðx; yÞ; (1)

where the kernel hðx; yÞ stands for the point-spread function(PSF) of the imaging system. The model (1) is a frequentlyused compromise between universality and simplicity — itis general enough to describe many practical situations suchas out-of-focus blur of a flat scene, motion blur of a flat scenein case of translational motion, motion blur of a 3D scenecaused by camera rotation around x or y axis, and mediaturbulence blur. At the same time, its simplicity allows rea-sonable mathematical treatment.

In many cases we do not need to know the whole originalimage the restoration of which may be ill-posed, time con-suming or even impossible; we only need, for instance, tolocalize or recognize some objects on it (typical examplesare matching of a blurred template against a database and afeature-based registration of blurred frames, see Fig. 1). Insuch situations, the knowledge of a certain incomplete butrobust representation of the image is sufficient. However,such a representation should be independent of the imagingsystem and should actually describe those features of theoriginal image, which are not affected by the degradations.We are looking for a functional I that is invariant to the deg-radation (1), i.e.

IðfÞ ¼ Iðf � hÞ (2)

� The authors are with the Institute of Information Theory and Automationof the ASCR, Pod vod�arenskou v�e�z�ı 4,182 08 Praha 8, Czech Republic.E-mail: {flusser, suk, zitova}@utia.cas.cz, [email protected].

Manuscript received 13 Jan. 2013; revised 19 Aug. 2013; accepted 15 Aug.2014. Date of publication 3 Sept. 2014; date of current version 3 Mar. 2015.Recommended for acceptance by T. Tuytelaars.For information on obtaining reprints of this article, please send e-mail to:[email protected], and reference the Digital Object Identifier below.Digital Object Identifier no. 10.1109/TPAMI.2014.2353644

786 IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 37, NO. 4, APRIL 2015

0162-8828� 2014 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

must hold for any admissible hðx; yÞ. Descriptors satisfyingthe condition (2) are called blur invariants or convolutioninvariants.

1.3 Current State of the Art

Although the PSF is supposed to be unknown, we still haveto accept certain assumptions about it to find invariants (foran unconstrained PSF no non-trivial blur invariants exist1).In our fundamental paper [7] we supposed that the PSF isbrightness-preserving, i.e.

Z Zhðx; yÞ dxdy ¼ 1;

which is a non-restrictive natural assumption, and that it iscentrosymmetric, which means hðx; yÞ ¼ hð�x;�yÞ. Underthese assumptions, we derived in [7] a system of blur invari-ants which are recursive functions of standard (geometric)moments.

These invariants, along with the centrosymmetryassumption, have been adopted by numerous researchers.They (as well as their equivalent counterparts in Fourierdomain) have become very popular image descriptors andhave found a number of applications, namely in matchingand registration of satellite and aerial images [7], [9], [10],[11], [12], in medical imaging [13], [14], [15], in face recogni-tion on out-of-focus photographs [6], in normalizing blurredimages into canonical forms [16], [17], in blurred digit andcharacter recognition [18], in robot control [19], in imageforgery detection [20], [21], in traffic sign recognition [22],[23], in fish shape-based classification [24], in wood industry[25], [26], in weed recognition [27], in cell recognition [28]and in focus/defocus quantitative measurement [29]. In thelast few years yet another broad application area of blurinvariants has appeared. When performing multichanneldeconvolution and/or superresolution of still images orvideo, registration of blurred low-resolution input frames isalways the necessary preprocessing step (see [30], [31] for astate-of-the-art survey). Having registration methods whichare particularly suitable for blurred images is of greatdemand and the blur invariants are one of the possiblesolutions.

Several authors have further developed the theory ofblur invariants. Since image blurring is in practice often

coupled with spatial transformations of the image, an efforthas been put into developing so-called combined invariantsthat are invariant simultaneously to convolution and to cer-tain transformations of spatial coordinates. Although a fewattempts to construct combined invariants from geometricmoments can be found already in [7], only the introductionof complex moments into the blur invariants and a conse-quent understanding of their behavior under geometrictransformations made this possible in a systematic way.Combined invariants to convolution and to rotation wereintroduced by Zitov�a and Flusser [32], who also reportedtheir successful usage in satellite image registration [33] andin camera motion estimation [34]. Additional invariance toscaling and/or to contrast changes can be achieved by anobvious normalization (see [35]). The first attempt to findthe combined invariants both to convolution and affinetransform was published by Zhang et al. [17], whoemployed an indirect approach using image normalization.Later on, Suk derived combined affine invariants in explicitforms [36]. Their use for aircraft silhouette recognition [37],for sign language recognition [38], for the classification ofwinged insect [39] and for robust digital watermarking [40]was reported. A slightly different approach to the affine-blur invariant matching was presented in [41], where thecombined invariants are constructed in Fourier domain.

The existence of imaging devices providing 3D data,namely in medical imaging, stimulated generalization ofthe blur invariants from 2D into higher dimensions. Boldyset al. first extended blur invariants into n-D [42] and then,for 3D case, they also added invariance to rotation [43].Their latest paper [44] presents a general approach to con-structing blur and affine invariants in arbitrary number ofdimensions. Candocia [45] analyzed in detail the impact ofdiscretization on the n-D blur invariants.

Some authors extended the blur invariants to otherdomains. Ojansivu and Heikkil€a [41], [46] and Tang et al. [47]used equivalent blur-invariant properties of Fourier trans-form phase for image registration and matching. Makaremiand Ahmadi [48], [49] and Galighere and Swamy [50]observed that the blur invariants retain their properties evenif wavelet or Radon transform is applied on the image. TheRadon domain was also used by Xiao et al. [51] in order totransform 2D blur and rotation into 1D blur and a cyclic shift.

Several researchers attempted derivation of blur invari-ants which are functions of orthogonal (OG) momentsrather than of the geometric moments. Legendre moments[52], [53], [54], [55], Zernike moments [56], [57], [58], andChebyshev moments [59] were employed for this purpose.Zuo et al. [60] even combined moment blur invariants andSIFT features [61] into a single vector with weighted compo-nents but without a convincing improvement. However, aswas proved by Kautsky and Flusser [62], moment invariantsin any two different polynomial bases are mutually depen-dent and theoretically equivalent. He showed that, whenknowing invariants from geometric moments, one can eas-ily derive blur invariants in arbitrary polynomial basis.

In all papers quoted above, the invariance property wasconsidered—exactly as in the original paper [7] — only tocentrosymmetric PSF’s. Few authors were apparently awareof this limitation which decreases the discriminationpower (as is discussed later in this paper) and tried to

Fig. 1. Blurred template (a) to be matched against a database (b). A typi-cal situation where the convolution invariants can be employed.

1. An attempt to construct invariants to arbitrary PSF was publishedin [8] but that method was designed in a discrete domain only. It doesnot have a continuous-domain counterpart in principle. The assump-tion of the PSF symmetry was replaced by the limitation of its supportsize. However the paper [8] lacks a convincing analysis of the recogni-tion power.

FLUSSER ET AL.: PROJECTION OPERATORS AND MOMENT INVARIANTS TO IMAGE BLURRING 787

construct invariants to more specific blurs. Flusser et al.derived invariants to motion blur [63], to axially symmet-ric blur in case of two axes [64], to circularly symmetricblur [65], and to Gaussian blur [1]. Similar motion-blurinvariants were later proposed by Stern et al. [66]. Pengand Jun used the motion blur invariants for weed recog-nition from a camera moving quickly above the field [27]and for classification of wood slices on a moving belt [25](these applications were later enhanced by Flusser et al.[26], [67]). Zhong used the motion blur invariants for rec-ognition of reflections on a waved water surface [68],although the motion-blur model used there is question-able. Some other authors [69], [70] used a least-squaremoment matching of motion blurred and clear imagesinstead of an explicit usage of motion blur invariantswhich might have some advantages in numeric computa-tion.2 Invariants to circularly symmetric blur equivalentto [65] but expressed in terms of Fourier-Mellin momentswere proposed in [71]. Tianxu and Zhang [72] realizedwithout a deeper analysis that the complex moments, oneindex of which is zero, are invariant to Gaussian blur.Xiao et al. [51] seemingly derived invariants to Gaussianblur and rotation but, since he did not employ theparametric Gaussian form explicitly, he actually did notnarrow the class of centrosymmetric PSF’s. Gaussianparametric shape was employed by Zhang et al. [73] whoproposed a blur-invariant similarity measure betweentwo images without deriving blur invariants explicitly.

1.4 Contribution of this Paper: N-fold BlurInvariants

As one can see from the literature review above, blur invari-ants have formed a well-established research area duringlast 15 years with many application-oriented papers pub-lished. However, there has been only a little development ofthe theory since 1996, although the need for a progress onthis field is evident.

All methods reviewed in the previous section eitherassume the knowledge of the parametric form of the blur-ring function (such as motion or Gaussian blur), which istoo restrictive, or suppose centrosymmetric blur withhðx; yÞ ¼ hð�x;�yÞ. This assumption is usually justified inpractice but it is too weak — a vast majority of real blurringfunctions have a higher degree of symmetry. For instancethe PSF of out-of-focus blur is determined by the shape ofthe aperture. As it is formed by the diaphragm blades (com-mon cameras use to have from 5 to 11 straight or slightlycurved blades) it often takes a form similar to a polygon. Ifthe aperture is fully open, then the PSF approaches circular

symmetry hðx; yÞ ¼ hðffiffiffiffiffiffiffiffiffiffiffiffiffiffiffix2 þ y2

pÞ. This case includes the

well-known ideal ”pillbox” out-of-focus model but it is notlimited to it — some objectives may exhibit circular PSF’swhich are far from being constant on a circle and ratherresemble a ring (see Fig. 2 for some real examples of out-of-focus blur). Also the diffraction blur, if present, is given bythe aperture shape. For circular aperture it takes a well-

known form of Airy function, however for a polygonalaperture the corresponding PSF is more complicated If theblur invariants were designed specifically for a particularsymmetry of the blurring PSF, they should exhibit betterperformance than the invariants to centrosymmetric blur.

In this paper, we present a new general theory of blurinvariants with respect to PSF’s havingN-fold rotation sym-metry, for N ranging from one to infinity. The main contri-butions of the paper are the following.

� For any N we present a specific system of blur invar-iants. They are defined equivalently both in imagedomain (based on complex moments) as well as infrequency domain by means of projection operators.We prove that these invariants form a complete set.We introduce a notion of a primordial image as anequivalence class of all images which differ from oneanother by a convolution with an arbitrary N-foldsymmetric PSF.

� We analyze the nullspace and the discriminationpower of each set of the invariants. We demonstratehow they depend on N .

� We demonstrate that the new blur invariants caneasily be made invariant also to translation, rotation,and scaling.

� We show the original centrosymmetric blur invari-ants [7] are just a special case of the new theory forN ¼ 2.

Summarizing, the papers provides the readers with atwo-fold benefit. For theoreticians it gives an insight intothe construction and structure of blur invariants while topractically oriented researchers it offers powerful featuresfor object recognition in explicit forms along with instruc-tions on how to select a proper set for the given task.

Fig. 2. Top row: Examples of the real out-of-focus blur PSF at circularaperture (left), at polygonal aperture formed by the diaphragm blades(middle), and the ring-shaped PSF of a catadioptric objective (right).Bottom row: images degraded by out-of-focus blur. The shape of therespective PSF can be observed as an image of bright points in the out-of-focus background (this effect is in photography called ”bokeh”).3

2. Moment matching is theoretically equivalent to the invariantswhenever the invariants exist but can also be applied in some caseswhen the relation between blurred and clear image moments is knownbut the blurring does not form a group.

3. The photographs in the bottom row of Fig. 2 were provided bycourtesy of Wikipedia (http://en.wikipedia.org/wiki/Bokeh)and PALADIX (http://www.paladix.cz/clanky/bokeh.html).


2 BLUR INVARIANTS FOR N-FOLD SYMMETRIC

PSF’S

In this section we first define basic terms needed along withtheir important properties. Then we define N-fold blurinvariants and discuss their independence, completenessand recognition power.

2.1 Preliminaries on Moments

Definition 1. By an image function (or image) we understandany absolutely integrable real function fðx; yÞ defined on acompact support4 D � R�R and having a nonzero integral.

Definition 2. Let fðx; yÞ be an image function and p; q be twonon-negative integers. Then the functionals

mðfÞpq ¼

Z 1

�1

Z 1

�1xpyqfðx; yÞdxdy (3)

and

cðfÞpq ¼Z 1

�1

Z 1

�1ðxþ iyÞpðx� iyÞqfðx; yÞdxdy; (4)

where i is imaginary unit, are called geometric moment andcomplex moment of order pþ q.

It follows from the definition that only the indices p � q pro-vide independent complex moments because cpq ¼ c�qp (the

asterisk denotes complex conjugate). Both geometric andcomplex moments carry the same amount of informationabout the image. To see this, we express each complexmoment in terms of geometric moments of the same orderas

cpq ¼Xpk¼0

Xqj¼0

p

k

� � q

j

� �ð�1Þq�j � ipþq�k�j �mkþj;pþq�k�j (5)

and vice versa

mpq ¼ 1

2pþqiq

Xpk¼0

Xqj¼0

p

k

� � q

j

� �ð�1Þq�j � ckþj;pþq�k�j: (6)

Characterization of the image by means of complex (as wellas by geometric) moments is complete and unambiguous inthe following sense. The moments of all orders of any imagefunction exist and are finite. The image function can beexactly reconstructed from the set of all its moments.5

We use complex moments in this paper instead of morecommon geometric moments because the complex momentsreflect various symmetries of the image (and of the PSF’s) ina transparent way. This is implied by their favorable behav-ior under image rotation, as will be discussed below. Theuse of complex moments is theoretically not necessary butallows a simple and elegant mathematical treatment of theproblem. Thanks to the equivalence of all polynomial bases,one could derive equivalent blur invariants in terms of

arbitrary moments [62]. However, in all other bases suchderivation would be much more laborious (the earlier invar-iants to centrosymmetric blur [7] were derived as functionsof geometric moments which is practically impossible for ageneralN-fold symmetry).

Lemma 1. Let f 0 be a rotated version (around the origin) of f byan angle a. Let us denote the complex moments of f 0 as c0pq.Then

c0pq ¼ e�iðp�qÞa � cpq: (7)

To prove this lemma, we express the image and itsmoments in polar coordinates ðr; uÞ:

cpq ¼Z 1

0

Z 2p

0

rpþqþ1eiðp�qÞufðr; uÞ drdu: (8)

Since rotation reduces in polar coordinates to a shiftf 0ðr; uÞ ¼ fðr; u þ aÞ, the rest of the proof is straightforward.Eq. (7) says that under rotation the moment magnitude jcpqjis preserved while the phase is shifted by ðp� qÞa (we recallthe clear analogue with the Fourier Shift Theorem).

Another useful property of complex moments is theirsimple transformation if the image is convolved withanother image function.

Lemma 2. Let fðx; yÞ and hðx; yÞ be two arbitrary image func-tions and let gðx; yÞ ¼ ðf � hÞðx; yÞ. Then gðx; yÞ is also animage function and it holds for its moments

cðgÞpq ¼Xpk¼0

Xqj¼0

p

k

� � q

j

� �cðhÞkj c

ðfÞp�k;q�j

for any p and q.

This lemma can be easily proven just using the defini-tions of complex moments and of 2D convolution. It holdsfor geometric moments too in exactly the same form.

For derivation of blur invariants we will employ also theconnection between complex moments and Fourier trans-form. This is however not as straightforward as in the caseof geometric moments. Let us use the traditional definitionof Fourier transform of f :

FðfÞðu; vÞ F ðu; vÞ¼Z 1

�1

Z 1

�1fðx; yÞ � e�2piðuxþvyÞdxdy:

Since f 2 L1, its Fourier transform always exists. Afterexpansion of the exponential function into a power serieswe obtain the well-known formula

F ðu; vÞ ¼X1j¼0

X1k¼0

ð�2piÞjþk

j!k!ujvkmjk; (9)

which tells us that geometric moments of an image are Tay-lor coefficients (up to a constant factor) of its Fourier trans-form. We can find a similar meaning for complex moments,too. Let us make a substitution U ¼ uþ v; V ¼ iðu� vÞ.Then

F ðU; V Þ ¼X1j¼0

X1k¼0

ð�2piÞjþk

j!k!ujvkcjk: (10)

4. Assumption of the compact support could be omitted if we con-sider functions of fast decay, so their moments would be well-definedin R2. However, in practice the images are always of a finite extent.

5. More general moment problem is well known in statistics: can agiven sequence be a set of moments of some compactly-supported func-tion? The answer is yes if the sequence is completely monotonic.


2.2 The Space SN

Now we define the N–fold rotation symmetry (N-FRS), whichis the central term of this paper. Function f is said to haveN-FRS if it is ”rotation periodic”, i.e. if it repeats itself whenit rotates around the origin by aj ¼ 2pj=N for allj ¼ 1; . . . ; N . In polar coordinates this means that

fðr; uÞ ¼ fðr; u þ ajÞ j ¼ 1; . . . ; N:6

Particularly, N ¼ 1 means no symmetry in a commonsense and N ¼ 2 denotes a the central symmetryfð�x;�yÞ ¼ fðx; yÞ. We use this definition not only for finiteN but also for N ¼ 1. Thus, in our terminology, a functionhaving circular symmetry fðr; uÞ ¼ fðrÞ is said to have1-FRS. Later in this paper, the assumption of N–fold rota-tion symmetry will be imposed on the blurring PSF’s.

We denote a set of all functions with N–fold rotationsymmetry as SN . For anyN , the set SN is closed under addi-tion, multiplication and convolution. Considering point-wise addition and convolution, SN forms a (commutative)ring.7 Later on, we will employ particularly the closureproperty of SN with respect to convolution.

In this paper we do not deal explicitly with axial (reflec-tion) symmetry but it is worth mentioning that axial androtational symmetries are related in the following way. If fhas N axes of symmetry then it belongs to SN . On the otherhand, if f 2 SN then it has either none or N symmetry axes.Hence, axial symmetry cannot exist without rotation sym-metry (such compound symmetry is called dihedral symme-try) but the opposite case is possible.

The N–fold rotation symmetry implies vanishing of cer-tain moments. This property allows the existence of blurinvariants.

Lemma 3. If f 2 SN ,N finite, and if ðp� qÞ=N is not an integer,then cpq ¼ 0.

Proof. Let us rotate f around the origin by a ¼ 2p=N . Due toits symmetry,

f 0ðr; uÞ fðr; u þ aÞ ¼ fðr; uÞ:In particular, it must hold c0pq ¼ cpq for any p and q. Onthe other hand, as follows from eq. (7),

c0pq ¼ e�2piðp�qÞ=N � cpq:Since ðp� qÞ=N is assumed not to be an integer, thisequation can be fulfilled only if cpq ¼ 0: tu

Lemma 3a. If f 2 S1 and if p 6¼ q, then cpq ¼ 0.

Proof. The proof is similar to that of Lemma 3 with a beingan arbitrary angle. We get

c0pq ¼ e�iðp�qÞa � cpq ¼ cpq;

which can be fulfilled only if cpq ¼ 0: tu

Note that these two lemmas do not hold the other wayround – an integer ðp� qÞ=N (or p ¼ q) does not necessarilyimply a non-zero cpq.

The space SN is closed also with respect to Fourier trans-form: If f 2 SN then also F 2 SN (if it exists). This followsimmediately from the rotation property of Fourier trans-form. However, note that F is real-valued only for evenN .

Interesting relations hold among the sets SN for variousN . AssumingN can be factorized as

N ¼YLi¼1

Ni;

whereNi ¼ knii and ki are mutually different primes. Then

SN ¼\Li¼1

SNi:

For SNiwe have a nested sequence of the sets

SkiffiSk2iffi � � �ffi Sk

nii SNi

:

Particularly,

S1 ¼\1k¼1

Sk:

2.3 Projection Operators

In this section we introduce projection operators onto SN .These operators decompose any function into its N-foldsymmetric part and ”the rest”, similarly as in 1-D one candecompose any function into an even and an odd parts.

Definition 3. Projection operator PN is for a finiteN defined as

ðPNfÞðr; uÞ ¼ 1

N

XNj¼1

fðr; u þ ajÞ;

where aj ¼ 2pj=N , and

ðP1fÞðrÞ ¼ 1

2p

Z 2p

0

fðr; uÞ du:

Operator PN rotates f repeatedly by 2p=N and calculatesan average. The following properties are valid for any f andN .

� PNf 2 SN (i.e. PN projects f onto SN ).� If f 2 SN then PNf ¼ f and vice versa.� Operator PN is linear:

PNðaf þ gÞ ¼ aPNf þ PNg:

� Operator PN is idempotent, i.e. PNðPNfÞ ¼ PNf .� Any function f can be expressed as f ¼ PNf þ fA,

where fA is its N-fold antisymmetric part.8 Clearly,PNfA ¼ 0.

6. We can also define exactN–fold rotation symmetry whereN is thehighest number with this property. However, such a definition is notvery useful because we lose the closure property of SN .

7. Since the validity of this assertion is intuitive, we skip a formalproof which requires some algebraic manipulations.

8. The word ”antisymmetric” here means ”anything but for N-foldsymmetry”, so fA may include terms with other symmetries as well asasymmetric terms.


� Operator PN commutes with Fourier transform:

FðPNfÞ ¼ PNF:

� Complex moments of a function f are either pre-served or zeroed by the projection operator PN . For afinite N it holds

c PNfð Þpq ¼ cðfÞpq iff ðp� qÞ=N is an integer;

c PNfð Þpq ¼ 0 otherwise:

ForN ¼ 1we get

c P1fð Þpp ¼ cðfÞpp ;

c P1fð Þpq ¼ 0 for p 6¼ q:

The last property follows from the properties of complexmoments of N-fold symmetric functions and will play animportant role later when the invariants will be constructed.

A visual example of the projection operators is shown inFig. 3.

2.4 Definition of N-fold Blur Invariants

Let us consider an image which was blurred according to (1)with an unknown PSF hðx; yÞ which has an N-fold rotationsymmetry.

Now we formulate the central theorem of this paper.

Theorem 1. Let f be an arbitrary image function, then

INðu; vÞ ¼ FðfÞðu; vÞFðPNfÞðu; vÞ

is an N-fold blur invariant, i.e. IðfÞN ¼ I

ðf�hÞN for any N-fold

symmetric hðx; yÞ.

Proof.

Iðf�hÞN ¼ Fðf � hÞ

FðPNðf � hÞÞ ¼F �H

PNðF �HÞ¼ N � F �HPN

j¼1 F ðr; u þ ajÞHðr; u þ ajÞ:

Since H 2 SN , we have Hðr; u þ ajÞ ¼ Hðr; uÞ for anyj ¼ 1; . . . ; N . Consequently,

Iðf�hÞN ¼ F �H

H � PNF¼ F

PNF¼ I

ðfÞN :

tuThe above theorem holds also for N ¼ 1, the proof is

similar.The Fourier-domain blur invariant IN is a ratio of two

functions, that can be expressed by absolutely convergentpower series the coefficients of which are geometricmoments. As we have shown in Section 2.1, after the substi-tution U ¼ uþ v; V ¼ iðu� vÞ we obtain expressions interms of complex moments. Hence, we can express IN as aconvergent power series

INðU; V Þ¼ F

PNFðU; V Þ¼

X1j¼0

X1k¼0

ð�2piÞjþk

j!k!ANðj; kÞujvk:

The coefficients ANðj; kÞ can be easily obtained from theconstraint

X1j¼0

ðj�kÞ=N

X1k¼0

is integer

ð�2piÞjþk

j!k!cðfÞjk u

jvk

0BB@

1CCA

�X1j¼0

X1k¼0

ð�2piÞjþk

j!k!ANðj; kÞujvk

!

¼X1j¼0

X1k¼0

ð�2piÞjþk

j!k!cðfÞjk u

jvk

!:

(11)

The first factor is an expansion of PNF ðU; V Þ. Recalling therelation between the complex moments of a function andthose of its projection derived in Section 2.3, we can seewhy the summation goes over indices with an integervalue of ðj� kÞ=N only — all other complex moments of

PNf are zero, while the non-zero ones equal cðfÞjk . Compar-

ing the coefficients of the same powers we get, after somemanipulation,

ANðp; qÞ ¼cðfÞpq

cðfÞ00

�Xpj¼0

Xqk¼0

p

j

� �q

k

� � cðfÞjk

cðfÞ00

ANðp� j; q � kÞ:

0<jþk

ðj�kÞ=N is integer

(12)

Since IN was proven to be invariant to blur, all coeffi-cients ANðp; qÞ must also be blur invariants. If we further

assume that the blurring is brightness-preserving, i.e. cðhÞ00 ¼

1, then c00 is also blur invariant and by excluding it from

Fig. 3. Performance of the projection operators. (a) original image f, itsprojections (b) P2f, (c) P4f and (d) P1f.


(12) we obtain the final form for moment invariants toN-fold symmetric blur9

KNðp; qÞ ¼ cpq � 1

c00

Xpj¼0

Xqk¼0

p

j

� �q

k

� �KNðp� j; q � kÞ � cjk:

0<jþk

ðj�kÞ=N is integer

(13)

The assumption of overall brightness preservation is notnecessary for construction of blur invariants. However, it ismostly fulfilled in real imaging systems and we also keep itbecause of the consistency with [7].

There is another (seemingly different but actually equiv-alent) instructive approach leading to the same result. It isnatural to express problems related to rotation symmetryusing the circular harmonics basis. Applied in the Fourierdomain, where convolution becomes multiplication, it leadsto the following conclusion. The linear space spanned by allthe harmonics can be divided into N independent subspa-ces, where each of them stays invariant under convolutionwith an N-fold symmetric kernel. Thus, it can be expected,that an invariant to N-fold symmetric convolution shouldbe based on circular harmonics from one of these subspaces.Looking for a relative invariant, it is then easy to find theexpressions identical with the invariants defined in Theo-rem 1 by projection operator and with those defined bymoments (13).

Let us discuss some important properties of the blurinvariantsKNðp; qÞ.

� Complex values. The invariants (13) are generallycomplex. If we prefer working with real-valuedinvariants, we should use their real and imaginaryparts separately. On the other hand, for any invari-ant (13)KNðp; qÞ ¼ KNðq; pÞ�, so it is sufficient to con-sider the cases p � q only.

� Zero-order invariant. For any N (finite or infinite) itholds KNð0; 0Þ ¼ c00. This is a ”singular” invariant,the invariance of which comes from the fact that thePSF is assumed to have a unit integral and not, con-trary to all other invariants, from its symmetry.

� First-order invariants. For anyN , invariantKNð1; 0Þ ¼c10 is non-trivial. However, since in practical applica-tions c10 uses to be employed for image normaliza-tion to shift, KNð1; 0Þ becomes useless for imagerecognition.

� Non-symmetric PSF. Invariance property formallyholds also for N ¼ 1 (i.e. PSF without any symmetry)but all invariants except K1ð0; 0Þ ¼ c00 are identicallyzero.

� Factorization of N . Let N be a product of L integers,N ¼ k1k2 � � � kL. Then the PSF has also kn-fold sym-metry for any n ¼ 1; 2; . . . ; L. Thus, any Kknðp; qÞ isalso a blur invariant. However, using these invari-ants together with KNðp; qÞ’s is useless since they areall dependent on KNðp; qÞ’s. (Note that the

dependence here does not mean equality, generallyKNðp; qÞ 6¼ Kknðp; qÞ).

� N ¼ 1. In the case of a circularly symmetric PSF,Theorem 1 obtains much simpler form because thesummation goes over the indices j ¼ k only. Hence,provided that p � q, we get

K1ðp; qÞ ¼ cpq � 1

c00

Xqk¼1

p

k

� � q

k

� �K1ðp� k; q � kÞ � ckk:

Circularly symmetric PSF’s appear in imaging asout-of-focus blur and diffraction blur on a circularaperture, and describe also a long-term atmosphericturbulence blur.

� Relation to earlier work: N ¼ 2. The invariants to cen-trosymmetric blur introduced in [7] are nothing buta particular case of Eq (13). The invariants in [7] haveformally exactly the same form as those in Eq (13)with complex moments replaced by geometricmoments (this ”direct” substitution is possible onlyfor N ¼ 2). To see that both systems are actuallyequivalent, one can substitute from (5).

2.5 The Null-Space and the Discrimination Power ofthe Invariants

Unlike many geometric invariants, the invariants (13) do nothave a straightforward ”physical” interpretation. However,understanding what image properties they reflect is impor-tant for their practical application. Two mutually connectedkey questions pertain to the null-space of the invariants (i.e.to the set of images having, for a given N , all invariantsKNðp; qÞ zero) and to their discrimination power.

First of all, note that certain invariants (13) are alwayszero for any object and are of course completely useless forrecognition. For any N; p; q such that ðp� qÞ=N is integer(except p ¼ q ¼ 0) we get KNðp; qÞ ¼ 0. A formal proof caneasily be done by induction, we present only the core ideahere.

If ðp� qÞ=N is integer and the summation in (13) goesover ðj� kÞ=N integer only, then also ððp� jÞ � ðq � kÞÞ=Nis always an integer. In other words, if ðp� qÞ=N is integer,then on the right-hand side of (13) we have only KN ’s of thesame property. Now we look what always happens withthe highest-order term. Since ðp� qÞ=N is integer, then thelast term in the summation is KNð0; 0Þcpq c00cpq whichcancels with the cpq standing outside the sum. Knowingthat, is easy to prove by induction that all other terms arezero because they contain the KN of orders less than pþ qand of the same nature. The existence of only trivial invari-ants for ðp� qÞ=N being integer is unavoidable. The

moment cðhÞpq of the blurring PSF is non-zero and there is no

way how to eliminate it. This is for instance the reason whyfor a centrosymmetric blur ðN ¼ 2Þ cannot exist valid invar-iants of even orders. On the other hand, for ðN ¼ 1Þ theonly trivial invariants are K1ðp; pÞ’s (for p > 0), all othersare valid invariants.

The number of valid (i.e. non-trivial) invariants up to thegiven order r increases as N increases, reaching its maxi-mum for all N > r (see Table 1). This is in accordance withour intuitive expectation — the more we know about the

9. Thanks to the projection operators, the proof of invariance of IN isvery elegant and overcomes the necessity of proving invariance ofKNðp; qÞ directly. This is of course possible by induction but such proofis long and tedious, see [7] for the caseN ¼ 2 and geometric moments.


PSF (i.e. the less ”degrees of freedom” the PSF has), themore invariants are available. The structure of invariantmatrix Kij ¼ KNði� 1; j� 1Þ is shown in Fig. 4a. There arealways zero-stripes on the main diagonal regardless of Nand also on all minor diagonals for ðp� qÞ=N being an inte-ger. As one can see, the invariant matrix of the image iscomplementary to the moment matrix of the PSF in terms ofzero and non-zero elements (see Fig. 4b). The discriminationpower of the set of the invariants up to the given order thusincreases as N increases. This is also a property we couldexpect—the discrimination power goes always against theinvariance and here the ”maximum invariance” is providedby N ¼ 2.

What is the null-space of the non-trivial blur invariants?Let us denote ZN the joint null-space of all (non-trivial)invariants KNðp; qÞ; p � q; p > 0. For any N it holdsSN � ZN . This is clear because any N-fold symmetric func-tion f can be considered as a blurring PSF acting on a delta-function, which always lies in ZN . Hence, also f 2 ZN . Onthe other hand, if f 2 ZN , we can reconstruct its moments.The reconstruction is ambiguous—the moments with aninteger value of ðp� qÞ=N may be chosen arbitrary10 whilethe moments with a non-integer ðp� qÞ=N are zero. Thus,the reconstructed function is always from SN . Summarizing,we proved an important equivalence

SN ¼ ZN:

This is one of the intrinsic limitations of discriminativepower of the invariants. Any invariant to convolution withan N-fold symmetric PSF cannot distinguish differentN-fold symmetric objects because it gives zero responses onall such images.

Let us conclude this section by a remark concerning theproper choice of N . It is the only user defined parameter ofthe method and its correct estimation might be a trickyproblem. Ideally, it should be deduced from the physicalmodel of the blurring source or from other prior informa-tion. In fact, we assume that in most situations N is givenby the aperture shape and is known. If this is not the case,we may try to estimate N directly from the blurred imageby analyzing its spectral patterns or the response to an idealbright point or a spot of a known shape, if available. If noneof the above is applicable and we overestimate N in orderto have more invariants available then we lose the invari-ance property of some (or even all) of them. On the otherhand, some users might rather underestimate it to be on a

safe side (they might choose for instance N ¼ 2 instead ofcorrect N ¼ 16). This would always lead to the loss of dis-criminability (note that Z2 ffi Z16) and sometimes could also

violate the invariance, depending on the actual and chosenN . Obviously, the worst combination occurs if the true andestimated fold numbers are small coprime integers such as2 and 3 or 3 and 4. If h is not exactly N-fold symmetric, thenthe more the ratio H=PNH differs from a constant 1, themore violated the invariance of theKN ’s.

2.6 Completeness of the Invariants

A crucial question concerns the general discriminationpower—what are the ”equivalence classes” of images hav-ing the same values of all invariants? We may ask an equiv-alent question about the reconstruction possibility.Knowing, for a given N , all invariants KNðp; qÞ (or, equiva-lently, IN ), what image can we reconstruct? What are thedegrees of freedom of this reconstruction? The previoussection gave us the answer for the case of KNðp; qÞ ¼ 0, buthow is it in a general case? Apparently, any shape descrip-tor invariant to a certain group of transformations cannot,in principle, distinguish objects that differ from one anotheronly by transformations from this group. Complete descrip-tors are able to distinguish all other cases, incompletedescriptors do not have this ability. Below we demonstratethe completeness of the blur invariants.

The frequency domain provides us with a goodinsight. IN is a ratio of two Fourier transforms which maybe interpreted as a deconvolution. Having an image f , weseemingly ”deconvolve” it by the kernel PNf . This”deconvolution” exactly eliminates the symmetric part off (more precisely, it transfers PNf to d-function) and actson the antisymmetric part:

IN ¼ FðfÞFðPNfÞ ¼

PNF þ FðfAÞPNF

¼ 1þ FðfAÞPNF

1þCN:

Hence, IN can be viewed as a Fourier transform of a primor-dial image (although such an image may not exist in a com-mon sense) and then KNðp; qÞ are its complex moments. Inother words, this is a kind of normalization. We seeminglycalculate a blind deconvolution with an N-fold symmetrickernel which is chosen in such a way that it eliminates theN-fold symmetric component of the image. Hence, the pri-mordial image, which plays the role of a canonical form of

TABLE 1The Number of Non-Trivial N-Fold Blur Invariants

Up to the Order r

N r 1 2 3 4 5 6

2 1 1 3 3 6 63 1 2 3 5 7 94 1 2 4 5 8 105 1 2 4 6 8 116 1 2 4 6 9 117 –1 1 2 4 6 9 12

Fig. 4. The structure (a) of the invariant matrix and (b) of the PSFmoment matrix. The gray elements are zero for any f and h. The whiteelements stand for non-trivial invariants in (a) and non-zero moments ofthe PSF in (b). Note the complementarity of both matrices (except theð0; 0Þ element).

10. Provided they satisfy complete monotonicity constraint.


f , is the ”maximally deconvolved” antisymmetric part.Moments of the primordial image are obviously invariantsto N-fold convolution and the trick is that we are able to cal-culate these moments directly using (13) without actuallyperforming the deconvolution. Now we can again see whythose KNðp; qÞ where ðp� qÞ=N is integer must be trivial.Note that the primordial image depends only on the anti-symmetric part of the image. Hence, KNðp; qÞ can be alsoviewed as a measure of antisymmetry of f . This corre-sponds to our intuitive expectation that the antisymmetricpart can be modified by a symmetric convolution kernelonly such that certain significant features are always pre-served while the symmetric part may be changed to anyother symmetric function.

Taking this into consideration, we can understand thelimitation of the recognition power: two images having thesame primordial image cannot be distinguished. This con-clusion is incorporated into the following theorem, which atthe same time says that in all other cases the images are dis-tinguishable, i.e. the blur invariants are complete.

Theorem 2. Let f and g be arbitrary image functions andN be aninteger orN ¼ 1. Then

KNðp; qÞðfÞ ¼ KNðp; qÞðgÞ p; q ¼ 0; 1; . . .

if and only if there exist functions h1 and h2 from SN such that

f � h1 ¼ g � h2:

Proof. The backward implication follows directly from theinvariance property. To prove the forward implication,let us realize that if all invariants equal then I

ðfÞN ¼ I

ðgÞN

which means

F

PNF¼ G

PNG

and, consequently,

f � PNg ¼ g � PNf:

Since both PNf and PNg are from SN regardless of f andg, the proof is completed. tuNote that the completeness is guaranteed only if an infi-

nite set of invariants for all p; q is used. In practice wealways work with a finite (sometimes very small) subset, sothe actual discriminability is influenced by this factor.

2.7 Combined Invariants

In practice, image blurring is often coupled with spatialtransformations of the image. To handle these situations,having combined invariants that are invariant simultaneouslyto convolution and to certain transformations of spatialcoordinates is of great demand.

Shift invariance is achieved easily by using centralmoments and scale invariance is provided by normalizingeach invariant KNðp; qÞ by c

ðpþqÞ=2þ100 , which is equivalent to

the use of normalized complex moments in (13) (note thatthe invariance to convolution is not violated by normaliza-tion because c00 itself is a convolution invariant for anyN).

Achieving rotation invariance is harder but still can beaccomplished in an elegant way. Note that the N-fold sym-metry of the blurring PSF is not violated by an image rota-tion (applied either before or after the blurring), so it makessense to look for combined invariants. When investigatingthe behavior of the invariants KNðp; qÞ under rotation, weobserve that they change in the same way as the complexmoments themselves, i.e.

K0Nðp; qÞ ¼ e�iðp�qÞa �KNðp; qÞ: (14)

Hence, the simplest way is to take the magnitudes jKNðp; qÞjwhich provide combined invariants but create only anincomplete system. A more sophisticated method of creat-ing a complete set of combined invariants is based on phasecancellation by multiplication of proper invariants. It isvery similar to the construction of pure rotation invariantsfrom the complex moments [74] (including selection of theindependent set and symmetry problems) where we justreplace the moments by the invariants.

3 IMPLEMENTATION AND PRACTICAL ISSUES

In this section we answer several questions that could beraised by anyone who is thinking about the implementationof the convolution invariants.

� How many invariants shall we use in practice? There isno general answer and it is not true that the more thebetter. The noise sensitivity, complexity and othernumerical problems increase with an increasingorder (this is a well-known property of momentswhich we do not investigate here; the noise sensitiv-ity of convolution invariants for N ¼ 2 was studiedin [7] and it is the same for other N). On the otherhand, low-order invariants often do not provide suf-ficient discriminability. The ”optimal” number ofinvariants always depends on the data and shouldbe found by a feature selection procedure whichoptimizes the separability (measured for instance byMahalanobis distance) of a particular training set/database.

� Shall we preferably use Fourier-domain or moment-basedinvariants? Although theoretically equivalent, theymay behave differently. In most applications theinvariants are typically required for relatively smallimages or templates and a few of them are sufficient.In such a case moment-based invariants are the pre-ferred choice. If the image in question is large (sev-eral hundreds or thousands of pixels in onedimension) then Fourier domain might be betterbecause the moment values may overflow. On theother hand, when constructing IN , we possiblydivide by very small numbers which requires certaincare. High frequencies of IN are sensitive to noise soit is better to suppress them by a low-pass filter.

Another reason why we prefer constructing com-bined invariants from moments in the image space isthat we usually need only a few invariants and it isnot necessary to calculate all of them.

� What shall we do if the images to be compared are blurredwith PSF’s having different number of folds (say N1 and


N2)? To ensure invariance to both and the best possi-ble discriminability, we choose N as the greatestcommon divisor of N1 and N2. If N1 and N2 arecoprime, the task is not correctly solvable. In such acase we choose either N1 or N2 depending on whichblur is more severe.

� How to avoid over/underflow of moment values? If theimage size is large, we may face overflow (if a pixelmeans 1 on a coordinate axes) or underflow (if thewhole image is mapped into ð0; 1Þ � ð0; 1Þ domain).A prevention of this is using orthogonal polynomialsand OG moments instead of the power basis andcomplex moments. The range of values of OG poly-nomials uses to be small so the OG moments arekept within a reasonable interval. Kautsky andFlusser [62] shows that there is no need to re-builtthe theory of convolution invariants when changingthe basis; the formulas for convolution invariants inany polynomial basis can be obtained from (13) bysimple matrix operations.

� How to eliminate different dynamic range of differentinvariants? If the dynamic range of the invariants issignificantly different, then in terms of Euclideanmetric the ones with high range are preferred whichis not desirable. One way to suppress this is usingOG moments as described above. Another (simpler)possibility is the value normalization into the samerange or to the same standard deviation or, equiva-lently, using weighted Euclidean norm. All these sol-utions might be misleading when the different rangeactually reflects a different significance which ishard to recognize in advance. Since the scale-normal-ized versions of the invariants have much less range,scale normalization is often a good choice even if theimages are not spatially scaled.

� How to handle the boundary effect? For discrete imagesof a finite support and non-zero background, theboundary effect limits the usability of the invariants.The convolution model is violated, the boundarystripes are affected also by pixels laying in the sceneoutside the visual field and the convolution invari-ants are no longer truly invariant. The influence ofthe boundary effect on template matching was stud-ied thoroughly in [7] for N ¼ 2 and in [1] also formotion blur. Since it is the same for any N , we donot repeat experiments of this kind here. It wasshown that boundary effect cannot be removed butif the blur size is less than 15% of the template sizemost templates are still recognized correctly.

� What is the complexity of (13)? The complexity of eval-uation of (13) is given solely by the complexity ofmoment calculation, other operations consume neg-ligible time. For the review of fast algorithms formoment computation we refer to [1], Chapter 7, andto [75]. As soon as the moment values are calculated,we implement (13) directly from the definitionbecause Matlab (as well as many other languages)supports recursive operations. It is of course possibleto derive, by backward substitution, non-recursivedefinition of theKNðp; qÞ’s but it does not lead to anymeasurable speed-up.

4 EXPERIMENTS

In this section we demonstrate the performance and limita-tions of the convolution invariants. We focus on the experi-ments showing the specific behavior of the invariantsKNðp; qÞ for variousN .

4.1 Basic Experiment on Simulated Data

The aim of the first experiment was to illustrate the behaviorof the invariants (13) in situations, where the convolutionmodel and the assumption of the symmetry of the PSF areperfectly valid. The only source of errors could be imagesampling (note that the theory was derived in a continuousdomain only) and finite precision of the calculations, so theproperties of the invariants can be precisely analyzed.

We took 1,000 images (common photographs), blurredthem by convolution with masks of various sizes, coeffi-cients, and symmetries (we used N ¼ 2; 3; 4; 6; 8 andN ¼ 1). We prevented the boundary effect by zero-pad-ding and calculated the invariants (13) up to the order 10for every blurred image. The relative error of each individ-

ual invariant was about (10�10Þ as one expects from the the-ory. Then we compared the values that according to thetheory need not match—for instance K4ðp; qÞ of the originalwith K4ðp; qÞ of its blurred version but with the PSF’s ofN ¼ 2. Here the relative errors ranged up to 100% depend-ing on the size of the blur. This clearly shows the necessityof choosing a correct N namely in case of a heavy blur or,equivalently, the necessity of having special invariants foreachN . See Fig. 5 for an illustration of this experiment.

4.2 Leaf Recognition

Automatic recognition of leafs has been studied thoroughlyin the last decade. Many methods and several publiclyavailable leaf databases have appeared, some of themimplemented in a very user-friendly form in mobile devices(for instance the Leafsnap [76] running on Apple i-phonesand i-pads). Several leaf recognition algorithms usemoments and moment invariants to rotation and scale asfeatures for the leaf description [77], [78], [79], [80]. Our

Fig. 5. The ratio between the invariants of the blurred image and of theoriginal. Blur mask of N ¼ 2 (elongated rectangle in this case). K2 is aperfect invariant while K3 and K4 vary significantly. The size of the origi-nal was cca 1,000 � 1,000 pixels.


group maintains the largest leaf database of Central Euro-pean wood species called MEW which contains the leafs ofall domestic and most of the imported trees and shrubsgrowing in the Czech Republic (altogether 9,745 leaf items,representing 151 species) along with a web-based recogni-tion system [81]. The classification algorithm used there [82]is based on moments and Fourier descriptors.

The aim of this experiment is twofold. First, we demon-strate the invariance and recognition power of the new blurinvariants in situations when the blur model is exactly aconvolution, N is known and we recognize blurred versionsof the leaves from the training set. In this experiment thefact that the objects are leaves is not essential; the invariantsbehave in the same way for any object set.

The second experiment is more related to real leaf recog-nition. It is aimed to recognize the generic species from theblurred image of a leaf, which is not a member of the train-ing set. Since here the degradation between the query imageand the training template(s) includes not only the blur butnamely the variations of the shape, color, size, etc., the taskis much more challenging.

We selected 15 species (classes) of the MEW database.We intentionally chose the species with very similar leavesto make the recognition difficult even for humans. In thefirst experiment the training set consisted of only one leafper class (see Fig. 6). We blurred each of these leaves ten

times by a simulated out-of-focus blur of various size, so weobtained 150 blurred leaves to be recognized (see Fig. 7 forsome examples). First we tried to recognize the blurredleaves by plain moments c11 and c22, which are not invariantto blur. The space of these two features is depicted in Fig. 8.The clusters that correspond to the species are spread, over-lap one another and cannot be separated (with two excep-tions). The classification has failed completely, whichillustrates the necessity of using blur invariants. Then theclassification was done by means of the invariants KNðp; qÞand the situation changed significantly. We achieved 100%success rate already for almost any couple of the invariants.The clusters are so compact that they look like single points

Fig. 6. The leaves used in the experiment—one sample per each spe-cies. Top row: Aesculus hippocastanum (leaflet of palmately compoundleaf), Ailanthus altissima (leaflet of pinnately compound leaf), Betulapapyrifera, Chaenomeles japonica, Cornus alba. Middle row: Cotoneas-ter integerrimus, Deutzia scabra, Fagus sylvatica, Fraxinus excelsior(leaflet of pinnately compound leaf). Bottom row: Gymnocladus dioicus(leaflet of pinnately compound leaf), Juglans regia (leaflet of pinnatelycompound leaf), Lonicera involucrata, Prunus laurocerasus and Staphy-lea pinnata (leaflet of pinnately compound leaf).

Fig. 7. Examples of the blurred leaves.

Fig. 8. The feature space of two non-invariant moments c11 and c22. Leg-end: – Aesculus hippocastanum, – Ailanthus altissima, – Betula pap-yrifera, – Cornus alba, – Cotoneaster integerrimus, – Deutziascabra, – Euonymus europaea, – Fagus sylvatica, – Fraxinus excel-sior, – Gymnocladus dioicus, – Chaenomeles japonica, – Juglansregia, – Lonicera involucrata, – Prunus laurocerasus, – Staphyleapinnata.


and are easy to separate (see Fig. 9 for an example). In thisexperiment the success rate almost did not depend on theparticular invariants and, for N � 4, on the choice ofN .

In the second experiment we used the same 15 classesbut now each class was represented by 20 training leaves.Then we created an independent test set consisting of 10leaves per class, each of them degraded by an out-of-focusblur. Each test leaf was classified by a minimum-distancerule in the space of the invariants KNðp; qÞ up to the order16. The success rate was about 70% for any N � 8. Since inthis case theoretically the true PSF fold number is infinity,theKNðp; qÞ’s should be invariant for any choice ofN . How-ever, as we explained in Section 2, the lower N the less rec-ognition power—if K2ðp; qÞ were used, the success rate wasonly about 50%.

The success rate of course depends on the number of clas-ses and the size of the training set. For bigger training setsand/or for classes which are more distinguishable, the suc-cess rate is between 80 and 90 percent. The achieved successrate 71 percent is surprisingly high. The moments were notspecifically designed to tolerate the intra-class variations ofthe leaves of the same species and emphasize the differencesbetween the species but apparently they exhibit this abilitywhile providing invariance to blurring. As a concluding test,we classified the leaves by the system [81], which usesmainlyFourier descriptors as the features. The recognition rate wasonly 23 percent because this system was not designed to berobust to the image blurring. This illustrates one possibleapplication of the blur invariants—they can be incorporatedinto existing recognition systems even if they use differentprinciples in order to achieve robustness to low-pass filtering.

4.3 Matching of Blurred Templates

This experiment was performed on real data under chal-lenging conditions where reaching a good matching score isdifficult. We intentionally choose the scenario with a heavyblur and the scenes with many self-similar parts in order to

make differences between the performance of various fea-tures apparent. We used the camera Nikon D5100 withseven diaphragm blades and adjusted the aperture suchthat the blades form a clear 7-fold symmetric PSF (seeFig. 10 for the image of a bright point). We took a pair ofimages, one sharp and the other one out of focus, of the size2,048 � 2,048 pixels (see Fig. 11; note the PSF shape which isapparent on the blurred image).

We selected randomly 230 circular templates of theradius 200 pixels in the blurred image. Each template wasmatched against the sharp image by a full search over thewhole scene, without using any prior information about itsposition. The matching criterion was the minimum distancein the space of blur invariants. We used the invariants K2

and K7 up to the sixth order. Since we know the groundtruth, we can measure the matching error (i.e. the Euclideandistance between the correct and matched location). First,we compared the invariants by the absolute number ofaccurate matches. The match is considered accurate if theerror is less than 10 pixels (we choose this threshold becausemultichannel processing algorithms such as [83] which, arein practice applied after the matching, are able to compen-sate for such registration errors). The invariants K2 yielded47 correct matches while K7 yielded 64 correct matches. Weused for comparison also plain moments, which is a wellknown matching technique, which lead to only 45 correctmatches. The absolute number of correct matches mightseem to be low even in the case of K7 but one should keepin mind that the task was intentionally difficult and theimpact of the boundary effect (which is given by the ratiobetween template and blur sizes) is extraordinary high.

Since the evaluation by the number of correct matchesdoes not take into account the errors of other trials, we eval-uated the results also in a different way using all trials.Assuming the errors in horizontal and vertical directionsare independent and identically normally distributed, thenthe Euclidean errors in 2D have Rayleigh distribution. Thisdistribution has a single parameter s which defines both

Fig. 9. The feature space of real parts of the invariant ReK1ð3; 0Þ andReK1ð4; 0Þ. Each cluster is formed by ten blurred images and one train-ing image of the leaf. Legend: – Aesculus hippocastanum, – Ailanthusaltissima, – Betula papyrifera, – Cornus alba, – Cotoneaster integer-rimus, – Deutzia scabra, – Euonymus europaea, – Fagus sylvatica,– Fraxinus excelsior, – Gymnocladus dioicus, – Chaenomeles japon-

ica, – Juglans regia, – Lonicera involucrata, – Prunus laurocerasus,– Staphylea pinnata.

Fig. 10. The 7-fold PSF captured as the image of a LED diode.

Fig. 11. Sharp test image (left) and out-of-focus image of the samescene (right). The 7-fold PSF shape is apparent on the blurred image.


mean and variance and can be easily estimated from thedata. Now the matching methods can be objectively com-pared by respective s—the lower its value, the better themethod. The s-value for K2 was 268 while for K7 only 145,which again illustrates the superiority of K7 invariants inthis case.

We repeated this experiment with other images andanother camera having circular aperture. Although particu-lar values vary, the main trend was always the same—theinvariants which correspond to the actual PSF shape per-form better than the others.

4.4 Registration of Blurred Images

Image registration in general is a process of overlaying twoor more images of the same scene. It is one of the mostimportant and most frequently discussed topics of imageprocessing in the literature (see [84] for a survey). In manyapplications, blurred frames are thrown away and not proc-essed. This is why the traditional methods do not considerimage blur, cannot handle it properly and usually fail whenregistering blurred images. However, there are situationswhere the images are inevitable blurred by camera shake,wrong focus, atmospheric turbulence, sensor imperfection,low resolution and other factors. A typical example is takingpictures from a hand by a cell-phone or a compact camera.In last few years new application areas have appeared,which inherently require registration of blurred low-resolu-tion images—multichannel blind deconvolution [30] and/or superresolution [31] of still images or video frames.Hence, a development of special methods for registeringblurred images is highly desirable.

Registration methods for blurred images can be, as wellas the general-purpose registration methods, divided intotwo groups—global and landmark-based ones. Globalmethods do not search for particular landmarks in theimages but rather try to estimate the between-image trans-formation directly. Most blur-invariant global methodswere motivated by traditional phase correlation [85], wherethe blur-insensitivity was achieved by modifications of thecross-power spectrum [41], [46], [47], [86]. Global methodsare fast and easy to implement, but their limiting require-ments—simple between-frame distortion (most of themallow only translation and/or rotation), the need for a largeoverlap of the images and the assumption of the uniformblur—might be a drawback in more complicated situations.

Since their first appearance in 1996, invariants to a cen-trosymmetric blur (N ¼ 2) have been several times success-fully used as the features in landmark-based registration ofremotely sensed [7], [10], [11], [12], [33], [87], medical [13],[14], [15], indoor [88] and outdoor [49], [55], [57] scenes. Aswe demonstrate, introducing new invariants to N-FRS blurwith higher discrimination power broadens their registra-tion capability.

In this experiment we register out-of-focus images thatare shifted and rotated with respect to one another. Bothblur and rotation are real and were intentionally introducedby the camera setting. We first studied the PSF of the cameraand found it to be circularly symmetric (at least in the first-order approximation, see Fig. 12). Hence, we employed theinvariants K1ðp; qÞ or, more precisely, their rotation-

invariant magnitudes. To show the differences in recogni-tion power, we repeated the same experiment usingK2ðp; qÞ. Since the images do not have a complete overlap,we apply the invariants locally as described below.

First, control point candidates (CPC’s) are detectedboth in the reference and the sensed images. Significantcorners and other corner-like dominant points are consid-ered as the candidates. To detect them, a method devel-oped particularly for blurred images [89] is employed.We detected 15 most prominent CPC’s in each frame. Toestablish the correspondence between the CPC’s, we cal-culated the blur invariants up to the order r over a circu-lar neighborhood of radius 14 pixels of each CPC. TheCPC’s are matched in the space of the invariants by mini-mum-distance rule and two pairs of the most similarCPC’s are found. This is the most complicated part whichis influenced by the blur. Having found these two pairs,the rest of the procedure is obvious. We estimate thetranslation, rotation and scale parameters, transform oneset of the CPC’s over the other one and match the rest ofthe CPC’s in the image domain. The invariants may serveas a consistency check.11 After establishing the correspon-dence between all CPC’s (those CPC’s having no close-enough counterpart are rejected), we calculate the finalaffine mapping parameters via least-square fit and resam-ple and overlay the sensed image over the reference one.

First we set r ¼ 5. Since in that case both K1 and K2

exhibit enough discrimination power, a correct match wasfound in each case as can be verified visually in Fig. 13 (notethat the two initial matching pairs found by K1 and K2 aredifferent but both correct). The same situation occurs forr > 5. Now let us repeat the experiment with setting r ¼ 4.The invariants K1 found a correct match again while K2

failed (check Fig. 14). The reason is that K1 provide betterdiscrimination because they are designed particularly forcircularly symmetric blurs. If we restrict to low order invari-ants r ¼ 3 only, both methods fail—the similarity of differ-ent CPC’s is too high to be distinguished by so few features.

We finally registered (using K1 and r ¼ 4) and sub-tracted the images (see Fig. 15 for the difference image).There are slight misregistration artifacts but in this experi-ment we do not want to measure the overall registrationaccuracy. We are only interested in the validity of the CPCmatching step. If the matching by invariants failed, it would

Fig. 12. The PSF of the camera used in the bookcase experiment.

11. Additional refinement step may be inserted here. For every pixelfrom a small neighborhood of the CP, its invariant vector is calculated.The point having the minimum distance to the invariant vector of theCP counterpart is set as the refined position of the CP. By an interpola-tion in the distance matrix, we can even achieve subpixel accuracy.Since this refinement has usually only slight impact on the transforma-tion parameters, we did not apply it in this experiment.


result in heavy artifacts in the difference image, which is notthe case here. Hence, all CPC’s were matched correctly,even if their initial positions might be detected with slighterrors.

The above described registration algorithm has, in addi-tion to r and N the choice of which we already discussed,another user-defined parameter—the radius R of the neigh-borhood the invariants are calculated from. Its choice isinfluenced by the type of the scene and by the extent of theblur. There is no explicit relationship between R and otherparameters, just heuristic conclusions based on our experi-ments can be made. Generally, the higher R the better dis-criminability and the higher computing complexity. In a”normal” clear image, R around 10 should be sufficient. Ifthe image contains periodic structures and thus many simi-lar CPC’s, higher R is required. In a blurred image there

is an additional constraint—to keep the boundary effectinsignificant, R should be chosen such that the size of theblur does not exceed 10-15% of the size of the R-neighbor-hood. The order of the used invariants r and the radius Rare ”inversely proportional”. Increasing one of themallows to decrease (up to some limit) the other one andvice versa. For the given image, the upper bound of R isset up by the homogeneity constraint—the blurring PSFcan be different for different neighborhoods but shouldnot vary within each of them.

5 CONCLUSION

In this paper we revised and substantially generalized thetheory of blur invariants. We presented the invariants toconvolution with an arbitrary N-fold symmetric PSF, bothin Fourier and image domain. We introduced the notion ofa primordial image as a canonical form of blurred images.This construct is defined in spectral domain by means ofprojection operators. We proved that the moments of theprimordial image are invariant to blur and we derivedrecursive formulae for their direct computation withoutactually constructing the primordial image. We furtherproved they form a complete set of invariants. We alsoshowed how to easily extent their invariance also to rota-tion. We discussed the properties of the proposed invariantsand the implementation issues. We illustrated by experi-ments the recognition power of the new invariants andshowed how it depends on N . We envisage the applicationof the new theory in tasks and systems where one has to rec-ognize and/or register blurred images.

We can see future challenges on this field namely in con-structing combined invariants to N-fold symmetric blurand affine transform, because we can meet affine-deformedimages in practice often. Unlike translation, rotation andscaling, the affine transform does not preserve N-fold sym-metries (except N ¼ 2), so the extension is not straightfor-ward even if both affine and blur invariants are known.Another direction is to look for invariants w.r.t. specifictypes of blur (Gaussian, motion, vibration, etc.). Knowingthe parametric expressions of the PSF, we should be able toderive more invariants than under a general assumption of

Fig. 15. The difference of two registered bookcase images usingK1 andr ¼ 4. The slight misregistration artifacts are cased mainly by the factthat the actual transformation between the images includes also per-spective projection and possibly also by an inaccurate CPC localization.The CPC matching itself was error-free.

Fig. 13. The bookcase, r ¼ 5. Small circles shows the detected CPC’s.The two initial matching pairs are denoted by “1” and “2”. Correct matchfound byK1 (top) andK2 (bottom).

Fig. 14. The bookcase, r ¼ 4. Small circles shows the detected CPC’s(they are the same as in Fig. 13). The two initial matching pairs aredenoted by “1” and “2”. Correct match found by K1 (top), incorrectmatch byK2 (bottom).


certain symmetry. Considering extensions to signals ofother dimensions than two, probably only 3D case is mean-ingful because 1D is irrelevant (except N ¼ 2) and higherdimensions are extremely complicated and of limited appli-cability. Even in 3D the set of possible symmetries of thePSF is much more rich than in 2D and a straightforwardextension of the 2D theory is impossible.

ACKNOWLEDGMENTS

The authors express their gratitude to the Czech ScienceFoundation for financial support of this work under thegrants No. P103/11/1552 and GA13-29225S. They alsowould like to thank Dr. Filip �Sroubek for providing thetest images for the experiment with registration, Dr. Jaro-slav Kautsky for his advice concerning numerical imple-mentation of moments, Matteo Pedone for the discussionabout matching accuracy evaluation, and a former Ph.Dstudent Michal Breznick�y for discussion and suggestionsregarding projection operators.

REFERENCES

[1] J. Flusser, T. Suk, and B. Zitov�a, Moments and Moment Invariants inPattern Recognition. Chichester, UK: Wiley, 2009.

[2] M.-K. Hu, “Visual pattern recognition by moment invariants,”IRE Trans. Inform. Theory, vol. 8, no. 2, pp. 179–187, 1962.

[3] D. Hilbert, Theory of Algebraic Invariants. Cambridge, UK:Cambridge Univ. Press, 1993.

[4] T. H. Reiss, “The revised fundamental theorem of moment invari-ants,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 13, no. 8,pp. 830–834, Aug. 1991.

[5] J. Flusser and T. Suk, “Pattern recognition by affine momentinvariants,” Pattern Recognit., vol. 26, no. 1, pp. 167–174, 1993.

[6] J. Flusser, T. Suk, and S. Saic, “Recognition of blurred images bythe method of moments,” IEEE Trans. Image Process., vol. 5, no. 3,pp. 533–538, Mar. 1996.

[7] J. Flusser and T. Suk, “Degraded image analysis: An invariantapproach,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 20, no. 6,pp. 590–603, Jun. 1998.

[8] R. Gopalan, P. Turaga, and R. Chellappa, “A blur-robust descrip-tor with applications to face recognition,” IEEE Trans. PatternAnal. Mach. Intell., vol. 34, no. 6, pp. 1220–1226, Jun. 2012.

[9] V. Ojansivu, “Blur invariant pattern recognition and registrationin the fourier domain,” Ph.D. dissertation, Department of Electri-cal and Information Engineering, Oulu University, Oulu, Finland,2009.

[10] Y. Bentoutou, N. Taleb, K. Kpalma, and J. Ronsin, “An automaticimage registration for applications in remote sensing,” IEEE Trans.Geosci. Remote Sens., vol. 43, no. 9, pp. 2127–2137, Sep. 2005.

[11] Z. Liu, J. An, and L. Li, “A two-stage registration algorithm for oilspill aerial image by invariants-based similarity and improvedICP,” Int. J. Remote Sens., vol. 32, no. 13, pp. 3649–3664, 2011.

[12] S. X. Hu, Y.-M. Xiong, M. Z. W Liao, and W. F. Chen, “Accuratepoint matching based on combined moment invariants and theirnew statistical metric,” in Proc. Int. Conf. Wavelet Anal. Pattern Rec-ognit., 2007, pp. 376–381.

[13] Y. Bentoutou, N. Taleb, M. Chikr El Mezouar, M. Taleb, and L.Jetto, “An invariant approach for image registration in digital sub-traction angiography,” Pattern Recognit., vol. 35, no. 12, pp. 2853–2865, 2002.

[14] Y. Bentoutou and N. Taleb, “Automatic extraction of controlpoints for digital subtraction angiography image enhancement,”IEEE Trans. Nucl. Sci., vol. 52, no. 1, pp. 238–246, Feb. 2005.

[15] Y. Bentoutou and N. Taleb, “A 3-D space-time motion detectionfor an invariant image registration approach in digitalsubtraction angiography,” Comput. Vis. Image Understanding,vol. 97, pp. 30–50, 2005.

[16] Y. Zhang, C. Wen, and Y. Zhang, “Estimation of motion parame-ters from blurred images,” Pattern Recognit. Lett., vol. 21, no. 5,pp. 425–433, 2000.

[17] Y. Zhang, C. Wen, Y. Zhang, and Y. C. Soh, “Determination ofblur and affine combined invariants by normalization,” PatternRecognit., vol. 35, no. 1, pp. 211–221, 2002.

[18] J. Lu and Y. Yoshida, “Blurred image recognition based on phaseinvariants,” IEICE Trans. Fundam. Electron., Commun. Comput. Sci.,vol. E82A, no. 8, pp. 1450–1455, 1999.

[19] X.-J. Shen and J.-M. Pan, “Monocular visual servoing based onimage moments,” IEICE Trans. Fundam. Electron., Commun. Com-put. Sci., vol. E87-A, no. 7, pp. 1798–1803, 2004.

[20] B. Mahdian and S. Saic, “Detection of copy-move forgery using amethod based on blur moment invariants,” Forensic Sci. Int.,vol. 171, no. 2–3, pp. 180–189, 2007.

[21] B. Mahdian and S. Saic, “Detection of near-duplicated imageregions,” in Computer Recognition Systems 2 (ser. Advances in SoftComputing), vol. 45. New York, NY, USA: Springer, 2007,pp. 187–195.

[22] L. Li and G. Ma, “Recognition of degraded traffic sign symbolsusing PNN and combined blur and affine invariants,” in Proc. 4thInt. Conf. Natural Comput., 2008, pp. 515–520.

[23] L. Li and G. Ma, “Optimizing the performance of probabilisticneural networks using PSO in the task of traffic sign recognition,”in Proc. 4th Int. Conf. Intell. Comput. Adv. Intell. Comput. TheoriesAppl.Aspects Artif. Intell., 2008, pp. 90–98.

[24] Y. Zhang, C. Wen, and Y. Zhang, “Neural network based classifi-cation using blur degradation and affine deformation invariantfeatures,” in Proc. 13th Int. Florida Artif. Intell. Res. Soc. Conf., 2000,pp. 76–80.

[25] C. Guang-Sheng and Z. Peng, “Dynamic wood slice recognitionusing image blur information,” Sens. Actuators A: Phys., vol. 176,pp. 27–33, Apr. 2012.

[26] J. Flusser, T. Suk, and B. Zitov�a, “On the recognition of wood sli-ces by means of blur invariants,” Sens. Actuators A: Phys., vol. 198,pp. 113–118, 2013.

[27] Z. Peng and C. Jun, “Weed recognition using image blurinformation,” Biosyst. Eng., vol. 110, no. 2, pp. 198–205, 2011.

[28] P. Zhao, “Dynamic timber cell recognition using two-dimensionalimage measurement machine,” AIP Rev. Sci. Instrum., vol. 82,pp. 1–8, 2011.

[29] Y. Zhang, Y. Zhang, and C. Wen, “A new focus measure methodusing moments,” Image Vis. Comput., vol. 18, no. 12, pp. 959–965,2000.

[30] P. Campisi and K. Egiazarian, Blind image deconvolution: theory andapplications. Boca Raton, FL, USA: CRC Press, 2007.

[31] P. Milanfar, Super-Resolution Imaging. Boca Raton, FL, USA: CRCPress, 2011.

[32] J. Flusser and B. Zitov�a, “Combined invariants to linear filteringand rotation,” Int. J. Pattern Recognit. Artif. Intell., vol. 13, no. 8,pp. 1123–1136, 1999.

[33] J. Flusser, B. Zitov�a, and T. Suk, “Invariant-based registration ofrotated and blurred images,” in Proc. IEEE Int. Geosci. Remote Sens.Symp., Jun. 1999, pp. 1262–1264.

[34] B. Zitov�a and J. Flusser, “Estimation of camera planar motionfrom defocused images,” in Proc. IEEE Int. Conf. Image Process.,2002, pp. 329–332.

[35] B. Zitov�a and J. Flusser, “Invariants to convolution and rotation,”in Invariants for Pattern Recognition and Classification, M. A.Rodrigues, Ed. Singapore, World Scientific, 2000, pp. 23–46.

[36] T. Suk and J. Flusser, “Combined blur and affine moment invari-ants and their use in pattern recognition,” Pattern Recognit.,vol. 36, no. 12, pp. 2895–2907, 2003.

[37] Y. Li, H. Chen, J. Zhang, and P. Qu, “Combining blur and affinemoment invariants in object recognition,” in Proc. 5th Int. Symp.Instrum. Control Technol., vol. SPIE 5253, 2003.

[38] R. Palaniappan, M. P. Paulraj, S. Yaacob, and M. S. Z Azalan, “Asimple sign language recognition system using affine momentblur invariant features,” in Proc. Int. Postgraduate Conf. Eng., Oct.2010.

[39] Y. Gao, H. Song, X. Tian, and Y. Chen, “Identification algorithm ofwinged insects based on hybrid moment invariants,” in Proc. 1stInt. Conf. Bioinform. Biomed. Eng., 2007, pp. 531–534.

[40] H. Ji, J. Zhu, and H. Zhu, “Combined blur and RST invariant digi-tal imate watermarking using complex moment invariants,”in Proc. 2nd Int. Conf. Signals, Circuits Syst., 2008, pp. 1–6.

[41] V. Ojansivu and J. Heikkil€a, “A method for blur and affine invari-ant object recognition using phase-only bispectrum,” in Proc. Int.Conf. Image Anal. Recognit., 2008, pp. 527–536.


[42] J. Flusser, J. Boldy�s, and B. Zitov�a, “Invariants to convolution inarbitrary dimensions,” J. Math. Imaging Vis., vol. 13, no. 2,pp. 101–113, 2000.

[43] J. Flusser, J. Boldy�s, and B. Zitov�a, “Moment forms invariant torotation and blur in arbitrary number of dimensions,” IEEE Trans.Pattern Anal. Mach. Intell., vol. 25, no. 2, pp. 234–246, Feb. 2003.

[44] J. Boldy�s and J. Flusser, “Extension of moment features’ invari-ance to blur,” J. Math. Imaging Vis., vol. 32, no. 3, pp. 227–238,2008.

[45] F. M. Candocia, “Moment relations and blur invariant conditionsfor finite-extent signals in one, two and n-dimensions,” PatternRecognit. Lett., vol. 25, pp. 437–447, 2004.

[46] V. Ojansivu and J. Heikkil€a, “Image registration using blur-invari-ant phase correlation,” IEEE Signal Process. Lett., vol. 14, no. 7,pp. 449–452, Jul. 2007.

[47] S. Tang, Y. Wang, and Y.-W. Chen, “Blur invariant phase correla-tion in X-ray digital subtraction angiography,” in Proc. Int. Conf.Complex Med. Eng., May 2007, pp. 1715–1719.

[48] I. Makaremi and M. Ahmadi, “Blur invariants: A novel represen-tation in the wavelet domain,” Pattern Recognit., vol. 43, no. 12,pp. 3950–3957, 2010.

[49] I. Makaremi and M. Ahmadi, “Wavelet domain blur invariants forimage analysis,” IEEE Trans. Image Process., vol. 21, no. 3, pp. 996–1006, Mar. 2012.

[50] R. R. Galigekere and M. N. S Swamy, “Moment patterns in theRadon space: Invariance to blur,” Opt. Eng., vol. 45, no. 7, pp. 1–6,2006.

[51] B. Xiao, J.-F. Ma, and J.-T. Cui, “Combined blur, translation, scaleand rotation invariant image recognition by Radon and pseudo-Fourier-Mellin transforms,” Pattern Recognit., vol. 45, pp. 314–321,2012.

[52] H. Zhang, H. Shu, G.-N. Han, G. Coatrieux, L. Luo, and J. L.Coatrieux, “Blurred image recognition by Legendre momentinvariants,” IEEE Trans. Image Process., vol. 19, no. 3, pp. 596–611,Mar. 2010.

[53] C.-Y. Wee and R. Paramesran, “Derivation of blur-invariant fea-tures using orthogonal Legendre moments,” Comput. Vis., IET,vol. 1, no. 2, pp. 66–77, Jun. 2007.

[54] X. Dai, H. Zhang, H. Shu, and L. Luo, “Image recognition by com-bined invariants of Legendre moment,” in Proc. IEEE Int. Conf.Inform. Autom., Jun. 2010, pp. 1793–1798.

[55] X. Dai, H. Zhang, H. Shu, L. Luo, and T. Liu, “Blurred image reg-istration by combined invariant of Legendre moment and Harris-Laplace detector,” in Proc. 4th Pacific-Rim Symp. Image Video Tec6h-nol., 2010, pp. 300–305.

[56] H. Zhu, M. Liu, H. Ji, and Y. Li, “Combined invariants to blur androtation using Zernike moment descriptors,” Pattern Anal. Appl.,vol. 3, no. 13, pp. 309–319, 2010.

[57] B. Chen, H. Shu, H. Zhang, G. Coatrieux, L. Luo, and J. L.Coatrieux, “Combined invariants to similarity transformation andto blur using orthogonal Zernike moments,” IEEE Trans. ImageProcess., vol. 20, no. 2, pp. 345–360, Feb. 2011.

[58] H. Ji and H. Zhu, “Degraded image analysis using Zernikemoment invariants,” in Proc. IEEE Int. Conf. Acoustics, Speech Sig-nal Process., 2009, pp. 1941–1944.

[59] Q. Liu, H. Zhu, and Q. Li, “Image recognition by combined affineand blur Tchebichef moment invariants,” in Proc. 4th Int. Conf.Image Signal Process., 2011, pp. 1517–1521.

[60] X. Zuo, X. Dai, and L. Luo, “M-SIFT: A new descriptor based onLegendre moments and SIFT,” in Proc. 3rd Int. Conf. Mach. Vis.,2010, pp. 183–186.

[61] D. G. Lowe, “Distinctive image features from scale-invariant key-points,” Int. J. Comput. Vis., vol. 60, no. 2, pp. 91–110, 2004.

[62] J. Kautsky and J. Flusser, “Blur invariants constructed from arbi-trary moments,” IEEE Trans. Image Process., vol. 20, no. 12,pp. 3606–3611, Dec. 2011.

[63] J. Flusser, T. Suk, and S. Saic, “Recognition of images degraded bylinear motion blur without restoration,” Comput. Suppl., vol. 11,pp. 37–51, 1996.

[64] J. Flusser, T. Suk, and S. Saic, “Image features invariant withrespect to blur,” Pattern Recognit., vol. 28, no. 11, pp. 1723–1732,1995.

[65] J. Flusser and B. Zitov�a, “Invariants to convolution with circularlysymmetric PSF,” in Proc. 17th Int. Conf. Pattern Recognit., 2004,pp. 11–14.

[66] A. Stern, I. Kruchakov, E. Yoavi, and N. S. Kopeika, “Recognitionof motion-blured images by use of the method of moments,” Appl.Optics, vol. 41, pp. 2164–2172, 2002.

[67] J. Flusser, T. Suk, and B. Zitov�a, “Comments on ”Weed recogni-tion using image blur information by Peng, Z. & Jun, C.” Biosys-tems Engineering, vol. 110, pp. 198–205, 2014.

[68] S. Zhong, Y. Liu, Y. Liu, and C. Li, “Water reflection recognitionbased on motion blur invariant moments in curvelet space fulltext sign-in or purchase,” IEEE Trans. Image Process., vol. 22,no. 11, pp. 4301–4313, Nov. 2013.

[69] B. Guan, S. Wang, and G. Wang, “A biologically inspired methodfor estimating 2D high-speed translational motion,” Pattern Recog-nit. Lett., vol. 26, pp. 2450–2462, 2005.

[70] S. Wang, B. Guan, G. Wang, and Q. Li, “Measurement of sinusoi-dal vibration from motion blurred images,” Pattern Recognit. Lett.,vol. 28, pp. 1029–1040, 2007.

[71] Q. Liu, H. Zhu, and Q. Li, “Object recognition by combined invari-ants of orthogonal Fourier-Mellin moments,” in Proc. 8th Int. Conf.Inform., Commun. Signal Process., 2011, pp. 1–5.

[72] J. Liu and T. Zhang, “Recognition of the blurred image by com-plex moment invariants,” Pattern Recognit. Lett., vol. 26, no. 8,pp. 1128–1138, 2005.

[73] Z. Zhang, E. Klassen, and A. Srivastava, “Gaussian blurring-invariant comparison of signals and images,” IEEE Trans. ImageProcess., vol. 22, no. 8, pp. 3145–3157, Aug. 2013.

[74] J. Flusser and T. Suk, “Rotation moment invariants for recognitionof symmetric objects,” IEEE Trans. Image Process., vol. 15, no. 12,pp. 3784–3790, Dec. 2006.

[75] T. Suk, C. H€oschl IV, and J. Flusser, “Decomposition of binaryimages – a survey and comparison,” Pattern Recognit., vol. 45,no. 12, pp. 4279–4291, 2012.

[76] N. Kumar, P. N. Belhumeur, A. Biswas, D. W. Jacobs, W. J. Kress,I. Lopez, and J. V. B. Soares, “Leafsnap: A computer vision systemfor automatic plant species identification,” in Proc. 12th Eur. Conf.Comput. Vis., 2012, pp. 502–516.

[77] E. J. Pauwels, P. M. de Zeeuw, and E. B. Ranguelova, “Computer-assisted tree taxonomy by automated image recognition,” Eng.Appl. Artif. Intell., vol. 22, no. 1, pp. 26–31, 2009.

[78] A. Kadir, L. E. Nugroho, A. Susanto, and P. I. Santosa, “Foliageplant retrieval using polar Fourier transform, color moments andvein features,” Signal & Image Process. : An Int. J., vol. 2, no. 3,pp. 1–13, 2011.

[79] L. Jiming, “A new plant leaf classification method based on neigh-borhood rough set,” Adv. Inform. Sci. Serv. Sci., vol. 4, no. 1,pp. 116–123, 2012.

[80] X.-F. Wang, D.-S. Huang, J.-X. Du, H. Xu, and L. Heutte,“Classification of plant leaf images with complicated back-ground,” Appl. Math. Comput., vol. 205, no. 2, pp. 916–926, 2008.

[81] “Recognition of woods by shape of the leaf,” 2012. [Online].Available: http://leaves.utia.cas.cz/index?lang=en

[82] P. Novotn�y and T. Suk, “Leaf recognition of woody speciesin Central Europe,” Biosyst. Eng., vol. 115, no. 4, pp. 444–452, 2013.

[83] F. �Sroubek and J. Flusser, “Multichannel blind deconvolution ofspatially misaligned images,” IEEE Trans. Image Process., vol. 14,no. 7, pp. 874–883, Jul. 2005.

[84] B. Zitov�a and J. Flusser, “Image registration methods: A survey,”Image Vis. Comput., vol. 21, no. 11, pp. 977–1000, 2003.

[85] E. de Castro and C. Morandi, “Registration of translated androtated images using finite Fourier transform,” IEEE Trans. PatternAnal. Mach. Intell., vol. 9, no. 5, pp. 700–703, Sep. 1987.

[86] M. Pedone, J. Flusser, and J. Heikkila, “Blur invariant translationalimage registration for N-fold symmetric blurs,” IEEE Trans. ImageProcess., vol. 22, no. 9, pp. 3676–3689, Sep. 2013.

[87] Y. Bentoutou, N. Taleb, A. Bounoua, K. Kpalma, and J. Ronsin,“Feature based registration of satellite images,” in Proc. 15th Int.Conf. Digital Signal Process., 2007, pp. 419–422.

[88] B. Zitov�a and J. Flusser, “Estimation of camera planar motionfrom blurred images,” in Proc. Int. Conf. Image Process., Sep. 2002,pp. 329–332.

[89] B. Zitov�a, J. Kautsky, G. Peters, and J. Flusser, “Robust detectionof significant points in multiframe images,” Pattern Recognit. Lett.,vol. 20, no. 2, pp. 199–206, 1999.


Jan Flusser (M’93 – SM’03) received the MScdegree in mathematical engineering from theCzech Technical University, Prague, CzechRepublic in 1985, the PhD degree in computerscience from the Czechoslovak Academy of Sci-ences in 1990, and the DSc degree in technicalcybernetics in 2001. Since 1985, he has beenwith the Institute of Information Theory and Auto-mation, Academy of Sciences of the CzechRepublic, Prague. From 1995 to 2007, he washolding the position of a head of the Department

of Image Processing. Currently (since 2007), he is a director of the Insti-tute. He is a full professor of computer science at the Czech TechnicalUniversity and at the Charles University, Prague, Czech Republic, wherehe gives undergraduate and graduate courses on digital image process-ing, pattern recognition, and moment invariants and wavelets. Hisresearch interest include moments and moment invariants, image regis-tration, image fusion, multichannel blind deconvolution, and super-reso-lution imaging. He has authored and coauthored more than 150research publications in these areas, including the monographMomentsand Moment Invariants in Pattern Recognition (Wiley, 2009), tutorialsand invited/keynote talks at major international conferences. In 2007, Hereceived the Award of the Chairman of the Czech Science Foundationfor the best research project and won the Prize of the Academy of Scien-ces of the Czech Republic for the contribution to image fusion theory. In2010, he was awarded by the prestigious SCOPUS 1000 Award pre-sented by Elsevier. He is a senior member of the IEEE.

Tom�a�s Suk received the MSc degree in electricalengineering from the Czech Technical University,Faculty of Electrical Engineering, Prague, 1987.The CSc degree (corresponds to PhD) in com-puter science from the Czechoslovak Academyof Sciences, Prague, 1992. From 1991, he is aresearcher with the Institute of Information The-ory and Automation, Academy of Sciences of theCzech Republic, Prague, a member of theDepartment of Image Processing. He has auth-ored 14 journal papers and 31 conference

papers. He is a co-author of the book Moments and Moment Invariantsin Pattern Recognition (Wiley, 2009). His research interests includes dig-ital image processing, pattern recognition, image filtering, invariant fea-tures, moment and point invariants, geometric transformations ofimages, and applications in remote sensing, astronomy, medicine, andcomputer vision. In 2002, he received the Otto Wichterle premium of theAcademy of Sciences of the Czech Republic for young scientists.

Ji�r�ı Boldy�s received the MSc degree in physicsfrom the Charles University, Prague, CzechRepublic in 1996, and the PhD degree in mathe-matical and computer modeling from the sameuniversity in 2004. Since 1998, he has been withthe Institute of Information Theory and Automa-tion, Academy of Sciences of the Czech Repub-lic, Prague. He has also been working in privatecompanies in the fields of image processing andmathematical modelling. His research interestsinclude mainly object and pattern recognition,

moment invariants, physics and image processing in nuclear medicine,medical imaging, wavelet transform and thin film image processing.

Barbara Zitov�a received the MSc degree in com-puter science from the Charles University,Prague, Czech Republic in 1995 and the PhDdegree in software systems from the Charles Uni-versity, Prague, Czech Republic in 2000. Since1995, she has been with the Institute of Informa-tion Theory and Automation, Academy of Scien-ces of the Czech Republic, Prague. Since 2008,she is the head of the Department of Image Proc-essing. She gives undergraduate and graduatecourses on Digital Image Processing and Wave-

lets in Image Processing at the Czech Technical University and at theCharles University, Prague, Czech Republic. Her research interestsinclude geometric invariants, image enhancement, image registrationand image fusion, and image processing in cultural heritage applica-tions. She has authored/coauthored more than 50 research publicationsin these areas, including the monograph Moments and Moment Invari-ants in Pattern Recognition (Wiley, 2009) and tutorials at major confer-ences. In 2003, she received the Josef Hlavka Student Prize, in 2006the Otto Wichterle premium of the Academy of Sciences of the CzechRepublic for young scientists, and in 2010 she was awarded by the pres-tigious SCOPUS 1,000 Award presented by Elsevier.

" For more information on this or any other computing topic,please visit our Digital Library at www.computer.org/publications/dlib.


786 IEEE TRANSACTIONS ON PATTERN ANALYSIS AND...

Documents

Transcript of 786 IEEE TRANSACTIONS ON PATTERN ANALYSIS AND...