Pattern Recognition Letters 11 (1990) 267-274

April 1990

North-Holland

A hierarchical morphologicalimage decomposition

Alexander TOETInstitute for Perception TNO, Kampweg 5, Soesterberg NL-3 769-DE, The Netherlands

Received 28 August 1989

Abstract: This paper presents a new hierarchical image description based on a morphological skeleton representation . Theprimitives of this representation are size scaled versions of well defined primitive shapes (structuring elements) . The construc-tion of the skeleton itself and the determination of the hierarchical relation between its components are performed in parallel .The resulting tree of skeleton components is an efficient tool for qualitative and quantative image analysis .

Key words: Mathematical morphology, morphological skeleton, multiresolution techniques, hierarchical image representation,shape description .

1. Introduction

A skeleton is a representation of a closed boun-dary shape by thin lines that retain the connectivityof the original shape (e .g . Blum and Nagel, 1978 ;Nackman and Pizer, 1985) . Under certain restric-tions, a shape can be reconstructed from its skele-ton representation by means of an inverse skeletontransform . The concepts of size and connectivityinduce a natural hierarchy (partial order) on the setof skeleton components .

A hierarchical skeleton representation of a shapehas many useful properties . It is invariant to thesize, orientation and position of the shape . Thus ashape can be compared to prototypes withouthaving to normalize its size or orientation . Noise isrelegated to the lower parts of the hierarchy andcan be ignored without affecting the description athigher levels in the hierarchy . Hence, the represen-tation can still be used when an image has beencorrupted with noise or when the object boun-daries are blurred or poorly defined . Objects canbe tracked efficiently through sequences of imagesby top-down matching of the hierarchical descrip-tions .

Features that persist over a large range of resolu-tions are more global or prominent than featuresthat exist only over a small range of resolutions .This notion recently inspired several authors toestablish a partial order on the components of askeleton by monitoring their order of eliminationon progressive blurring (Dill, 1987 ; Pizer et al .,1987 ; Gauch and Pizer, 1988). However, in animage that has been reduced in resolution by theapplication of a convolution filter, features will bedisplaced with respect to their true physical posi-tion . As a result the correspondence between eventsat different scales can be ambiguous (Dill et al .,1987) .In contrast, morphological operations specifi-

cally manipulate certain shapes in the image with-out affecting the remaining image structure .Mathematical morphology makes it possible todescribe a shape directly in terms of its (skeleton)components (i .e . subsets in morphological terms)rather than by abstract boundaries or curves .Hence, the use of mathematical morphology in thegeneration of a multiresolution sequence of imagesand in the extraction of shape primitives at subse-quent levels of resolution results in a natural hier-

0167-8655/90/$3 .50 © 1990, Elsevier Science Publishers B .V. (North-Holland)

267

Volume 11, Number 4

PATTERN RECOGNITION LETTERS

archical decomposition of shapes in the image .In this paper we propose a new hierarchical shape

representation called the morphological skeletontree. The nodes of the tree represent skeleton com-ponents and correspond to size scaled versions ofprimitive shapes . The partial order on the skeletoncomponents induced by their size and connectivityis explicitly encoded in the branching structure ofthe skeleton tree . Major features are stored at thehigher levels and finer details are stored at thelower levels in the hierarchy . This new representa-tion can efficiently be constructed and stored andis particularly useful in shape recognition applica-tions since the matching process in such a structurebecomes very simple and fast .

Section 2 describes the extraction of the mor-phological skeleton components . Section 3 presentsan algorithm for the construction of the morpho-logical skeleton tree. Section 4 shows the results ofthe application of the algorithm on 1-D and 2-Dsignals. Finally, some concluding remarks arepresented in Section 5 .

2. Discrete morphology

In this section we briefly define the basic mor-phological transformations for discrete (i .e . sam-pled) functions . In the sequel Z will denote the setof integers and R the set of real numbers . Discrete(i .e . sampled) images and structuring elements willbe denoted by capital letters A, B, C, . . . , theirdomains and ranges by script letters . d, Q, 8, . . . .

Let F(x) be a function defined on a finite dis-crete d-dimensional domain SC7Ld (d= 1, 2, . . .)and with a discrete or continuous amplitude range

Table I

268

Name

set translation of ,& by psymmetric set of .9symmetric function of BMinkowski addition of F and BMinkowski subtraction of B from Fdilation of F by Berosion of F by Bclosing of F by Bopening of F by B

Xp={b+p :be .fW}

,l ={-b :be .ss}

B(x) =B(-x)(FQ B)(x) = max{F(y) : y e 5R x}(F(DB)(x) = min{F(y) : y e ax }(FO A(x) = max{F(Y) : ye .X x}(FO e)(x) = min{F(y) : y e 58x }

Fs B(x) = (F(DB) G B(x)FOB(x) = (FQ B)eB(x)

Definition (x,y,peZ", .J6CZd,d=1,2, . . .)

April 1990

MC7L or P . A structuring element is any functionG(x) whose support & is compact and connectedsubset of 77 d. In the sequel we will only use flat(binary or bivalued) structuring elements B(x),with B(x) = 0 for x c W . The extension of ourresults to multivalued (grayscale) structuring ele-ments is straightforward . (For details see Haralicket al., 1987 ; Maragos, 1987 and Serra, 1982 .)

The basic morphological operations on discretefunctions with binary structuring elements are de-fined in Table 1 .

To implement these operations we assume thatF(x)=-c for xffS and B(x) = -co for xe 2 . LetB be a structuring element of size one (by conven-tion) . The n-fold Minkowski addition of B withitself ; i .e .

nB=BQBQB . . • QB (n=0, 1,2, . . .),

n-times

generates a family of patterns parametrized by thediscrete size parameter n . If n=0 then nt=0 byconvention . If the set . is convex, then nB isshaped like (homotopic to) B but has size n . Wedefine

Fo nB(x) _ (F® nB) (B nB

=[(FC) )eB . . .QB[Q+B(jB . . .D+Bv

n-times

n-times

as the multi-scale opening of F by B at scale n=0, 1, 2, . . . . Likewise, the function

F•nB=(FE[ nB) enB

is the multi-scale closing of F by B . At large scalesn the opening (closing) with nB creates large flatplateaus (sinks) shaped like nB . Henceforth we will

Volume 11, Number 4

PATTERN RECOGNITION LETTERS

April 1990

only consider structuring elements with the pro-perty B(O)>0 so that the erosion by B is an anti-extensive operation . Thus we have

FO- nB > FQ (n + l) B

for all n . Since 9 is finite there will be a maximumindex N such that FO (N+ 1) B will be equal to -ooeverywhere . The skeleton subfunctions of F withrespect to B can be defined as the functions

[Sn(F)1(x) = (F(3nB) -(FO nB) o B,n=0,1,2, . . .,N

where '-' denotes the usual algebraic difference(Peleg and Rosenfeld, 1981) . Note that the S„ (F)'sare nonnegative functions everywhere because theopening is anti-extensive . The union of all S„ (F)'sgives the morphological skeleton of F with respectto B (Serra, 1982; Maragos and Schafer, 1986), IfB is a (discrete approximation of a) disk, then theskeleton is the well-known medial axis of F (Blumand Nagel, 1978) .

The original function F can be reconstructed ex-actly from its set of skeleton subfunctions by sequen-tially pointwise summing the S„ (F)'s and dilatingthe partial sums by B (Peleg and Rosenfeld, 1981) :

F= [[[SN(F)O+ Bl +SN-,(F)1 O B+SN_2(F) . . .1

Q+ B+ S0(F) .

In general the function dilation does not commutewith the function addition, because

(F+ G)Q+B<(F(DB)+(GQ+B)

for any F, G, B . Thus the definition of the skeletonsubfunctions does not allow the exact reconstruc-tion of F as the union of all S„ (F)'s dilated by nB,in contrast to Serra's reconstruction algorithm forbinary images (Serra, 1982). Recently Maragos(1988a) defined the skeleton subfunctions suchthat an exact reconstruction of F can be obtainedin both ways . As the skeleton components off withrespect to B he defines

[D„(F)1(x)

'FenB(x) if F®nB>(F(DnB)cB(x),-oc

if FQnB=(FOnB)oB(x)

where 0<n<N=max{n :F®nBEA-oo} . The re-

construction of F from its skeleton subsets D„ (F)can be obtained from

Fo kB = [[[DN (F) Q+ B1 VDN- 1 (F)] ® BV DN_ 2 (F) . . .VDk (F)l O+ kB

V D„Q+nBk<n<N

where V denotes the operation of taking thesupremum. If k=0, FokB=F and we have anexact reconstruction . If 1 < k <N the opening of Fby kB is obtained as a partial reconstruction(smoothed version) of F.

3 . The morphological skeleton tree

Let E„ (F) denote the family of images that isobtained by eroding F n-times with a structuringelement B . (For brevity we will sometimes omit theargument F.) According to their definition theskeleton subfunctions of F with respect to B areobtained from the pointwise comparison of thefunction value of each element of the family E„and its opening with B . A skeleton component isextracted wherever the local function values of E„and E„ o B differ. The skeleton components inherittheir function value from E,, . The equivalenceE„oB=_E„ +1 Q+B suggests an implementation inwhich each element of the family E„ is comparedwith the dilation of its successor. Note that thedilation with a binary structuring element B merelyspreads the local maximum function value over itsregion of support J6 . Hence, on implementationthe dilation can be replaced by a regionwise com-parison over W :

[Da (F)](x)

E„(x) ifE„(x)>E +1 (x+b)Vbe e,-00 if E„(x)=VE +1(x+b) bEE .

Essential in the construction of the morphologicalskeleton tree is the function value tracing routine .This routine links an element E„ (x) to an elementEi+1 (x+d) if

dfl=min{lld1 : jE~I1(x+d;)-E„(x)~

< jjEn+1(x+b) - E„ (x)llVbca,d,e .t1 .

269

Volume 11, Number 4

PATTERN RECOGNITION LETTERS

A function value E„ (x) annihilates when it is nolonger represented in E,,,I(x+b), be . . In thiscase x represents the center of a skeleton compo-nent and n represents the scale parameter of thecomponent . In the image obtained by opening F benB the location x corresponds to the center of alarge flat plateau shaped like nB .

A skeleton component can be obtained by startingat a certain location xE3* in the original imageF=E0 and tracing the function value F(x) throughthe family E„ until annihilation . The complete setof skeleton components is obtained when this pro-cess is performed at every element of the supportof the original image (i .e. Vxeg). A hierarchicalrelation between skeleton components with differ-ent scale parameters is established by continuingthe function value tracing process after the detec-tion of a skeleton component until the next skeletoncomponent is extracted . Thereafter this newlyobtained skeleton component is linked with theprevious one . The linking process stops whenE,,,,==-E„ or when a predefined number of itera-tion steps have been performed . This process resultsin a tree of skeleton components in which com-ponents that have spatial overlap and that are closein their function values are linked . The leaf nodesrepresent the smallest details in the original image ;the root node corresponds to the background ofthe entire scene . The size of the skeleton compo-nents strictly increases along a branch and in thedirection of the root .

On implementation the function value tracingand skeleton component extraction process is per-formed in parallel for all nodes at a certain level inthe hierarchy .

The skeleton construction starts by the computa-tion of E, as EGQB. The function value tracingroutine links every element of E0 to an element ofEl . Object nodes representing skeleton compo-nents are created for elements of E0 with a func-tion value that cannot be traced to E, . A leaf nodeis created for every element in E0 that is not repre-sented by an object node . Every (leaf and object)node inherits a link to an element of E, from theelement in E0 which it represents .

Thereafter E2 is computed as El QB. Pointersto elements of E2 are computed for all elements ofE, that are linked to either leaf nodes or object

270

April 1990

nodes of E0 . Object nodes are created for ele-ments of Et with a function value that cannot betraced to E2 . These object nodes inherit their linksfrom the elements of El which they represent . Alllinks relating object nodes and leaf nodes in E0 toelements of E, which are not represented by objectnodes are replaced (updated) by the links of theseelements of E, to elements of E2 .

The process of function erosion, function valuetracing, skeleton component extraction, objectnode creation and link updating is repeated until asingle object node remains or until a predefinednumber of iteration steps has been performed .

The original image can be reconstructed exactlyfrom its morphological skeleton tree representa-tion by projecting all the object node functionvalues along the branches to the leaf nodes, takingthe supremum of the value being projected and theobject node value each time an object node is en-countered along the branch . Partial reconstruc-tion can be obtained by replacing the functionvalue of a selected set of object nodes by -=(thereby deactivating them in the reconstructionprocess) .

The morphological skeleton tree provides a hier-archical shape description with major featuresrepresented at the lower levels and details at thehigher levels . If a shape is corrupted by noise, mostof the disturbance is limited to the lower levels ofthe hierarchy . Thus, noise does not affect therepresentation at higher levels in the hierarchy .

4. Results

The extraction of the skeleton subsets D„ andthe construction of the morphological skeleton treeof a one-dimensional signal is illustrated in Figure1 . Figure la shows an arbitrary 1-D signal F (itera-tion step 0) and the family of waveforms E.,n=0, . .., 13, obtained by repeated application ofan erosion with a bivalued 1-D structuring elementof length 3 . The line segments in this figure repre-sent the links between elements of successive mem-bers of the family E„ (i .e. indicate the pathstraversed by the function value tracing and linkingroutines). Object nodes are delineated by squareboxes. Note that every function value in the origi-

Volume 11, Number 4

01

•

2w 3

•

,

Z•

5s

0

E

7

89

•

10r 111213

5

PATTERN RECOGNITION LETTERS

•

23

SAMPLE NUMBER

23456789 10 11 12 13 04 15 16 17 18 19 20 21 22 23 24 250

45

105 145

1 5

+5 85

195

1

1 5 1

5

105/651

515

65105

4

I5~8~ 1 5 1

5 1 5 105

105 105 65

55

5~I5~55 65

5 +5 45

45+

~1 5 1 5 1 5 105

5 05' 65' 5 55

5

5"1

45 45 45 4 A 45 45' 155105 1 5 105 5 1 ~5

5

5 54

5 5

5 5

15 45 45 5 45 45 45 47"85'1 5 1

5- 6y- s 5 5 5

•

S 5 5 5 5 5N' 45 45 45 45 45 45 15 45 45 65" 5 5 5 5 5•

6 5 5 5 5 5 5 5 45 45 4' 45 45 45 45 45 457

5

5

5

5

5

5

5"45 45 4

45 45 45 45 45 4

5

5

5

58

5 5 5 5 5 5 5 5X45 ~4S 45 45 45 45 45__ 5 5 5 5 5•

9 5 5 5 5 5 5 5 5 5 45 45 45 45 45 5 5]0

5

5

5

5

5

5

5

5

5

545 45 45~ 5

5

5

5

5

5

511

5

5

5

5

5

5

5

5

5

5

5

5

5

3

5

5

5

512

5

5

5

5

5

5

5

5

5

5

5

5

5

5

5

5

5

5

513

5

5

5

5

5

5

5

5

5

5

5

5

5

5

5

5

5

5

5

SAMPLE NUMBER2345678910II1213141516171819201--_O~B o~o~; ,g- o~0 8 0

" 16,58

0,~2~~

o 0~0~0 0"l0

~0 0

0 0~0~ p b 0o 0 0~e~s5

0 0

0 0 0 0~°

°0

0

0

0~

0

0

0

p

0

0

0

0

0'~ 0

0

6 I00

0

0

0

0

0

0

0

0

0

0

0

0 -85'105~0~0 0 0 0 0 0~0 0 0 6 0 0 0 0 0 0~95~

00

0

0

0

0

0

0

00

0

0

0

0

0

0

0~0

0

0

0

0

0~0

0

0

00

0

0

0

0

0

0

~1

0

0o

0

0

0

0

0

0

0

0

0~0

0

0'0

0

0

0

0

00

0

0

0

0

0

0

0

0

0

0'1 io

0

0

0

0

0

00

o

0

0

0

0

0

0

0

0

0\ 0

0

0

0

0

0

00

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

SAMPLE NUMBER

21 22 23 24 250 125 0Vo~o0~0 00

0

00

0

0

00

0

0

00

0

0

00

0

0

00

0

0

00

0

0

00

0

0

00

0

0

00

0

0

00

0

0

April 1990

Figure 1 . Numerical representation of : (a) An arbitrary i-D signal F (row 0) and the family of waveforms E,,,n=0, . . ., 13, obtainedby repeated application of an erosion with a btvalued 1-D structuring element of length 3 . Line segments represent the links betweenelements of successive members of the family E, Object nodes are delineated by square boxes . ((b) The skeleton subfunctions D,, .

(c) The resulting skeleton tree .

27 1

Volume 11, Number 4

PATTERN RECOGNITION LETTERS

nal image is represented by an object node in thetree . In this example the root node (correspondingto the background) was determined after 12 itera-tion steps . The non-zero elements in row n inFigure lb correspond to the skeleton componentsthat are determined after n iteration steps . FigureIc represents the resulting skeleton tree of thesignal in Figure la .

Figure 2 shows the graph of the waveform ofFigure I (2a) together with the skeleton subfunc-tions S0 and Do (2b and 2c respectively) and thepartial reconstruction obtained by filtering eitherof the aforementioned skeleton subfunctions (2d) .Note that the waveform in Figure 2d can be ob-tained by opening the waveform in Figure 2a by a1-D structuring element of length 3 . Complete re-constructions are obtained (i) by pointwise addi-tion of the two waveforms in Figures 2b and 2d or(ii) by taking the supremum (pointwise maximum)

272

250

200

1500c

^ 100

50

0 1

V

50C r

, /m 020

25

250

200

150-

100-

Q

0 4A10 20

Q

5

April 1990

of the waveforms in Figures 2c and 2d .The morphological skeleton tree provides a com-

plete image description . Partial descriptions can beobtained by selecting a subset of the tree nodeswith a partial order induced by the branches in thecomplete tree. Image reconstructions based on par-tial descriptions correspond to logically filteredversions of the original image . Some examples oflogically filtered images are shown in Figure 3 . Thereconstruction shown in Figure 3b was obtained byfiltering the tree nodes corresponding to smallsquare objects in the original image of Figure 3a .Filtering tree nodes corresponding to elongated ob-jects results in the reconstruction shown in Figure3c. Figure 3d shows that annular objects can be fil-tered from the original image by filtering the corre-sponding nodes in the reconstruction process . Notethat none of these results can be obtained by con-ventional linear filtering techniques .

250

200

150

100

50

25

10

15

20

25

10

15

20

25sample

sample

Figure 2 . The graph of the waveform of Figure I (a) together with the skeleton subfunctions So (b) and Do (c) and the partialreconstruction obtained by filtering either of the aforementioned skeleton subfunctions (d) .

Volume 11, Number 4

5. Conclusion

Previous studies established a hierarchical rela-tion between skeleton components by repeatedskeletonization of an image at multiple levels ofresolution (Dill et al ., 1987; Gauch et al ., 1987 ;Gauch and Pizer, 1988 ; Pizer et al ., 1987) . Thesemethods are inherently time consuming and re-quire a large amount of computer memory . Theskeletons are computed independently at all levelsof resolution . Thus, the computation of a skeletonat a lower level of resolution does not benefit fromprevious results obtained at a higher level of reso-lution . Moreover, the hierarchical skeleton repre-sentation obtained by these methods only providequalitative information about shape components .In the approach presented here the image is

skeletonized only once . The hierarchical relationbetween the skeleton primitives is established inparallel with the skeletonization process itself. Thecomponents of the morphological skeleton tree aresize scaled versions of well defined shape primi-tives (structuring elements) .

The morphological skeleton tree provides bothqualitative (structural) and quantitative (size) in-formation about the shapes in an image . Partial

PATTERN RECOGNITION LETTERS

Figure 3 . This figure shows an original image (a) together with the logically filtered images obtained by filtering the tree nodes corres-

ponding to small square objects (b), elongated objects (c) and chain nodes (d) .

reconstructions correspond to adaptively spatial-scale invariant smoothed versions of the originalimage (Verbeek and Verwer, 1989) .

References

Blum, H. and R.N. Nagel (1978) . Shape description usingweighted symmetric axis features . Pattern Recognition 10,167-180 .

Crowley, 7 .L_ and A .C. Parker (1984) . A representation forshape based on peaks and ridges in the difference of low-passtransform . IEEE Trans . Pattern. Anal- Machine Intell . 6,156-170 .

Dill, A.R ., M. Levine and P .B. Noble (1987) . Multiple resolu-tion skeletons . IEEE Trans. Pattern . Anal. Machine Intell. 9,495-504 .

Gauch, J.M ., W.R. Oliver and S .M . Pizer (1987) . Multiresolu-tion shape descriptions and their applications in medicalimaging . In : C.N. de Graaf and M .A . Viergever, Eds ., Infor-mation Processing in Medical Imaging, Proc . 10th Conf.Plenum Press, New York, 131-149 .

Gauch, J.M . and S .M. Pizer (1988) . Image description via themultiresolution intensity axis of symmetry . Proc. 2nd lm .Conf Comp. Vision . IEEE Comp . Soc. Press, Washington,269-274 .

Pizer, S.M ., W.R. Oliver and S .H. Bloomherg (1987) . Hier-archical shape description via the multiresolution symmetricaxis transform . IEEE Trans. Pattern. Anal . Machine Intell.9, 505-511 .

April 1990

2 7 3

Volume 11, Number 4

PATTERN RECOGNITION LETTERS

Nackman, L.R. and S.M. Pizer (1985) . Three-dimensionalshape description using the symmetric axis transform . I .Theory . IEEE Trans. Pattern. Anal. Machine Intell. 7,187-202 .

Haralick, R.M., S .R . Sternberg and X. Zhuang (1987) . Imageanalysis using mathematical morphology . IEEE Trans. Pat-tern Anal. Machine Intel! . 9(4), 532-550 .

Maragos, P . and R.W. Schafer (1986) . Morphological skeletonrepresentation and coding of binary images . IEEE Trans.Acoust. Speech Signal Process . 34, 1228-1244 .

Maragos, P. (1987) . Pattern spectrum of images and morpho-logical shape-size complexity . IEEE Proc. ICASSP 87. IEEEComp. Soc . Press, Washington, 241-244 .

Maragos, P . (1988a) . Symbolic signal representation using non-linear filtering . IEEE Proc. ICASSP 88. IEEE Comp . Soc .Press, Washington, 944-947 .

274

April 1990

Maragos, P. (1988b). Morphology-based symbolic imagemodelling, multi-scale nonlinear smoothing, and patternspectrum . Proc. 1988 IEEE Comp . Soc . Conf. on Comp.Vision and Patt. Recognition . IEEE Comp . Soc . Press,Washington, 766-772 .

Peleg, S. and Rosenfeld, A. (1981). A min-max medial axistransformation, IEEE Trans. Pattern Anal. Machine Intell.3, 208-210 .

Serra, J. (1982). Image Analysis and Mathematical Mor-phology. Academic Press, London, New York .

Verbeek, P.W. and B .J.H . Verwer (1989). 2-D adaptivesmoothing by 3-D distance transformation . Pattern Recogni-tion Letters 9, 53-65 .