FRIP: A Region-Based Image Retrieval Tool Using Automatic Image Segmentation and Stepwise Boolean AND Matching

IEEE TRANSACTIONS ON MULTIMEDIA, VOL. 7, NO. 1, FEBRUARY 2005, pp. 105-113ByoungChul Ko and Hyeran ByunReporter: Jen-Bang Feng

2

Outline

Image Retrieval Content-Based Image Retrieval The Proposed Scheme Experimental Results

3

Image Retrieval

ImageDB

Image retrievalscheme FeaturesFeaturesFeaturesFeaturesFeaturesFeaturesFeatures

QueryImage

Image retrievalscheme

Feature

Compare SearchingResults

4

Content-Based Image Retrieval From text-based retrieval scheme

WWW search engine Query-by-image in early 90’s From global to local (region)

Region Of Interest

5

The Proposed Scheme

1. Image Segmentation Two-Level Segmentation Using Adaptive Circular Filter a

nd Bayes’ Theorem Iterative Level Using Region Labeling and Iterative Regio

n Merging2. Feature extraction

Color Texture Normalized Area Shape and Location

3. Stepwise Similarity Matching

6

Two-Level Segmentation Using Adaptive Circular Filter and Bayes’ Theorem

Adaptive Circular Filter

Image(RGB)

Image(CIE Lab)

SmoothedImage(CIE Lab)

Remove middle frequency

Color histogram

Separate regions by circular filters

RegionsRegionsRegionsRegions

7

Two-Level Segmentation Using Adaptive Circular Filter and Bayes’ Theorem

a is similar to c in colorbut a is closer to b than c Example of circular filtering process

8

Two-Level Segmentation Using Adaptive Circular Filter and Bayes’ Theorem

xyyx

CyxyxM

MyxMMyxM

MyxMyxM

cc

MccCP

CcPCPCcPCP

CcPCPcCP

,

,,

,,

,,

else

then ,5.0| if

||

||

Three circular filters3x3, 7x7, 11x11

CM: the most frequently observed histogram binsCM: other binscx,y: center value of CM

MC: the major class color

9

Two-Level Segmentation Using Adaptive Circular Filter and Bayes’ Theorem

division according to the edge distribution Selected filter, 3x3, 7x7, 11x11

Segmentation result Final segmented image

10

Iterative Level Using Region Labeling and Iterative Region Merging

Image(RGB)

Image(CIE Lab)

SmoothedImage(CIE Lab)

Remove middle frequency

Color histogram

Separate regions by circular filters

RegionsRegionsRegionsRegions

RegionsRegions

Merge regions

11

Iterative Level Using Region Labeling and Iterative Region Merging

N

i

imbb

imaa

imLL TRRRRRRIf

For the N neighbor regions

Then merge the regions

If the number of regions is larger than 30Then increase the threshold and repeat the circular filter

12

Feature extraction Color

Average AL, Aa, Ab

Variance VL, Va, Vb

Color distance of Q and T

2,,,,

,

VbVaVLCCC

AbAaAlCCC

CTQ TQTQd

13

Feature extraction Texture

Biorthogonal wavelet frame (BWF) The X-Y directional amplitude Xd, Yd

The distance in texture

T

T

Q

QTTQ Xd

Yd

Xd

Ydd ,

14

Feature extraction Normalized Area

NPQ =

(Size of the region) / (Size of the image)

TQNArea

TQ NPNPd ,

15

Feature extraction Shape and Location

The global geometric shape feature eccentricity

Estimate the bounding rectangle for each segmented region

For the major axis Rmax and minor axis Rmin

max

min

max

min

T

T

Q

Q

R

R

R

RE

16

Feature extraction Shape and Location

The local geometric shape feature MRS (modified radius-based shape signature)

invariant under shape’s scaling, rotation, and translation

17

Feature extraction Shape and Location

The local geometric shape feature MRS (modified radius-based shape signature)

Extracts 12 radius distance values

2

1 1

,

11

,min

N

i i

iN

j j

jC

ckwisecountercloclockwiseMRS

TQ

T

Q

T

Q

NNd

ddd

18

Stepwise Similarity Matching

r

i

p

j

ij

ijjj tqDwYXSim

1 1

,,

19

Experimental Results

query: flower best case

20

Experimental Results

query: shipworst case

21

22