IMAGE SLICING AND STATISTICAL LAYER APPROACHES FOR CONTENT...

IMAGE SLICING AND STATISTICAL LAYER APPROACHES FOR

CONTENT-BASED IMAGE RETRIEVAL

By

JEHAD QUBIEL ODEH AL-NIHOUD

Thesis Submitted to the School of Graduate Studies, Universiti Putra Malaysia, in

Fulfilment of the Requirements for the Degree of Doctor of Philosophy

December 2004

ii

To My Family with My Love

iii

Abstract of thesis presented to the Senate of Universiti Putra Malaysia in fulfilment of the

requirement for the degree of Doctor of Philosophy

IMAGE SLICING AND STATISTICAL LAYER APPROACHES FOR

CONTENT-BASED IMAGE RETRIEVAL

By


December 2004

Chairman: Associate Professor Hajah Fatimah Ahmad, Ph.D.

Faculty: Computer Science and Information Technology

Two new approaches for colour features representation and comparison in digital images

to handle various problems in the field of content-based image retrieval are proposed.

The first approach is a double-layered system utilising a new technique, which is based

on image slicing, combined with statistical features extracted and compared in each layer

(ISSL). The images database is filtered in the first layer based on the similarities of

brightness compared with the query image and ranked in the second layer, based on the

similarities of the contrast values between the query image and the set of candidate

images retrieved through the first layer. Although different distance measurements are

available, the city block known as L1-norm distance measurement is used. This is due to

its speed efficiency and accuracy. Different experiments are applied to different database

sets, containing different number of images. The results show that the approach is

scalable to the varying size of the database, robust, accurate, and fast. A comparison

iv

between the colour histogram approach and the proposed approach shows that the

proposed system is more accurate and the speed of performance is much better. A new

paradigm to choose the proper threshold value is proposed based on the autocorrelation

of the distance vector. Moreover, an image retrieval system based on entropy as a visual

discriminator is developed and compared with ISSL. The results show that the proposed

ISSL approach is able to achieve better precision and reaches higher recall levels as

compared with entropy approach.

The second proposed technique for colour based retrieval is the Eigenvalues approach.

Findings show that the interpretation of the Eigenvalues, as identity or signature for the

square matrix, makes it possible to map this concept to the different bands of the image.

The approach relies on calculating the accumulative distances between the query image

and the images database, using the accumulative Eigenvalues of each band. The approach

is tested, using different image queries over different database sets and the results are

promising. Furthermore, the proposed approach is compared with ISSL approach and

entropy approach, using different query images over a database set of 2000 images.

In addition, a shape-based retrieval system is proposed. The system is double-layered, in

which the first layer is used to filter the images database based on colour similarity. This

allows the reduction in the number of candidate images, which need to be manipulated,

using the shape retrieval technique in the second layer. The technique utilises a low-level

image processing operations with “Dilate” as a morphological operator. Laplacian of

Gaussian (LoG) is used to smoothen and detect the edges of the objects. Dilate on the

v

other hand is used to solidify the object and fill in the holes, and correlation coefficient is

proposed as a new means to shape similarity measurement. Experiments show that the

approach is fast, flexible, and the retrieval of images is highly accurate. It is also able to

overcome the numerous problems that are associated with the usage of the low-level

image processing operation in image retrieval.

vi

Abstrak tesis yang dikemukakan kepada Senat Universiti Putra Malaysia sebagai

memenuhi keperluan untuk ijazah Doktor Falsafah

PENDEKATAN PENGHIRISAN IMEJ DAN LAPISAN BERSTATISTIK UNTUK

DAPATAN SEMULA IMEJ BERASASKAN KANDUNGAN

Oleh


Disember 2004

Pengerusi: Profesor Madya Hajah Fatimah Ahmad, Ph.D.

Fakulti: Sains Komputer dan Teknologi Maklumat

Hasil kajian ini mencadangkan dua pendekatan untuk perwakilan dan perbandingan ciri-

ciri warna di dalam imej digital bagi menangani pelbagai masalah di dalam bidang

dapatan semula imej berasas kandungan. Pendekatan pertama ialah penggunaan teknik

baru iaitu sistem lapisan berganda. Sistem ini berasaskan penghirisan imej iaitu

penggunaan gabungan ciri-ciri statistik dan perbandingan di dalam setiap lapisan (ISSL).

Ianya dilakukan dengan menapis imej-imej dalam pangkalan data dalam lapisan pertama

berdasarkan kepada kesamaan kecerahan sesuatu imej dan dibandingkan dengan imej

carian dan dikelaskan di dalam lapisan kedua. Pengkelasan ini adalah berdasarkan kepada

kesamaan dalam nilai keterangan bagi sesuatu kecerahan imej di antara imej carian dan

set imej-imej capaian terpilih melalui lapisan pertama.

vii

Pendekatan ini juga menggunakan teknik pengukuran L1-norm sebagai pengukuran jarak

disebabkan oleh kecekapan dan ketepatan kelajuannya dalam membuat penapisan dan

perbandingan imej-imej. Pelbagai kajian dijalankan ke atas set-set pangkalan data yang

berbeza yang mengandungi bilangan imej yang berlainan. Hasil kajian ini menunjukkan

bahawa pendekatan ini bersesuaian digunakan untuk pelbagai saiz pangkalan data dimana

ianya, teguh, tepat dan pantas dalam menghasilkan dapatan semula imej. Perbandingan

juga dibuat di antara penggunaan pendekatan histogram warna dan pendekatan yang

dicadangkan dan ianya menunjukkan bahawa sistem yang dicadangkan lebih tepat dan

prestasi kelajuannya adalah lebih baik. Paradigma baru juga digunakan untuk memilih

nilai ambang yang dicadangkan berdasarkan kepada autokorelasi bagi jarak vektor.

Sistem berasaskan dapatan semula imej ke atas entropi sebagai visual diskriminator juga

dibangun dan dibandingkan dengan ISSL dan hasil daripada ini ianya menunjukkan

bahawa pendekatan ISSL yang dicadangkan boleh digunakan untuk mencapai ketepatan

yang lebih baik dan mencapai tahap perolehan yang lebih tinggi berbanding dengan

pendekatan entropi.

Teknik kedua yang dicadangkan untuk dapatan semula berasas warna adalah berdasarkan

pendekatan Eigenvalues. Hasil kajian menunjukkan interprestasi Eigenvalues sebagai

identiti atau pengenalan untuk matriks segiempat sama bagi membolehkan ianya memeta

konsep ini ke atas jalur-jalur imej yang berbeza. Pendekatan ini bergantung kepada

perkiraan jarak terkumpul di antara imej carian dan imej-imej di dalam pangkalan data

menggunakan Eigenvalues yang terkumpul bagi setiap jalur. Pendekatan ini diuji dengan

menggunakan carian imej yang berbeza ke atas set-set pangkalan data yang berlainan dan

viii

hasilnya menunjukkan ianya adalah lebih tepat. Pendekatan ini juga dibandingkan dengan

pendekatan ISSL dan pendekatan entropi dengan melakukan ujian yang menggunakan

imej carian yang berbeza di dalam set pangkalan data yang mempunyai 2000 imej.

Kajian ini juga mencadangkan sistem dapatan semula berasas bentuk. Sistem ini

mempunyai dua lapisan di mana lapisan pertama digunakan untuk menapis imej-imej

dalam pangkalan data berdasarkan kepada kesamaan warna pada imej tersebut. Ini akan

menyebabkan penurunan dalam bilangan imej terpilih yang perlu dimanipulasi dengan

menggunakan teknik dapatan semula bentuk di dalam lapisan kedua. Teknik ini

menggunakan operasi pemprosesan imej bertahap rendah dengan ‘Dilate’ sebagai

operator morfologi. Laplacian of Gaussian (LoG) pula digunakan untuk melicinkan dan

mengesan sempadan-sempadan bagi sesuatu objek manakala ‘Dilate’ digunakan untuk

memadatkan dan mengisi objek-objek ini ke dalam lubang-lubang dan pekali korelasi

digunakan sebagai satu kaedah untuk mengukur kesamaan bentuk. Kajian menunjukkan

pendekatan ini lebih pantas, fleksibel dan dapatan semula imejnya lebih tepat. Ianya juga

boleh menyelesaikan pelbagai masalah yang melibatkan penggunaan operasi

pemprosesan imej bertahap rendah di dalam bidang dapatan semula imej.

ix

ACKNOWLEDGEMENT

In the name of ALLAH, The Most Gracious, and Most Merciful

This thesis would not have been possible without the help of many who have contributed

one way or another in its creation. While it is difficult to acknowledge them all, I would

like to express my thanks and appreciation to whomever contributes to the success of this

research work.

However, I would like to convey my sincere thanks and deepest appreciation to my

supervisory committee, Assoc. Prof. Hajah Fatimah Ahmad, Assoc. Prof. Mohamed

Othman, and Dr. Rozita Johari for their help, guidance, patience and encouragement

rendered throughout the different phases of this research.

I would also like to convey my thanks to the Faculty of Computer Science and

Information Technology, the Sultan Abdul Samad Library and Universiti Putra Malaysia

for assisting me in my study.

I express my warmest gratitude to my mother and the soul of my late father, who never

let me believe I could not succeed in this work or in any other challenges in my life. Also,

special thanks to my wife, brothers, and sisters, for their prayers, love and encouragement

during my study. Finally, I am grateful to all of my friends, Atheer Abas, Mueen Ahmad,

Belal Yousf, Nbhan Salih, and Hakim Mellah for their support.

x

I certify that an Examination Committee met on 2nd

December 2004 to conduct the final

examination of Jehad Qubiel Odeh, on his Doctor of Philosophy thesis entitled “Image

Slicing and Statistical Layer Approaches for Content-Based Image Retrieval” in

accordance with Universiti Pertanian Malaysia (Higher Degree) Act 1980 and Universiti

Pertanian Malaysia (Higher Degree) Regulations 1981. The Committee recommends that

the candidate be awarded the relevant degree. Members of the Examination Committee

are as follows:

MOHD HASAN SELAMAT, M.Phil.

Associate Professor

Faculty of Computer Science and Information Technology

University Putra Malaysia

(Chairman)

RAMLAN MAHMOD, Ph.D.

Associate Professor



(Member)

RAHMITA WIRZA, Ph.D.



(Member)

TENGKU MOHD TENGKU SEMBOK, Ph.D.

Professor

Faculty of Information Science and Technology

University Kebangsaan Malaysia

(Independent Examiner)

___________________________________

ZAKARIAH ABD RASHID, Ph.D.

Professor/Deputy Dean

School of Graduate Studies

Universiti Putra Malaysia

Date:

xi

This Thesis submitted to the Senate of Universiti Putra Malaysia and has been accepted

as fulfilment of the requirement for the degree of Doctor of Philosophy. The members of

the Supervisory Committee are as follows:

HAJAH FATIMAH AHMAD, Ph.D.

Associate Professor



(Chairman)

MOHAMAD OTHMAN, Ph.D.

Associate Professor



(Member)

ROZITA JOHARI, Ph.D.



(Member)

_____________________________

AINI IDERIS, Ph.D.

Professor/Dean

School of Graduate School


Date:

xii

DECLARATION

I hereby declare that the thesis is based on my original work except for quotations and

citations, which have been duly acknowledged. I also declare that it has not been

previously or concurrently submitted for any other degree at UPM or other institutions.

________________________________

JEHAD QUBIEL ODEH ALNIHOUD

Date:

xiii

TABLE OF CONTENTS

Page

ABSTRACT iii

ABSTRAK vi

ACKNOWLEDGEMENTS ix

APPROVAL x

DECLARATION xii

LIST OF TABLES xvii

LIST OF FIGURES xix

LIST OF ABBREVIATIONS xxv

CHAPTER

1 INTRODUCTION 1

1.1 Problems Statement 3

1.2 Proposed Approaches 6

1.3 Research Objectives 7

1.4 Scope of the Research 8

1.5 Main Contributions 8

1.6 Thesis Structure 9

2 LITERATURE REVIEW 12

2.1 Introduction 12

2.2 Content-Based Image Retrieval (CBIR) 12

2.3 Colour-Based Retrieval 17

2.3.1 Colour Histogram Approach 17

2.3.2 Colour Pair Segmentation 20

2.3.3 Entropy as a Visual Feature for Colour-Based Retrieval 22

2.4 Shape-Based Retrieval 23

2.4.1 Global Shape Feature-Based Methods 24

2.4.2 Region-Based Querying Methods 25

2.4.3 Region Graph-Based Image Retrieval 26

2.4.4 Model-Based Querying 28

2.5 Commercial Systems 29

2.6 Interfacing 30

2.7 Summary 32

3 METHODOLOGY AND SYSTEM DESIGN 34

3.1 Introduction

3.2 Prototyping

3.2.1 Evolutionary Development Process

34

35

37

xiv

3.2.2 Pilot System

3.3 Proposed Framework and Methodology

3.4 Similarity Measurements and Evaluation Techniques

3.4.1 Similarity Measurements

3.4.2 Evaluation Techniques

3.4.3 Threshold

3.4.4 Data

3.5 Summary

38

39

43

43

45

46

47

47

4 COLOUR RETRIEVAL MODEL BASED ON IMAGE SLICING

AND STATISTICAL LAYERS (ISSL)

49

4.1 Introduction 49

4.2 System Design and Implementation 49

4.2.1 The Interface 49

4.2.2 System Architecture 50

4.3 Image Slicing Technique to Preserve the Spatial Similarity 51

4.4 Definitions and Algorithms 53

4.5 Colour Spaces Comparison using ISSL 57

4.6 ISSL Evaluation 65

4.6.1 Experimental Setup 65

4.6.2 Average Recall versus Average Precision Testing 66

4.7 Colour Histogram Approach 86

4.8 ISSL versus Colour Histogram Approach 87

4.8.1 Experimental Setup 87

4.8.2 Recall and Precision of ISSL versus Colour Histogram 89

4.8.3 Ranking Accuracy of ISSL versus Colour Histogram 92

4.8.4 City Block versus Euclidean Distance Measures using

Colour Histogram

95

4.8.5 Run-Time Performance of ISSL versus Colour Histogram 101

4.9 Discussion 102

4.10 Summary 103

5 ISSL VERSUS ENTROPY AS A VISUAL DISCRIMINATOR

FOR COLOUR-BASED IMAGE RETRIEVAL

105

5.1 Introduction

5.2 Entropy Approach Overview

5.3 Definition and Algorithms

5.4 Experimental Setup

5.5 Experiments

5.5.1 Average Precision of Entropy-Based Similarity (Database Set

200 Images)

5.5.2 Average Precision of Entropy-Based Similarity (Database

Set 600 Images)

105

106

108

110

110

110

114

xv

5.5.3 Average Precision of Entropy-Based Similarity (Database

Set 1000 Images)

5.5.4 Entropy versus ISSL (Database Set 2000 Images)

5.5.5 Run-Time Performance of Entropy versus ISSL

5.6 Discussion

5.7 Summary

119

123

130

131

132

6 EIGENVALUES AS A STATISTICAL FEATURE

DISCRIMINATOR FOR COLOUR-BASED IMAGE

RETRIEVAL

133

6.1 Introduction

6.2 Eigenvalues Approach Overview

6.3 Eigenvalues Definition and Algorithms


6.5 Experiments

6.5.1 Average Precision of Eigenvalues-Based Similarity

(Database Set 200 Images)





6.5.4 Eigenvalues versus ISSL and Entropy (Database Set 2000

Images)

6.5.5 Run-Time Performance of Eigenvalues-Based Similarity

6.6 Discussion

6.7 Summary

133

134

135

138

139

139

143

146

151

156

157

158

7 INTEGRATION OF ISSL APPROACH FOR COLOUR

RETRIEVAL AND SHAPE APPROACH BASED ON

LOW LEVEL IMAGE PROCESSING OPERATIONS

159

7.1 Introduction

7.2 System Design

7.2.1 System Architecture

7.2.2 The Interface

7.3 Neighbourhood Operations

7.4 Mar/Hildreth Criteria for Edge Operators

7.5 Edge Detection Operators

7.5.1 Laplacian

7.5.2 Laplacian of Gaussian

7.6 Morphology

7.6.1 Structuring Elements

7.6.2 Dilate

159

160

160

160

161

162

162

162

164

166

166

167

xvi

7.7 Matching by Correlation Coefficient

7.8 Proposed Shape Retrieval Algorithms (SRAs)


7.10 Experiments

7.10.1 Retrieve Similar Images

7.10.2 Retrieve Less Images

7.10.3 Retrieve More Images

7.10.4 Experimental Results

7.10.5 Run-Time Performance

7.11 Discussion

7.12 Summary

8 CONCLUSION AND FUTURE WORK 8.1 Results Summary

8.2 Future Studies

8.2.1 Future Directions to Improve the Colour-Based Approaches

8.2.2 Future Directions to Improve the Shape-Based Approach

REFERENCES

APPENDICES

BIODATA OF THE AUTHOR

168

172

174

177

177

182

187

194

196

197

198

200

200

203

203

204

205

215

235

xvii

LIST OF TABLES

Table

Page

4.1 Recall and Precision (ISSL, Database Set 200 Images)










4.11 Overall Recall and Precision of ISSL

4.12 Number of Relevant Images to Queries (ISSL versus Colour

Histogram)

4.13 The Recall & Precision of ISSL versus Colour Histogram (Database

Set 2000 Images)

4.14 Speed of Performance of City Block versus Euclidean (Colour

Histogram Approach)

4.15 Speed of Performance of the ISSL versus Colour Histogram

5.1 Number of Relevant Images to Queries (Entropy, Database Set 200

Images)

5.2 Average Precision (Entropy, Database Set 200 Images)


Images)

68

69

70

71

72

73

74

75

76

77

78

89

90

97

102

112

112

116

xviii



Images)


5.7 Number of Relevant Images to Queries (Database Set 2000 Images)


5.9 Average Precision (ISSL, Database Set 2000 Images)

5.10 Run-Time Testing of Entropy versus ISSL

6.1 Number of Relevant Images to Queries (Eigenvalues, Database Set 200

Images)

6.2 Average Precision (Eigenvalues, Database Set 200 Images)

6.3 Number of Relevant Images to Queries (Eigenvalues, Database Set 600

Images)


6.5 Number of Relevant Images to Queries (Eigenvalues, Database Set

1000 Images)



6.8 Run-Time Testing of Eigenvalues versus ISSL and Entropy

7.1 LoG Mask

7.2 Number of Relevant Images to Queries (Shape Approach, Database Set

1000 images)

7.3 Recall versus Average Precision ( ε <= 0.30)



7.6 Run-Time Testing (Shape Approach)

116

120

120

124

124

125

131

140

140

143

144

147

148

152

156

165

176

182

187

194

197

xix

LIST OF FIGURES

Figure

Page

3.1 Traditional Image Retrieval Model

3.2 Prototyping in Software Development Process

3.3 Iterative Life Cycle

3.4 Proposed Image-Based Retrieval Framework

3.5 Colour Prototypes Creation Based on Incremental Development

Process and Iterative Enhancement Method

3.6 Pilot System of the Proposed Colour and Shape-Based Image Retrieval

Approaches

4.1 Interface of ISSL

4.2 System Overview of ISSL

4.3 Slicing Image to Four Subareas

4.4 Testing the Effectiveness of Slicing Technique

4.5 ISSL1 Algorithm

4.6 ISSL2 Algorithm

4.7 RGB Colour Space

4.8 HSV Colour Space

4.9 Query Image to Test Colour Spaces using ISSL

4.10 Retrieved Set (RGB Colour Space)

4.11 Retrieved Set (YCbCr Colour Space)

4.12 Retrieved Set (HSV Colour Space)

4.13 Colour Spaces Comparison (Recall versus Precision)

35

36

38

40

42

43

50

51

52

53

55

57

58

59

60

61

62

63

64

xx

4.14 Query Images (ISSL, Database Set 200 Images)










4.24 Regression between Database Size and Precision

4.25 Regression between Database Size and Recall

4.26 Sample Retrieved Set (ISSL Approach, Database Set 200 Images)










4.36 Colour Histogram Values Extraction

68

69

70

71

72

73

74

75

76

77

79

80

81

81

82

82

83

83

84

84

85

85

86

xxi

4.37 Query Images (ISSL versus Colour Histogram)

4.38 Sample1 of Retrieved Set (Colour Histogram Approach)



4.41 Query Image (Image Index 327)

4.42 Retrieved Set (ISSL)

4.43 Retrieved Set (Colour Histogram)

4.44 Recall versus Precision for ISSL and Colour Histogram

4.45 Query Image (Euclidean versus City Block)

4.46 Retrieved Set using Euclidean Distance Measure

4.47 Retrieved Set using City Block Distance Measure

5.1 System Overview (Entropy)

5.2 Entropy Prototype Interface

5.3 ENDA Algorithm

5.4 ENIQA Algorithm

5.5 Query Images (Entropy, Database Set 200 Images)

5.6 Recall versus Average Precision (Entropy, Database Set 200 Images)

5.7 Sample Retrieved Set (Entropy, Database Set 200 Images)






88

91

91

92

93

93

94

95

98

99

100

107

108

109

109

111

113

114

115

117

118

119

121

xxii


5.14 Query Images (Entropy versus ISSL, Database Set 2000 Images)

5.15 Recall versus Average Precision (Entropy versus ISSL)

5.16 Retrieved Set for Query 6 (Entropy, Database Set 2000 Images)

5.17 Retrieved Set for Query 4 (Entropy, Database Set 2000 Images)

6.1 System Overview (Eigenvalues Approach)

6.2 The Interface (Eigenvalues)

6.3 EVFEA1 Algorithm

6.4 EVFEA2 Algorithm

6.5 Query Images (Eigenvalues, Database Set 200 Images)

6.6 Recall versus Average Precision (Eigenvalues, Database Set 200

Images)

6.7 Sample Retrieved Set (Eigenvalues, Database Set 200 Images)



Images)




Images)

6.13 Sample Retrieved (Eigenvalues, Database Set 1000 Images)


6.15 Recall versus Average Precision (ISSL, Entropy, and Eigenvalues)


6.17 Eigenvalues Performance

122

123

126

128

130

134

135

136

137

139

141

142

143

144

146

147

148

150

151

153

155

157

xxiii

7.1 System Overview (Shape Prototype)

7.2 The Interface (Shape Prototype)

7.3 Laplacian Masks (Kernels)

7.4 Dilate Process

7.5 Dilate using Square Structuring Element

7.6 Correlation of f(x, y) and w(x, y) at the Point (s, t)

7.7 SRA1 Algorithm

7.8 SRA2 Algorithm

7.9 Set of Query Images (Shape Approach)

7.10 Result Set of Query 1 ( ε<= 0.30)


7.12 Result set of Query 3 ( ε<= 0.30)












160

161

164

167

168

169

172

174

174

177

178

179

179

180

180

181

183

183

184

184

185

185

186

xxiv








7.31 Recall versus Average Precision (Shape Approach)

188

189

190

191

192

192

193

195

IMAGE SLICING AND STATISTICAL LAYER APPROACHES FOR CONTENT...

Documents

Transcript of IMAGE SLICING AND STATISTICAL LAYER APPROACHES FOR CONTENT...