FEATURE SET PRUNING FOR CBIR SYSTEMS WITH … · FEATURE SET PRUNING FOR CBIR SYSTEMS WITH...
170
FEATURE SET PRUNING FOR CBIR SYSTEMS WITH APPLICATIONS TO VISUAL ART IMAGES A thesis submitted to Vinayaka Missions University for the award of the degree of DOCTOR OF PHILOSOPHY IN COMPUTER SCIENCE By Y. POORNIMA Under the Guidance of Prof. Dr. T. Arunkumar Ph.D VINAYAKA MISSIONS UNIVERSITY SALEM, TAMILNADU, INDIA JULY 2015
Transcript of FEATURE SET PRUNING FOR CBIR SYSTEMS WITH … · FEATURE SET PRUNING FOR CBIR SYSTEMS WITH...
FEATURE SET PRUNING FOR CBIR SYSTEMS WITH APPLICATIONS TO VISUAL ART IMAGES
A thesis submitted to Vinayaka Missions University
for the award of the degree of
DOCTOR OF PHILOSOPHY IN COMPUTER SCIENCE
By
Y. POORNIMA
Under the Guidance of
Prof. Dr. T. Arunkumar Ph.D
VINAYAKA MISSIONS UNIVERSITY
SALEM, TAMILNADU, INDIA
JULY 2015
DECLARATION
I . hereby declare that the thesis entitled “Feature set pruning for
CBIR Systems with applications to visual Art images”
submitted by me to Vinayaka Missions University for the Degree of
Doctor of Philosophy in Computer Science is the record of work
carried out by me under the guidance of Dr. T. Arunkumar and that
this has not previously formed the basis for the award of any
degree, diploma, associate-ship, fellowship or other titles in this
University or similar institution of higher learning.
Place: Signature of the Candidate
Date:
Dr.T. Arunkumar Professor, School of Computing Science & Engineering VIT, Vellore-632014 Tamil Nadu, India
CERTIFICATE BY THE GUIDE
This is to Certify that the thesis entitled ”Feature set pruning
for CBIR Systems with applications to visual Art images”
Submitted by Ms. Y. POORNIMA, to Vinayaka Missions University
for the Degree of Doctor of Philosophy in Computer Science is
the record of Research Work carried out by her under my
guidance and supervision and that this work has not formed the
basis for the award of any degree, diploma, associate-ship,
fellowship or other titles in this University or similar institution of
higher learning and it represents wholly her independent.
Place: Signature of the Supervisor with designation
Date:
ACKNOWLEDGEMENT
I take this opportunity to thank my beloved guide well wisher and mentor
Dr.T. Arunkumar Ph.D., professor department of computer science and
engineering, VIT, Vellor, Tamil nadu, India, for his valuable untiring and
continuous guidance during the tenure of research work. I am really
indebted to my guide and seek the same support in my future endeavous
without his guidance and encourage this dissertation would not have
come out in this shape. Dr. P.S Hiremath professor Gulbarga university
, Gulbarga. I acknowledge the constant support and constructive
discussions and suggestions throughout my research work.
I’m thankful to Vinayaka Missions University Salem, Tamilnadu.
Towards the early completion of my research.I express my regards to of
chancellor Dr.Ganeshan.E vice chancellor Dr. Rajendran and Prof.
Dr. K. Rajendran,Ph.D., Dean (Research) of Vinayaka missions
university salem, tamilnadu. Dr. Hanumanthappa, Dr. Ramegowda,
Mr.Chandra Bhusan, ravi Shankar, ravi, Amith , Sridhara Bangalore for
the continuous encouragement and concern about the progress of my
research.,
I expresss my greatfulness to Dr.Abdul roof., Dr.Diwakar Dr.Murthy
manasgangothri University of Mysore, Mysore. They all motivated me to
initiate research work and made me enroll for ph.D.degree and
providing all facilities and encouragement. I am extremely greateful to
Dr.N.U Bhajantri professor department of computer science and
engineering Chamarajanagar. Dr.Prahalada rao, M.s Rekha senior
lecturer department of biology vidyavardhaka college Mysore. This words
of appreciation and encouragement have always helped me to move
forward even in adverse circumstances.
I express my sincere thanks to Dr.Nagaraj murthy principal Maharaja
college Mysore, Dr.Vasanth Adminstrative officier Maharaja college
Mysore, Smt .Anita Vimala braggs, Asstiant professor Maharaja college
Mysore, Smt .Sana Gaythri former principal Maharaja college Mysore,
who have supported me during research work.
I express my special thanks to Dr.Ramanaryan principal evening college
Mysore, G.K.Anand former professor Maharaja college Mysore, priya
English professor Coimbatore who helped me during research work.
I express my thanks to Dr.T. Arunkumar sir wife Smt. karthiga.G.R
helped and took care during research work thank you for her hospitally.
I express my thanks to my friends like Ramesh, Ramesh, Prasad,
Nagalakshmi, Sarala, Ravindra, Lokesh, Rajendra, Shivamma,
Chandrashekar, Chethan kumar, Aravinda, Narendra, Mamatha,
Niveditha, Geetha, Meenakshi, Sureshkumar, Kavya, Shurthi, Priya who
helped me during research work .
I express my thanks to Mr. Vydhiyanathan deputy registrar University of
Mysore, Mysore and his wife Vanitha and their sons
I express my special thanks to Mr.Suresh accounts officier Bangalore
his wife Geetha their daughter Sunidhi, Smt. Sumati R Gowda, Ganesh
babu, Shabana all my friends , cousins brother Anand raj and family and
relatives and others who directly and indirectly helped in carry out my
research work.
I am extremely grateful to my family members, parents ands guardians
Late.Lakshmanan, Smt. Lakshmamma, Yathiraj, Rajeshwari,
Kannyakumari, Parthasarthi, Vani, Jayalakshmi, Anand,
Sathyanaryana, Sandhya, sisters, Shobha, Vandana, Dhakshitha,
brothers late Yogesh kumar, master Chiras. without their support this
project would not been successful .
Y.POORNIMA
1
TABLE OF CONTENTS
LIST OF FIGURES 6
LIST OF TABLES 8
LIST OF ABBREVIATIONS 9
ABSTRACT 10
Chapter 1 Introduction 11
1.1 Introduction 11
1.2 Challenges of Image Retrieval Systems 14
1.3 Query Formulation challenge 17
1.3.1 Image Pattern Recognition Methods 18
1.3.2 Supervised Methods 19
1.3.3 Unsupervised Methods 20
1.4 Problem Description 22
1.5 Research Aim and Objectives 24
1.6 Organization of Thesis work 25
1.7 Summary 26
Chapter 2 Review Of Literature 27
2.1 Introduction 27
2.2 Techniques Of CBIR 29 2.2.1 Feature Extraction Techniques 29
2.2.2 Boundary Detection Feature Extraction 30
2.2.3 Color Averaging Techniques for Feature Extraction 30
2.2.4 Gradient and 12 Directional Feature Extraction 31
2.2.5 Feature Extraction using Slope Magnitude Technique 31
2.2.6 Feature Extraction using Transforms 32
2.2.7 Feature Extraction using Fast Fourier Transform 33
2.2.8 Feature Extraction using Walsh Transform 33
2.2.9 Similarity Measurement 34
2
2.2.10 Euclidean Distance 36
2.2.11 Mean Square Error 37
2.2.12 Sum of Absolute Differences 38
2.2.13 Neural Network Classifiers 38
2.2.14 K-Nearest Neighbor Algorithm 38
2.3 Summary 42
Chapter 3 Need For The Study 43
3.1 Introduction 43
3.2 Need of Smart Database Building 44
3.3 Need To Overcome Challenges In Visual Art Images 46
3.4 Need To Enhance CBIR Process 48
3.5 Summary 49
Chapter 4 CBIR For Visual Art Images 50
4.1 Introduction 50
4.2 The Process Of Image Retrieval 50 4.2.1 Text-Based Retrieval 51
4.2.2 Content-Based Image Retrieval (CBIR) 51
4.3 The Image Domain 53
4.4 Image Proporties 55 4.4.1 Color 56
4.4.2 Shape 58
4.4.3 Texture ` 58
4.4.4 Interest Points 59
4.4.5 Image Retrieval 59
4.5 Content Based Image Retrieval 62 4.5.1 Database Content 63
4.5.2 Query Form 64
4.5.3 Image Description 64
4.5.4 Interaction between User and CBIR System 65
4.6 The Importances Of Images Retrieval 66
3
4.7 Database Contents 68 4.7.1 Producing a Sketch 71
4.7.2 Drawing Edges 71
4.7.3 Predefined Elements 72
4.7.4 Geometric Shapes 72
4.7.5 How Much Information do we need? 72
4.7.6 Complete Sketches 74
4.7.7 Incomplete Sketches 75
4.7.8 Extracting Comparable Information from Sketches and
Database Images 77
4.7.9 Extracting the Information from Geometric and
Freehand Sketches 78
4.8 Image Features For CBIR 79
4.8.1 Color 79
4.8.2 Find it 80
4.9 Summary 81
Chapter 5 Content Image Retrieval Approach (COIR)-(COIR SCHEME) 83
5.1 Introduction 83
5.2 Research Methodology 84
5.3 Functional Dependences 85
5.4 COIR Model : COIR 94
5.5 Summary 96
Chapter 6 Implementation of COIR 97
6.1 Introduction 97
6.1.1 mage Feature Representation 99
` 6.1.2 Classifier Combination 103
6.1.3 Image Filtering 104
6.1.4 Similarity Fusion 105
6.15 Algorithm using COIR 107
6.3 Summary 107
4
Chapter 7 Results And Discussions 108
7.1 Results Accomplished From COIR Model 109
7.2 Comparative Performance Analysis 117 7.2.1 Outcomes of Individual Approaches 120
7.2.1.1 Outcomes of Contribution-based Approach 120
7.2.1.2 Outcomes of Block-Truncation Coding and
K-Means Approach 123
7.2.1.3 Outcomes of K-Means Approach 125
7.3 Summary 132
Chapter 8 Conclusion and Future Work 133
8.1 Conclusion 133
8.2 CBIR Of Visual Art Images UsingContribution 137
8.3 Possible Enhancement 141
8.4 Future Work 143
References 145
List Of Publications 164
5
LIST OF FIGURES Chapter Figure Title Page No Abstract Figure A-Research work Organization and
structure of thesis work
1
Figure 1.1: Simplified view of CBIR query processing
13
Figure1.2: Challenges of Content Based Image Retrieval
14
Figure1.3: Structural similarities between a black-and-white drawing of a dolphin and an image of a banana
16
2 Figure 2.1:Flow of CBIR System 28
4
Figure 4.1: Image retrieval methods 61
Figure.4.2: The different steps in CBIR 62
Figure.4.3: The three kinds of data that can be associated to an image: semantic (in this case a natural language description and keywords), primitive (for example, contours and a segmentation) and factual (name of the photographer, date and place of the shoot, film and camera used).
65
Figure.4.4:-Visual art images 69
5 Figure 5.1:Schematic diagram of the COIR system 85
6 Figure 6.1: Partitioning in Visual art image using CLD 102
Figure 7.1: Visual Outcomes after applying Visual Descriptors 110
Figure 7.2: Visual Outcomes of Retrieved images with Ranking 113
Figure 7.3: Outcome of Relevance Feedback Processing
115
Figure 7.4: Final Outcome of the COIR Retrieval System
116
6
7
Figure 7.5: Visual Outcomes of Contribution-based approach
121
Figure 7.6: Visual Outcomes of BTC & K-Means approach
123
Figure 7.7: Visual Outcomes K-Means approach
125
Figure 7.8:Summary of the Precision and Recall for Existing System
127
Figure 7.9: Recall & Precision Analysis
129
7
LIST OF TABLES
Chapter Title Page No 7
Table 7.1: Summary of Adopted Techniques for Comparative Analysis
119
Table 7.2: Result for Contribution based Feature Set Pruning
122
Table 7.3:Numercal Outcomes of Block Truncation Coding Approach
124
Table 7.4: Numerical Outcomes of K-Means Approach.
126
Table 7.5: Summary of Processing Time and Accuracy
128
8
LIST OF ABBREVIATIONS CBIR - Content Base Image Retrieval
ANN - Artificial neural network
SVMs- Support vector machines
K-NN - K-nearest neighbor classifier
FCM - Fuzzy c-means
ICA - Independent Component analysis
SOM - Self organsing map
ART - Adaptive resonance theory
PCA - Principal Component analysis
ED - Euclidean distance
BTC - Block truncation coding
GPU - Graphic processing unit
CUDA - Computer unified device Architecture
BTDL - Bulgarian iconographical digital library
RGB - Red green blue
HSV - Hue saturation value
CMYK - Cyen magenta yellow key
SPCA - Shift invariant principal component analysis
DDL - Descriptor definition language
ART - Angular radial transform
EHD - Edge histogram descriptor
CLD - Color layout descriptor
9
ABSTRACT
The proposed study introduces a novel concept of performing feature
pruning process with an aid of Content based image retrieval process
(CBIR). The concept of CBIR technique has been implemented by
various researchers in the past, however, there are few research work
explored in the area of visual art images. Adoption of visual art image is
one of the most challenging environments considered for the proposed
study as they are quite different from natural images. The study also
develops a design of an cost efficient database management system that
stores the features being extracted using two types of visual descriptors
e.g. color layout descriptors and edge histogram descriptors. The
features are than subjected to supervised learning algorithm and hence a
smart database for extracted features is designed. The next part of the
study has focused on evaluating the effectiveness of the proposed
system by passing down the queried image as input for image retrieval
process. The outcome of the study shows an efficient and reliable image
retrieval system with an aid of proposed feature pruning process.
10
Chapter
Numbers
Description
Chapter 1 Introduction
Chapter 2 Review Of Literature
Chapter 3 Need For The Study
Chapter 4 CBIR for Visual art images
Chapter 5 Content Image Retrieval Approach(COIR)-(COIR
Scheme)
Chapter 6 Implementation of COIR
Chapter 7 Results and Discussions
Chapter 8 Conclusion and Future Work
References
Figure A- Research work Organization and structure of thesis work
11
Chapter 1 Introduction
1.1 Introduction
With the proliferation of advance signaling technique, digital image
processing has played a significant role in various fields. At present, it is
possible to collect various types of complex images, perform user-
dependent or extensive investigational study to yield reliable outcomes
[47]. The application of digital image processing was widely seen in
solving the problem of image enhancement, image denoising, image
compression, object recognition, image encoding for security etc.
However, the proposed study revolves around the domain of Content-
based Image Retrieval process or commonly known as CBIR technique.
The study has considered the case of visual art images in the
examination process as it is found to be one of the most ignored and less
researched topics till date in digital image processing.
Various eminent authors in past have emphasized on exclusive
investigational study towards historical importance and raised up a need
to conserve the historical contents for better cultural preservation for
future generation. In this regards, digital image processing has played a
significant role in conserving and studying historical artifacts and
contents with its unique technology [48]. Various kinds of historical
12
paintings maintained in the museum or in other archives is required to be
subjected for the restoration process.
The proposed study aims to apply advanced digital image processing
technique on visual art images. However, the turning point of the
proposed study is that it is not meant for restoration process as digitizing
the historical images[35] is again an age old topic. The significant point
here is that after the visual art images are digitized, they are archived in
digital repositories and all the problems starts from here. As the visual art
images are just the digitized version of the historical paintings that used
various sorts of color formation which is extremely different that the
natural colors or RGB recognized by the computer system. This problem
gives rise to recognition as well as identification process, if CBIR is
implemented on the visual art images. Hence, although it is easy to
digitize and archive the visual images, but it is quite difficult to retrieve
the relevant images using CBIR techniques. At present [92], the
searching technique of image provided by Google just uses text-based
image search, which cannot be applied. Various forms of other
applications equivalent to object recognition, face detection etc, also
cannot be directly applied to the visual art image. The existing feature
extraction techniques can be applied to visual art images but reliability is
quite poor owing to the distinct colors of the visual arts[14]. Hence, in sort
ensuring the reliable retrieving of the relevant images in visual art is one
13
of the most challenging and less discovered ideas in digital image
processing.
Figure 1.1 presents a simplified view of a CBIR image search. The
pictorial representation shows that input to the CBIR system is
considered as images, where the proposed system considers visual art
images as the input images. The input image, which can be also termed
as queried image, interacts with the core CBIR process using its user
interface. The input image is compared with the database of CBIR
normally built by using training methods, which as an output gives the set
of most similar images matching with the queried input image.
.
Image request
Query formulation
Visual query interface
Visual query
Image retrieval system Image Collection
Interface of result
User
Figure1.1: Simplified view of CBIR query processing
14
1.2 Challenges of Image Retrieval Systems
Image of any format and any nature is always associated with various
challenges, when it comes to design an efficient retrieval system [6].
Some of the challenges of image retrieval systems are shown below:
1. The Query Interpretation Challenge
2. The Query Mismatch Challenge
3. The Media Mismatch Challenge
Image request
Visual query interface
Image retrieval system
Image Collection
(a) Challenge of query formulation
(b) Challenge of query interpretation
(c) Challenge of query and media mismatch
Figure1.2: Challenges of Content Based Image Retrieval
Figure 1.2 shows a simplified overview of a query process, along with
four challenging problems related to queries involving visual structures.
The first form of the challenge is called as user’s query specification
15
process. This challenge is associated to the process of query
specification. This phase has got problems e.g. absence of efficient tools,
no cut-edge skills, time consumption etc [50]. There is unwanted time
consumption owing to the visual query process which is abnormally more
than the text-based search techniques. Adoption of such issues normally
complicates the problems of query processing.
After the query is being formulated, it is very important that the CBIR
system [93] should be able to perform interpretation so that it is feasible
to perform an efficient processing of the queries. In case such
processing is not done effectively that it is quite impossible to ascertained
relevancy of the outcomes and thereby yields certain outcomes which
are completely irrelevant. Even in the absence of an appropriate
segmentation process, the system fails to identify the structure of the
visual object. Interestingly, there are various components that is
responsible for performing the querying of the additional information
along with the queried object. However, CBIR techniques doesn’t merely
perform the searching and matching of the queried image with the
database image just simply with an aid of colors, edge, and other various
forms of features [51]. Hence, this step of interpreting the query is
second critical challenge in CBIR techniques.
16
Figure1.3: Structural similarities between a black-and-white drawing of a dolphin and an
image of a banana
The next critical issues associated with the existing system of CBIR is
that it is more dependent on performing similarity check based on the
structure, however, there are only few technique that uses semantics for
performing similarity check. Fig.1.3 shows the scenario of CBIR system
which is called as dolphin-banana problem. The figure shows that
queried image (dolphin) was found to match with some other objects
which are structurally similar (e.g. banana)[52]. This problem occurs
owing to no consideration of semantics in similarity check and therefore,
this problem of query-mismatch should be mitigated in any CBIR
techniques.
The ultimate issues associated with the CBIR technique is the matching
factor. The existing techniques of inter-media matching are quite
challenging to accomplish. Performing the visual query[53] with an aid of
linguistic approach is another bigger problem in CBIR technique that
17
makes the mechanism of cross checking the effectiveness of similarity
factor positively.
1.3 Query Formulation Challenge
As per various records, there are very less amount of evidence that could
support the potential usage of the visual query processing tool and also
there are very few investigational study towards CBIR interface design.
Although, there are various investigation towards the CBIR system
specifically focusing on the image retrieval system, but still there lies
various forms of challenging issues that are required to be solved [54].
Majority of the challenging issues are related to the relevancy check with
an aid of semantics over the visual query. Hence, there is only little
effective CBIR system that can be used efficiently by the user.
Designing of appropriate CBIR technique can be accomplished by
aggregating the empirical data about such problems and finding out how
a novel technique can improve the problems. First of all the area of visual
art has not been discovered much in the digital image processing
exclusively in the area of CBIR approach. The adoption of CBIR in visual
art images are also accompanied by challenges [55] e.g. query
interpretation processing and matching the query. However, the most
difficult challenge in implementing CBIR technique on visual art images
can be expressed as following:
18
• To find out the behavior of the user when requesting the visual art
images as a part of query processing.
• To identify the category of the visual art, when the visual art image
is considered as creation of the visual art database for learning
process.
• To identify the potential tool for performing the query processing
inspite of various problems associated with it.
1.3.1 Image Pattern Recognition Methods
The concept of pattern recognition is all about gathering and studying the
unique pattern of the image data normally using feature extraction
method that has the potential of performing sophisticated computation
[77] over symbolic of numeric data using various observational data.
Depending upon the availability of the pattern set, the segregation
process using various means of classifiers are deployed in image pattern
recognition process. Such collection of the statistical patterns are often
termed as training set of data and the yielding learning technique is
called as supervised learning algorithms. However, it is quite possible
that learning technique could also be unsupervised type, where the
system is not programmed for pre-defined labeling of specific patterns.
Therefore, if the pattern recognition technique implements supervised
learning technique, it can be termed as supervised pattern recognition
19
method, while if the technique applies on non-supervised learning
algorithm, it can be termed as unsupervised pattern recognition method.
1.3.2 Supervised Methods
This type of method is used for pattern recognition using supervised
learning algorithm and is widely adopted for classification techniques.
There are various forms of classifiers that perform pattern recognition
using supervised learning algorithms as follows:
1. Neural Network (Multilayer perception) - Neural Network and its
concept is derived from the potential characteristics of human brain
cells called as neurons. This technique is basically a concept of
mathematical modeling those targets to study any complex problems
with an aid of multilayer perceptions similar to human brain [15].
Majority of the complex problems is solved in neural network using
multilayer perceptions which is a sort of feed-forward algorithm.
Multilayer perception is a type of enhanced standard linear perception
with non-linear operations and is highly potential than the normal of
single layered perceptions.
2. Support Vector Machines – It is one of the frequently used
supervised learning algorithms that have the potential to classify both
linear and non-linear problems. SVM can perform classification of
both linear and non-linear data where it builds a system that can
perform prediction mechanism.
20
3. K-Nearest Neighbor Classifier – It is one of the famous classification
technique used on various complex problems and domains of study.
However, it is most heard in problems associated with pattern
recognition. In this process, the function is locally approximated and
hence it is also one of the simplest machine learning technique.
4. Naive Bayes – Bayes classifier is a concept design from Bayes’
theorem [39] by adopting potential or naïve independent assumptions.
It can also be termed as probability model using independent
features.
5. Decision tree – It is one the major implementation of standard graph
theory model, which is frequently used in machine learning and data
mining applications. It is basically a form of predictive framework that
maps various critical observations [62]. Decision tree is also called as
regression tree and classification techniques sometimes, where the
leaf nodes usually represents classification factor and edges
represents integration of features leading to the exclusive
classification.
1.3.3 Unsupervised Methods
The following are the unsupervised methods of pattern recognition
employing unsupervised learning:
1. Clustering – This process can be termed as the allocation of
specific set of observation into smaller groups called as clusters
[16]. Hence, clustering a mechanism adopted in unsupervised
21
learning method, where it is found applicable in various fields e.g.
data mining, machine learning, pattern recognition, analysis of
image etc. There are two frequently used clustering mechanism as
discussed below:
2. K-Means Clustering – It is one of the widely used clustering
methods deployed in statistical study and machine learning that
targets to perform n number of partition into a significant number of
clusters where each of the observations basically [63] belongs to
the clusters with only nearest mean. K-means clustering algorithm
is almost equivalent to the expectation-maximization algorithm for
mixtures of Gaussians data.
3. Fuzzy C Means Clustering – This is another form of clustering that
adopts the fuzzy logic or the soft data point allocation technique
and hence the name fuzzy c means clustering mechanism. The
fuzzy c means clustering algorithm intends to perform clustering of
the finite collection of n number of elements in the form of c fuzzy
clusters with an aid of certain conditional criteria. Finally, after
processing, the technique returns a list of c clusters and a partition
matrix where every elements highlights the extent of what elements
belongs to specific clusters.
4. Independent Component Analysis – It is a computational method
for segregating a signal of multivariate type in the form of additive
sub-components assuming the mutual and statistical [44] form of
22
independence of non-Gaussian source signals. It is also considered
as one of the special case of the blind source separation.
5. Self-Organizing Map – It is one of the significant part of the neural
network that adopts training mechanism using unsupervised
learning technique in order to generate two-dimensional and unique
representation of input space called as map [91] that consists of
processed training samples. Another distinctive characteristic of it is
its usage of neighborhood function for preserving the topological
properties of the problem space.
6. Adaptive Resonance Theory – It is one of the associated theory
which is studied alongside with neural network. This technique is
basically used to understand the quantity of the neural network
framework that uses both supervised as well as unsupervised
learning techniques for addressing various significant problems e.g.
predicting the pattern, recognizing the pattern.
1.4 Problem Description
The area of CBIR technique is associated with various challenges that is
discussed in the prior sections. An image retrieval process is
accompanied by various issues that will be required to be addressed
effectively [67]. The problems that have been identified in the proposed
study are as follows:
23
• Visual art images are very different from the natural / colored
images. In such images, the paintings are usually made using
various combinations of the color. These complex colors and their
existence in visual art images renders quite a challenging factor for
performing feature extraction. Hence, existing feature extraction
algorithms as well as training algorithm cannot be directly
implemented over the visual art images to perform CBIR
mechanism.
• It is necessary that a CBIR technique has to be highly enhanced
for making it ready for visual art images. The existing and
conventional CBIR techniques are more inclined towards structural
similarity rather than using semantic similarity. [82] Hence,
adoption of semantic similarity over complex visual art images may
pose various computational instability in the CBIR process owing
to the sophistication in similarity matching of the queried image
and trained image.
• Ineffective adoption of training algorithm may also pose various
problems in performing training. Usually training algorithms are
highly time consuming sometimes and may not lead to superior
outcomes at the end. However, in the presence of massive
database of features, training is one of the unavoidable solutions to
perform similarity match.
24
1.5 Research Aim and Objectives
The prime aim of the study is to introduce a novel and efficient framework
that has the potential capability of [83] performing pruning of feature set
for content-based image retrieval considering the complexities of visual
art images exclusively. In order to accomplish the above mentioned aim,
following objectives are accomplished:
1. To perform an in-depth review of literature for visualizing various
clustering techniques adopted for performing feature extraction in
content-based image retrieval system and for exploring the research
[84] gap/trade-offs in the studied domain.
2. To consider the massive dataset of visual art images and to design a
robust model in Matlab in order to perform content-based image
retrieval.
3. To design a learning based framework using SVM and PCA for
pruning content based image retrieval system on visual art images.
4. To develop a system for retrieval of visual art images using
conventional k-means clustering algorithm for extracting features.
5. To develop an enhanced color visual art image [85] clustering model
where k-means clustering algorithm integrated along with Block
Truncation Coding.
6. To develop an empirical contribution based model exclusively for
partitioned clustering for content based image retrieval for optimizing
25
both homogenous and heterogeneous cluster similarity measures for
visual art images.
7. To perform benchmarking of the [33] COIR framework by comparing
the accomplished result with that of conventional clustering
algorithms e.g. i) k-means, ii) k-means along with Block Truncation
Coding, and iii) Contribution Technique.
1.6 Organization Of Thesis Work
The thesis outline on the factors needs the concept of feature set pruning
for CBIR on Visual art images and objectives in Chapter-1, Chapter-2
discusses on literature survey and related research work carried out to
investigate and visualization of the research gap [36]. Chapter-3
discusses on need and motivation for research work and research gap
chapter-4 discusses on CBIR of visual art images.Chapter-5 discusses
on Content image retrieval Approach(COIR) COIR Scheme
methodologies employed in research work. Chapter-6 discusses the
implementation of COIR.Chapter-7 discusses all the simulation results
accomplished by using COIR systems the results and performs logical
algorithms with respect to performance analysis evaluations. Chapter-8
summarizes the research work conclusion with suggestions for future
work to be carried out.Chapter-9 References.
26
1.7 Summary
Chapter-1: This chapter discusses the introduction about the domain of
research and its significance from application and real-life utility of the
concept of feature set pruning for content based image retrieval on [37]
visual art images, research aim, objectives, motivation, problem
description, and scope of the research work. It also encapsulated the
thesis contribution.
27
Chapter 2 Review of Literature
2.1 Introduction
The concept of context based image retrieval (CBIR) has gained more
attention of researchers for enhancing the application developments in
the field of recognition systems and design. This chapter gives an
overview that how to investigate and analyze the existing prior studies
which talks about significant and efficient techniques associated with
CBIR systems [56]. The proposed study also aims to investigate various
existing traditional techniques and their efficiency with respect to
recognition rate. The proposed study also focuses on various challenges
associated with the existing research trends. The study of introduced
various experimental analysis on automatic retrieval of images from a
database which includes the color and shape features for the automatic
detection [103]. The proposed system uses feature extraction for
retrieving images from the databases. The comparative analysis of the
proposed system also shows that resultant feature of the proposed
system has been compared with feature vectors of the other query
images. The query image is compared with the relevant images of the
database and a closet image is returned. A flow diagram of CBIR
system has been highlighted below where it can be observed that the
system includes various steps such as query formation, feature
extraction engine, feature vectors, feature database, similarity matching
28
and generated query etc. It is also studied that retrieval of images by
manually-assigned keywords does not satisfy the characteristics of
proposed CBIR system. The CBIR system can be dissimilar from the
existing classical information based retrieval system with respect to
arrays of pixel intensities as various image databases [34] are purely
unstructured and digitized images consist of pixel intensities which does
not have any significant meaning.
Fig 2.1-Flow of CBIR System
One of the biggest research challenges arise in order to process various
useful information from the raw data of an image. The useful information
extraction includes recognizing the presence of particular shapes or
texts. In the study of image content retrieval techniques have been
presented [86]. It is found that the retrieval of useful contents from image
databases slightly differ fundamentally from the text databases as the
image databases are unstructured in nature. The proposed study also
focuses on search of information on visual media. There is no
comparable of level 1 retrieval which is applicable a text database.
29
2.2 Techniques Of CBIR
It is also observed that various techniques which can be applicable for
enhancing the context based image retrieval system [70] has been
introduced by many researchers. In this section many context based
image retrieval mechanisms till date have been presented.
2.2.1 Feature Extraction Techniques
Conversation on feature extraction needs information associated with
how the data is developed in a system. When data in the form of images
is developed in a system, the input is nourished into an algorithmic
technique, which receipts and runs with respect to data. To make sure
that the data nourished into a system is not large and not or no longer
needed or useful, feature withdrawal can be performed using data.
Feature extraction mechanism is based on minimizing the dimensions of
an object [94]. By minimizing the dimensions, the data of higher
dimensions can be converted into the data of lower dimensions. The
feature extractions convert the input data into some set of features which
are called as feature vector. The functionality of the feature vector
includes the extraction of significant data from the datasets to satisfy the
need. The following sub-sections highlights various feature extraction
mechanisms that have been utilized for different applications.
30
2.2.2 Boundary Detection Feature Extraction
A survey has been conducted for analyzing various existing techniques
of image processing based feature extractions [87]. The concept of
boundary detection mechanism says that a scan is performed in a binary
image until a boundary is found in that scanned image. Scanning can be
performed with respect to K- Nearest Neighbor Technique. A foreground
pixel P is taken into consideration and a set of foreground pixels which
are linked with P are termed as component containing P. The tracing is
performed in such a procession that the initial position has been set for
indicating the beginning of the boundary. The feature vector that is
measured using this technique is termed as Fourier Descriptors. Using
Fourier Descriptors, the Fourier Coefficients can be evaluated. For
confirmation of the boundary positions as they are fully covered up or
not, the first and the last position coordinates have to be investigated as
they are equal or not.
2.2.3 Color Averaging Techniques for Feature Extraction
The color averaging technique is a method of feature extraction which
can be applicable in the field of spatial domain. Feature extraction has
been introduced for the applicability in spatial domain for optimizing the
size of the feature vector. Color averaging based image retrieval
techniques [95] are increased with the use of even part of semblance
31
which can be attained by the addition of original image with the flip
image. The even image can be attained by
Even Image = [(Original Image + Flip)/2]
Row Column, Forward Diagonal Mean, Row column mean, Forward
Diagonal Mean of image has been utilized as feature vectors.
2.2.4 Gradient and 12 Directional Feature Extraction
In this method, first the incline of the handwritten character which has
been induced by Sobel’s Mask has been calculated. After calculating the
incline of each pixel of the character, these gradient values are [96]
plotted onto 12 path values with angle span of 30 degree between any
two adjacent direction values. Once the direction features are got; they
are fed into a neural network system.
2.2.5 Feature Extraction using Slope Magnitude Technique
For extracting shape features, the removed edges which are required for
connecting in order to characterize the boundaries. This is the main
reason why the slope magnitude method is used. The slope magnitude
technique is utilized with respect to gradient operators for [3] extracting
shape features in the form of linked boundaries. In this technique, the
shape feature vector is constructed by applying the slope magnitude
method which can be mapped to the gradient of images in the vertical
and horizontal directions. However, the problem with this technique is
32
that, all images must have the same measurement in terms of query
image.
2.2.6 Feature Extraction using Transforms
In this method, the DCT transform is utilized for designing the sectors
required for the search and retrieval of images from a database. The
comparative analysis shows that this method has been COIR in
presentation comparison of four, eight, and twelve Walsh Transform
Sectors Feature vectors which are used for image recovery from an
image database. The [1] rows in the discrete cosine transform matrix
have some characteristics of enhancing order. Thus, zeroeth and all
other even rows have even orders, whereas, all odd rows have odd
order. To form the feature vector plane the combination of coefficient of
consecutive odd and even co-efficient of every column has been
considered and put even co-efficient on x axis and odd co-efficient on y
axis.
In this method, the DST transforms which can be utilized for designing
the sectors to generate the feature vectors for the purpose of search
and recovery of database images. Like DCT the property of cumulative
order of the transform matrix is utilized.
33
2.2.7 Feature Extraction using Fast Fourier Transform
The Fourier transmute is used to procreate the shape vectors supported
on the signify excellence of royal and visionary ability of composite
numbers of polar coordinates in the crowd dominion. [97] The 8 signify
excellence of 4 superior partial sectors, royal and visionary ability of each
R, G and B components of a copy are weigh for form vector formation.
Every composite numeral can be delineate as a step in the composite
flat, and can therefore be uttered by mention either the moments
Cartesian coordinates. This serve to procreate eight components of a
shape vector supported on the intricate even as enumerate above.
2.2.8 Feature Extraction using Walsh Transform
This process companion use of the Walsh Transform to project the
sectors to procreate the form vectors in CBIR. The Walsh transmute of
the paint picture is fitted in all three R, G and B even. The intricate rough
delineate visionary components of the picture and the real rows [98]
delineate genuine components are curbed for the presage innovate
harmonious to the quadrants equivocatory in. The visionary and real
constituent of Walsh values are appoint to each quadrant. Once the form
vector is procreate for all likeness in the databank a form databank is
composed.
34
2.2.9 Similarity Measurement
The CBIR procedure is used for common applications resembling
concede archetype and duplicate finger mark or other biometric data.
Basically two copy extremity to be compare with and curbed to see
whether they are homogenous or whether they equal or not. To reckon
this [17] , it is need to have stated techniques by which one can
statistically rate if two images are homogenous or not. It is for this
argument that resemblance menstruation is done. Once we citation a
serviceable obstruct of form are citation, we compare with the citation
form for likeness for are compare with; it is expect that if serviceable
obstruct of form are citation, the resemblance between 2 images is
assumed to see how consolidate the citation shape are of the two copy.
There are different kinds of resemblance menstruation techniques; the
Euclidean Distance, Mean Square Error and Sum of Absolute
Differences for ordinary CBIR applications.
In CBIR systems, picture form are Specific discrepancy have been
explain for particular shape [42]: e.g. for histograms usually application
moderation are the histogram distinction, histogram intersection or the
square discrepancy. The latter attempt to rehearsal for the perceptual
distinction between any flights of box in the histogram. The Hausdorff
discrepancy has been application to compare with histograms as well as
regulate in. Many resemblance moderation are supported on the Lp
35
discrepancy between two appoint in the n-dimensional form room. For
two appoint x; y the Lp discrepancy is explain as.
(2.1)
Murkowski discrepancy. For p = 2 the Euclidean discrepancy is going to
and for p = 1 the Manhattan is going to village blockhead, or taxicab
discrepancy.
The L1 and L2 model are analyzed in, and their performances are
compare with. The recovery work of a system trust on the compact
between the likeness moderation application and nominal judgments of
resemblance [18], since the termination destroyer of CBIR arise is a
nominal. Therefore, several moderation in coincidence with the hominal
apprehension have been improved. COIR a form-supported “compare
standard” of likeness, in which usual form watch to grow the understand
resemblance of two concepts, and where form dissimilarity watch to
contract understand likeness.
For example, Tomato and Cherry are homogenous by bravery of their
ordinary form Round, Fruit, Red and Succulent. Likewise, they are unlike
by bravery of their distinction, namely Size [116] (Large versus Small)
36
and Seed (Stone versus No Stone). This is an unsurprising and
perceptive proclaim; however, Tversky’s standard proclaim that form
commonalities guard to grow discern resemblance more than shape
distinction can degrade it. That is, when assessing resemblance more
confidence is disposed to those form that concepts have in usual than to
those that discriminate them. Tversky’s compare standard is a non-metric
set-theoretic rehearsal of understand resemblance that endeavor to
betake some of the fault of the discrepancy plan. Tversky’s standard is
supported on appraise obstruct of duplicate and mismatching
characteristic: features:
(2.2)
where interpret the resemblance of x to y, X is the obstruct of
characteristic that delineate x, Y is the set of form that delineate y, is the
set of characteristic ordinary to x and y, is the set of characteristic
uniquely accomplish by x and is the set of characteristic uniquely
accomplish by y, α and β are unreserved parameters and f is a
performance over sets of characteristic narrated to the saliency of the
characteristic [4].
2.2.10 Euclidean Distance
In picture procedure, a discrepancy transfigure is the deduce portraiture
of a digital picture. A discrepancy transmute is also called a discrepancy
37
delineation. A discrepancy sketch is such that it tassels each pixel of the
likeness with the discrepancy of the proximal obstruction pixel. The
discrepancy moderation trust on the stamp of measure conscript. [108] In
Euclidean disagreement, the metric that is predestined is clearly the
Euclidean measure. For the variance moderation either a low worth or a
high value discover likeness. In the case of Euclidean discrepancy, a low
of value of discrepancy moderation show likeness. Indicates similarity.
2.2.11 Mean Square Error
Mean Square Error resembling Euclidean discrepancy is a contrariety
mensurative technique. A leas value of MSE discovers resemblance.
MSE in stats is a way to quantitative the distinction between an inferential
excellences of a value to the true value of the greatness to be estimated.
Mapping this description to the relevancy of picture acknowledgment
[99], it could be said that estimators would be the picture in the databank
and the excellence to be estimated would be the question picture to be
compare to. MSE measures the amount by which excellence of the
picture from the databank denote disagree by the real excellence of the
slice picture. This distinction appears inasmuch as of randomness or
because the picture from the databank could not account for intelligence
that would have generate a more correct calculate.
38
2.2.12 Sum of Absolute Differences
Sum of arbitrary distinction is an unmingled algorithmic rule for finding
the co-rehearsal between two copies. [8]It captures the absolute
distinction between each pixel in the primitive picture to the conformable
pixel in the picture used for illustration. Like Euclidean Distance and
Mean Square Error, a low worth of the compendium of arbitrary
distinction discover resemblance. The precision of this process is
beloved by agent such as lighting, paint, survey clew, and regulate or
size.
2.2.13 Neural Network Classifiers
Classifiers by themselves are a precedent of an Artificial Intelligence
relevancy. Classifiers are basically used to limit a closest equal in a set of
statistical data. A neural network is actually a network of neurons in the
brain which accomplish different activities. In a feigned neural network,
the genuine neural network is pretended by interconnections called
nodes [6]. An artificial neural network is used to equal patterns
depending on the succession of traversal through the nodes.
2.2.14 K-Nearest Neighbor Algorithm
This algorithmic rule is used in archetype acknowledgment as a process
of arrange oppose supported on the closest education precedent in a
form room. In this algorithmic rule, an oppose is categorized by a
39
superiority vote of its adjoining [22]. If k=1, the oppose under examination
is appoint the high-class of its proximal adjoining. The algorithmic rule
has an education faze and an assortment faze. The education faze of the
algorithmic rule consist in of plenty form vectors and high-class tassel of
the education specimen. The drawback is that during the assortment
disconcert, the high-class with the more habitual specimen watch to rule
the prophecy of the fresh vector, inasmuch as they guard to arrive up in
the k-proximal adjoining when the neighbors are reckon due to their large
specimen. Euclidean or Mahalanobis discrepancy are used as
discrepancy metrics [23]. The best choice of k confides upon the data;
commonly, larger worth of k subjugates the result.
In a diminutive databank, a unmingled consequential analyze is on the
whole employment for k Nearest–Neighbor (KNN) inquire. But for
copious data prepare, effective teacher algorithms are compulsory. High
dimensional data is in crescent in many common fields. As the count of
importance grow, many clump techniques exorcise to tolerate from the
anathematize of measurement, degrading the profession of the spring. In
proud proportion, data come very scattered and alienation measurement
come increasingly unmeaning. There is a syn categorization for full
dimensional data adjust mob [109]: 1-Dimension conquest, 2-
Parismonious design, 3-Subspace bunch. Feature choice and form
descent are most popular techniques in importance subjection. It is pure
40
that in both methods we some enlightenment will be lost which
spontaneously act on propriety. Ref, reviewed the literature on mean
pattern and Gaussian dummy from the most complex to simplest which
furnish a progress co similar to the K Means approaching. When there
are allay there are two vast approaches for subspaces methods: in first
philathea centers are considered on the same unknown subspace and in
another each class is placed on specifying subspace. The discrimination
of subtopics or [88] subgroups is set for handwriting crowd and topic
mining. Tensor factorization as an efficacious technique has been used
in. Inconsistency has been shown in those public data stop forasmuch as
of outlier. TRef COIR a framework which complete subspace selection
and bunch. Equivalency between kernel K-Means group and repeating
subspace selection has been shown.
Ref COIR an uncertain crowd improver shoot which is fenced in in two
might-hamper proceeding. In the first step [61], it uses real properties of
the data set for menstruation curtailment after that several repetition of a
compress algorithms are turn, each with clear distinguishing. Based on
BIC bound, the most copy will be chosen. There are some weaknesses
for this course e.g. since BIC depart a workmanship to specific data
preparation it cannot be opposition to simile design of separate data
regulate. Ref immediate a semi-superintend family process dishonorable
on globular K-Means via characteristic protuberance which is tailors for
41
manipulation sparse supercilious dimensional data. They first formulated
coerce-conductor representative protuberance then layer the necessity
orbed K-Means [19] algorithmic program to bunch data with subdue
dimension.
(a) Ref COIR two methods of allied objective office clustering and
diagram conjecture protuberance [74].– Method 1: intangible polygraph
criteria into an outward performance consonant to A traditional objective
cosecant Clustering for reproduce the verbal result An auto associative
additive system based on graph theory clustering for modifying the initial
result.
Ref COIR sequential coalition methods for data clustering – In improving
clustering production they COIR the use of more than one crowd method.
– They search the interest of important league throng as antagonist to
coeval association and found that successive [113] union is less difficult
and there are import without the above cost of coeval clustering. In
clustering characteristic lying in violent dimensional while, formulating a
pleasing moderation [24] of “co similarity” is more problematic. Recent
exploration shows that for proud dimensional path, computing the
alienation by shamelessness at all the scope is often useless, as the final
almost of an item is anticipate to be almost as complete as its nearest
neighboring. To recount mob in dissimilar decrease dimensional
42
subspaces, recent performance has focused on projective bunch, explain
as succeed.
2.3 Summary
This chapter has discusses about the existing technologies that are in
frequent practice for CBIR techniques.
43
Chapter 3 Need For The Study
3.1 Introduction
Digital Image processing has already played a significant role in various
forms of application right from engineering, medical science, geology,
psychology, etc. With the advancement of various sophisticated image
capturing devices and rise in superior versions of compression formats,
image processing allows to restore the most significant information of the
targeted object. Out of this, there is another area, where the contribution
of image processing is less significant and that area is retrieval of
museum images or images from art gallery. At present, majority of the
existing system are focused only on developing the storage system with
retention of the highest quality of signal with good resolution and less
noise while attempting to storage the image in the database.
It is also known that with the evolution of pervasive computing like cloud
services, storage is never a scarce resources. Hence, storing massive
number of such visual art images is never a challenging factor. However,
such storage is quite expensive, if the stored data is of no use in future.
Hence, the proposed system highlights a problem of content based
image retrieval process for visual art images and introduces a technique
that assists in building the database very different from existing system
and also developing an image retrieval system that can process and yield
the result of similar images from the novel dataset. The prime motive of
44
this chapter is to put forward a true picture in front of the readers about
the need to undertake the proposed study.
3.2 Need Of Smart Database Building
In the present era, there is no scarcity of the processer’s capability as
well as good configurations of memory (RAM) in personal computers as
well as in enterprise computer. Usually a database is a storage location
in the storage server that can be either a local server or in global server.
Usually, the size of local servers are quite limited enough for the
organizational usage and they depending upon the utilities as well as
availability of capital, an organization go for global servers (in distributed
domain, servers, cloud etc). Whatever the manner of availability of the
storage system, it cost money one or other way. Hence, a user always
needs to think how to optimize the storage. The biggest problem is visual
art images are usually very big in dimension where it can range from
mere 20 centimeters to 6 foot in terms of height. A sophisticated image
capturing devices as well as scanning machines are already available
now-a-days that can precisely capture the image and truly store the
image information in database. It is also known that 98% of the users
adopt RDBMS system, where the data is maintained in rows and
columns. Hence, it can be said that one digitized visual art image is now
stored as a whole in storage system.
45
Such mechanism of storage of visual art images is very good if it pertains
to only storage system. But when it comes to apply some more analysis
(e.g. performing CBIR), there are critical issues and problems in this
mechanism,
• Irrespective of size of storage, it is soon going to be over saturated
with many incoming digitized visual art images.
• Not necessary all the information of the stored digitized image is
essential for carrying out analysis and in such case the storage is
highly expensive.
• Even if conventional image retrieval system is build, it will be highly
time consuming task to make the query image with massive
number of images in the existing storage.
Hence, such problems discussed above can be eliminated if a smart and
cost effective database system is designed. The uniqueness of the
proposed system is it doesn’t storage the complete image in its
database. The system digitized the set of visual art images and then
extracts significant features. Usually, if the size of a high-resolution visual
art image is in megabytes than size of extracted feature will be in bytes
or lower than it. Hence, storing the features will consume less storage
and can easily scale up the storage system. Moreover, during similarity
match, the feature of queried image to get matched with feature of
images in the dataset consumes very less time in comparison to the
46
existing system. Hence, there is a need to develop a cost effective
database design.
3.3 Need To Overcome Challenges In Visual Art Images
The very first thing to understand here is reason to perform digitization of
visual art images. Usually, it is done to perform i) restoration of some
fragile or valuable paintings, ii) to apply image forensics to discover some
hidden truth behind the painting (e.g. image forgery), iii) to perform
advance analysis to extract and retain the most valuable and unique
information of expensive visual art images. There are various forms of
complications while applying image processing over visual art images.
The significant complexities are as follows:
• The style and manner of color used in visual art images are
something quite different from natural image. A natural image has
a good balance of brightness or contrast, which cannot be seen in
visual art images. They may be too bright or too contrastive. It may
be also too dark. Performing any image processing applications
(e.g. compression, denoising, image enhancement, or scaling)
may significantly result in loss of potential information.
• Although as a researcher, size and dimensions of the visual art
images are not to be bothered, but a researcher should maintain
the uniformity of shape and size of the visual art images while
47
subjecting the digitized images for any further image or statistical
analysis. The problem is the existing dataset have varieties of
visual art images of multiple dimensions.
Therefore, the critical challenge in designing an image retrieval system is
to develop or evolve up with a novel technology and methodology that
can effectively address the problems mentioned above. One solution is to
perform a categorization of the massive dataset of visual art images
depending on the specific characteristics of image. For an example,
consider grouping all the paintings pertaining to battle or combat under
the group (or category) name war, or consider grouping all the paintings
corresponding to music and dance as recreation. There can be different
forms of techniques for working on the low level features that could be
distinguished with an assistance of an essential characteristics of domain
images. The impediment for optimal performance in such processing
would be various factors e.g. variable illumination condition, multi-
resolution of an image, variable texture, occlusion etc. Apart from pixel-
based approach, it is also found that applicability and consideration of
semantics in image processing is quite a gap for understanding various
significant features and context from one image. It can also be said that
there is a big gap between understanding of features to be extracted and
the nature of the semantics that are understood by human. The extent of
the problem becomes much worst owing to the subjectivity in the visually
sensed natured of semantics. In order to perform feature set pruning, it is
48
highly essential that the processing time of feature as well as response
time of query should be highly optimized. A better scope of performance
enhancement could be carried out if the memory complications as well as
feature-dimensionality problems could also be enhanced for visual art
images. Hence, there is a need to develop a framework that can perform
feature set pruning for visual art images effectively.
3.4 Need To Enhance CBIR Process
In presence of large number of visual art images with variable shape and
size, it is a very difficult task to design an image retrieval system. The
existing techniques of retrieving the images will be based on visual
contents normally. Taking an example of existing image search
techniques in Google, it is using a text annotation and the search is
completely based on the text annotations and never on the visual
contents. Design and developing such precise image search technique
results in accurate feature pruning process in visual art images.
However, in existing system, there is almost no research being carried
out to incorporate CBIR technique over visual art images, which is a very
big question mark. With an availability of sophisticated CBIR techniques,
there is a positive hope for experimenting with challenging visual art
images. Hence, there is a need to enhance the existing CBIR technique
to make it suitable for retrieval process in visual art images.
49
3.5 Summary
This chapter discusses about some of the essential factors that has
encouraged the researcher to understand the existing problem and
thereby evolved up with a rationality to work on visual art images for
feature set pruning.
50
Chapter 4 CBIR for Visual Art Images
4.1 Introduction
This chapter presents an overview of the image retrieval and especially
Content-Based Image Retrieval (CBIR) as an alternative and supplement
of Text-Based Retrieval. [21]A little tour of some existing digital
repositories of art images is made. This chapter discusses about the
CBIR technique for enhancing the performance of image retrieval from
the visual art images. A short overview of existing techniques, which are
employed in the main processes in CBIR, is given.
4.2 The Process of Image Retrieval
The method of the information retrieval deals with the system that is
based on extracting the essentials of the metadata as well as significant
level of the image visual contents. The process of the image retrieval
also depends on various intrinsic as well as extrinsic factors[64]. It is
interdisciplinary and invite the interest of extended frequent of
researchers and code-writer from a multitude of domains - intelligence
knowledge, data processor knowledge, intelligence workmanship, moths,
library knowledge, cognitive psychology, linguist, physics, stats etc.
Image recovery is part of it; it shorten on the operation of “searching,
browsing and retrieving images” from gigantic data of digital images.
51
There are two standard methods in image recovery: “text-based retrieval”
(TBR) and “content-based image retrieval” (CBIR), which are used
unconnectedly or mutually
4.2.1 Text-Based Retrieval
Search systems begin on textual dispatch possess metadata regarding
the likeness such as sophism, keywords and descriptions of the copy; the
restoration is accomplish above [20]the commentate express. These
methods are willingly accomplish worn previously being technologies, but
extremity handbook data input for each likeness in the system. Manual
effigy annotation is time-erodent, diligent and costly and is a influential
impasse since the acceleration of handbook delineation and data
entrance is diminish than the quickness of digitization. This is impractical
for the colossal collections or automatically-produce copy. Usually theme-
supported descriptions are not purpose accurate and ceremonial and are
often deficient. Another statement with composition annotation is that it
often does not confirm any determine phraseology in a especial dominion
and may not describe the relations of the goal in the show – besides the
subjectivity of judgments of clear community who impress the data.
4.2.2 Content-Based Image Retrieval (CBIR)
Content-supported idol recovery, as is established today, is any
technology that in tenet back to methodize digital images supported on
52
their content. By this detail, anything rank from an image likeness
reverence to a unharmed conception annotation aptitude dismiss into the
scrutinize of CBIR [65].This characterization of CBIR as a field of study
places it at a unique juncture within the scientific community. While one
witnesses continued effort in solving the fundamental open problem of
robust image understanding, one also sees specialists from different
fields.
Typically, a content-supported image recovery system comprise of three
components:
1. Feature design.
2. Indexing.
3. Retrieval.
The characteristic sketch composing descent the ocular form(s)
intelligence from the images in the image database management system.
The form insignitor constituting systematizes the optical form intelligence
to haste up the question or procedure. The recovery ability progress the
user question [110] and stipulate a use interface. During this process the
central issue is to define a proper feature representation and similarity
metrics. CBIR systems extract visual features from the images
automatically. Similarities between two images are measured in terms of
the differences between the corresponding features. To take into account
53
the subjectivity of human perception and bridge the gap between the
high-level concepts and the low-level features, relevance feedback is
used as a means to enhance the retrieval performance.
All these steps are highly dependent of the domain where CBIR
technology is applied. For instance, in the fields such as aerial image
retrieval and medicine the goal is exactly defined, the [112] searched
objects in the images has homogeneous specifics, the received results
usually do not need communication with the user to refine the queries.
Absolutely different is the situation in the areas that are connected with
the creative side of the human beings, such as art, architecture and
design. The different kinds of users also stamp different requirements
into specifics of CBIR systems. This work focused in the area of feature
design for art images, presented in the web. Other components are
touched only as an integral part of the experimental CBIR system, which
is created in order to supply the appropriate environment for conducting
the experiments with examined features.
4.3 The Image Domain
The use of digital images dates from 1920, when the Bartlane Cable
picture transmission service was used to transfer images between
London and New York. These were codified in 5 gray levels (later 15)
and reconstructed using a telegraph printer. The use of digital images as
54
is known today appears in the 1960’s when improvements on computing
technology and the onset of the[115] space race led to a surge in digital
image processing, especially in the enhancement of pictures of the moon
taken by Ranger and Apollo missions. In the medical field, the digital
image appears in the 1970’s and its importance is recognized in 1979,
when Sir Godfrey N. Hounsfield and Prof. Allan M. Cormack shared the
Nobel Prize in medicine for the invention of tomography, the invention
behind the Computerized Axial Tomography. But what is an image? The
image, in a literal definition, is a two-dimensional pictorial representation.
The digital image is an approximation of a two-dimensional image by set
of values [101] called pixels. Each pixel is described in terms of its color,
intensity/luminance or value. Each digitalimage1 has a limited extent,
window or size, an outer scale, and a limited resolution, the inner scale.
Mathematically, the image is a real function : → mapping two real
variables into a third real variable. Thus ( ) ( ) α== YXIrI , where (X, Y) is
the full spatial domain of the image, i.e., a set of points in the Cartesian
plane, and is the luminance/color/value of the image point, with
RandRr αε2∈ . The value can be interpreted in many ways and it is not
necessarily a positive value. Also, depending on the [30] color system
used may have a higher dimensionality. In the digital image ( )YXr ,= are
pixel coordinates, whose values are bounded by the image size
55
{ } { }yX MmandNnwheremn ,...2,1,.....2,1 ==× mn× where. Therefore, the digital
image can be seen as × matrix of elements.
In order to retrieve images according to a given query we need to
enhance its relevant elements while reducing the remaining aspects. This
is the goal of image processing. Generically, we act on the image using
an operator, g, over the full spatial domain of the image, ( )yxI , , an image
patch, ( )yxI , , or an interest point, ( )mn yxI , to generate a feature space [57]
containing the information needed to identify the objects in the following
way:
( ) ( )YXIgrf ,o=
( )( ) ( )yxIgf yxr ,, o=
Where ( )YXI , is the full image: ( )YXI , is a image patch i.e., a connected
subset of Cartesian points with ( ) ( ) ;,,,, RyxYXyx ∈∀∈
( ) ( ) ;,,,, RyxYXyx ∈∀∈ is the α value at a interest point
( ) { } { }yxmn MmandNwithnyx ,...2,1,.......,2,1,, == ( ),, mn yx
4.4 Image Properties
The relationships between image properties like color, shape, texture and
interest points are certain to be fundamental for its characterization. But
( ) ( )mnymxn yxIgrf ,, o=
56
what are exactly these properties and how are they used for CBIR
purposes?
4.4.1 Color
Natural Conceptualists trial that color is the outcome of an epistemology
formulation: an oppose is melancholy inasmuch as that it is teach that it
is melancholy and it is approve fact. This is allowable if one can elucidate
why it is exactness. Therefore, [4]one is left with a difficult philosophical
problem where there is no consensus on the origin of color. Two very
distinctive points of view exist: some theorists agree that color is a
perceiver-relative property of the objects, e.g., dispositions or powers to
overspread exercise of a stated good, or to seem in stated ways to
observers of a stated kind while others condition that they are goal
external properties of the motive, e.g., color rely on the physical
microscopic properties of the bodies and are, therefore, irreducible [61].
There are many theories [66] about color: Color Factionalists state that
there are no colors at all (!) when exploiting the gaps of other theories,
thus supporting a perceiver-relative point of view; Simple Objectivists
stand for the concept of color is either related with physical properties of
the objects or in the nature of light, hoping that science will provide an
answer; Ecologists (!) diverge slightly from Color Factionalists arguing
that it is an relational property between the environment and the
individual. There are others theories attempting an integrated definition of
57
color but so far an agreement seems impossible. For the interested
reader details of color theory can be found in.
In CBIR, color is a widely visual feature to categorize objects. For this the
variable α is expressed in terms of a color space to represent image
colors. The RGB (Red, Green, Blue) system is commonly used to
represent color images where the gray level intensities are expressed as
a sum of red, green and blue grey level intensities [111]. There are many
other color systems, like the HSV (Hue, Saturation, Value) or the CMYK
(Cyan, Magenta, Yellow, Key).
The image can also be represented as an 8-bit grayscale image, where
pixel intensity is registered in terms of 256 shades of gray, or as a 2-bit
binary image, in black and white. A quick reference to color systems and
conversions between systems can be found.
Depending on the color system, one or more histograms are employed to
quantify the color distribution, defined by the number of bins used.
Differences in color distribution are sometimes, essential to determine
differences [100] between images. However such distribution can lead to
errors when different images present similar histograms. Aiming to
capture spatial relationships between colors, the image is partitioned in
smaller sub images and a color histogram is extracted from each of
58
these. This results in the color layout of the image. Exploring correlations
between pairs of similar colors based on their mutual distance within the
image can also be explored in what is called a color auto-correlogram.
4.4.2 Shape
The shape of an object is its apparent form. In the image domain
extracting, shape consists [68] in the identification of lines and curves.
Shape extraction is already a well-developed field in image processing
where two main streams exist: gradient-like methods using directional
maxima lookup to quantify the edge strength, like the Canny edge
detector, Sobel, Prewitt and Robert’s operators, and second derivative
zero crossing search methods, like the Laplacian-like approach. Other
methods based in the Hough transform, curve propagation and wavelets
also exist. Quantification of edges is made using histograms considering
the full image spatial domain or, like in color, sub images.
4.4.3 Texture
The concept of texture is somehow intuitive, being closely related with
visual patterns perceived in the surface of objects that present
homogeneity. However its definition is not exact. Perhaps a serviceable
demarcation for interweave for image protuberance is [13] that it is a
function of the spatial deviation in pixel intensities. In image processing
59
texture representation relies mainly in two methods: structural and
statistical. Structural methods aim to identify texture structural primitives
and placement of its elements, looking for regularity. Examples of this
method are adjacency graphs or morphological operations. Statistical
methods use first or higher order statistics to analyze the distribution of
luminance on the image. These include the popular co-occurrence
matrix, Fourier power spectra or shift-invariant principal component
analysis (SPCA). In CBIR co similarity, metrics are the most utility mode
to liken the structure of appearance during the recovery process. A
reexamination of web descent methods can be found.
4.4.4 Interest Points
Interest points are themselves the result of an operation comprehending
the full spatial domain ( )YX , . An image descriptor comprehending an
image patch around these points, instead of its putative use, plays a role
in image retrieval. Therefore, equations (3.2) and (3.3) are related if the
patch is centered on an interest point. Examples of [38] interest point
detector are the Difference of Gaussians (Dog), the Harris corner
detector of the Hessian matrix. In CBIR, the use of image descriptors
around interest points is grounded in two methods: direct image
matching, to find the same image in a collection, or together with a bag-
60
of-words approach to capture image concepts. A review of image
descriptors using information around interest points can be found.
4.4.5 Image Retrieval
Image retrieval is one of the main focus for the distributed multimedia
systems. In general, the image data base system has intelligently the
combination of the following major components:
1. Image Processing
2. Storage retrieval and management
3. User interaction
The first component focuses on processing the extraction of information
from original images and the second one [69] provides efficient tools for
storing retrieval and managing image data. For querying of database
needs a user friendly interface which is done by third component.
The image retrieval has been influenced by the language support from
conventional database management systems. To make it more flexible
and more intelligent, visual information embedded in images should be
preserved by exploiting[ 58] an efficient data structure to store them. The
symbolic descriptions of graphical information such as the shape sketch
or spatial relations using the traditional approaches are of difficult task.
61
Figure 4.1: image retrieval methods
The indexing technique for facilitating and accessing data base has been
established from conventional algorithms for image retrieving. Similarly
image indexing is also one of the techniques to elucidate image
information retrieval from the database. The use of the pictorial index is
the basis of iconic indexing methodologies. The Query by Visual
Example [43] developed by Hirata and Kato is one of the earliest systems
developed for image retrieval using image content. It defines the pictorial
indexes for the query images and then compares for the retrieval task.
This approach is very useful for visual art or the sketch based image
retrieval. The different approaches for the image retrieval is shown in
figure 4.1 and it has been categorized into attribute based, annotation
based, object recognition based and the last being low level image
feature based.
62
4.5 Content Based Image Retrieval
The act of selecting a subset of an likeness databank answering to a
representation as fixed by the user
Figure.4.2: the different steps in CBIR
A CBIR session can typically be summarized as follows (Figure 4.2).
1. a user generate a question delineate the image(s) he/she wants to
retrieve from the database management system and present it to the
CBIR system; the CBIR system computes images stored in the
63
database; this is done according to the internal description of query
and database image;
2. the CBIR system returns a list of images sorted according to their
similarity to the query;
3. The user modifies the query and/or uses part of the result to form a
new query.
Fields that can profit from CBIR applications are almost innumerous.
Among them one can list art galleries, architectural and engineering
project, internal sketch, geographic intelligence systems, [89]expert
databank guidance, weather prediction, sell, framework and
workmanship project, trademark and copyright databank guidance,
jurisprudence constraint and incendiary research, painting chronicles and
association systems, impress agencies, medicinal analysis, education
and training. CBIR systems can be classified according to their
application.
4.5.1 Database Content
The images contained in a database can be more or less heterogeneous:
databases range from very specific databases, where all images are of
the same kind and are taken under the same conditions (think of a
database of MRI scans of the brain taken with the same machine), [90]to
be specific, if not all the parameters are constant throughout the
database (like a database containing pictures of stamps against a dark
background, but where the pictures were taken using different cameras
64
and different lighting conditions), to monothematic (like the database
used by Das et al. , which contains only images of flowers), to
heterogeneous (images collected randomly from the Internet, for
example).
4.5.2 Query Form
Systems accept queries of very different kinds, depending on the kind of
data extracted from the database images and the targeted application.
Among the various possibilities there are keywords, [114] natural
language queries, feature values (color percentages, for example),
example images and user-produced pictorial examples
4.5.3 Image Description
When an image is added to the databank, some kind of image index term
is formed. During the recovery state, this index term is used to distinguish
the likeness between the query and the databank image. Hence, the
kinds of queries that can be content by the system are nearly [32]
narrated to the good of data hold in the index term. The data that can be
associated to an image is of three kinds (figure.4.3): semantic information
(a description of the image content: subjects, depicted action, place,
etc.), primitive information (colors, textures, shapes, edges, visually
homogeneous regions, etc.) and factual information (information that
cannot be extracted from the image, like name of the photographer, date
65
of the shoot, belonging of the image to a particular series, conditions
under which the image was taken, etc.).
Figure 4.3: the three kinds of data that can be associated to an image: semantic (in this
case a natural language description and keywords), primitive (for example, contours and
a segmentation) and factual (name of the photographer, date and place of the shoot, film
and camera used).
4.5.4 Interaction between User and CBIR System
Adding interaction between the user and the CBIR system can serve
realize ameliorate recovery effect. Interaction wander from merely
permit the user to surrender a fresh query supported on a prior one,[91]
to giving the user the choice to choose part of the effect image as
pertinent and/or non pertinent to tolerate the user to visually classify a
small set of the databank images into gather of homogenous images.
66
4.6 Importance of Images Retrieval
Why is finding images in a database that satisfies a particular
specification so important? Let’s pretend for a moment that a person is a
stamp collector, that his collection is composed of several thousand
stamps, and that he wants to be able to make lists of his stamps
according to the issuing nation, the date of issue, the nominal value, the
current market value [25], the shape, the pictorial content, the condition
(new, used, first day of issue, damaged, ...), the series they belong to, or
similarity to a particular stamp. In this case, storing all the stamps in a
large box would surely prove to be the wrong solution, since for every
search he has performed he would probably end up going through the
whole collection. It is clear that he will emergency to organize the crush
in such a moving that the making of these hearken will be a governable
work.
Sooner or later, all people will be in this situation: as more and more
information is produced as digital images, all of them will have to find a
way to perform efficient searches through the millions of images that will
be available. Efficient CBIR systems will be needed. After all, what good
is information in the Information [2] Age if one cannot find the images?
In any case, in order to be viable a CBIR system must deliver good
performance in the following three fields:
1. indexing of the database, which is essential in order to compare the
query only to a small subset of the database, the computation of
67
similarity between the query and a database image being in general a
time-consuming operation; comparison of query and database
images, which must correspond as well as possible to the human
perception of similarity, so that the retrieved sets will be considered
satisfactory by the user;
2. Retrieval efficiency, since the goal of a retrieval session is to return all
the relevant images with a better ranking than the [31] not relevant
ones. CBIR applications are almost countless. Extending the list given
by[85].
The list can be like this:
1. art galleries and museum management;
2. architectural and engineering design;
3. interior design;
4. geographic information systems;
5. scientific database management;
6. weather forecasting;
7. retail;
8. fabric and fashion design;
9. trademark and copyright database management;
10. law enforcement and criminal investigation;
11. picture archiving and communication systems;
12. press agencies;
13. medical analysis;
68
14. training and education;
15. Industrial quality control.
When trying to classify CBIR systems, big differences can be noticed in:
1. the content of the database;
2. the form of the query;
3. the type of data stored with the images;
4. The amount of interaction between user and CBIR system.
In the following sections these four subjects will be discussed in depth.
4.7 Database Contents
Images restrain in databases can be of dissimilar kinds, order from a
16x16 2-bit specimen to a 1200dpi, 32-bit color [102] scrutinize of an A4-
size page. Databases confine a more unchanging kind of images will in
common be easier to manage and will tolerate more scrupulous explore
than inhomogeneous databases, since particularize algorithms or
authority experts will be effectual to origin the wanted data. In this
section, a classification of image databases according to the variability of
the images they contain will be given.
Among the characteristics of an image one will have:
1. the size of the image (and the aspect ratio);
2. the color depth (black and white, grayscale, 8-bit color, 16-bit color,
24-bit color, 32-bit color);
69
3. the conditions under which it was taken (illumination, distance
between object and camera, kind of camera, ...), which can be
known or not;
4. the number of objects/subjects portrayed;
5. its origin (natural or synthetic);
6. the file format;
7. whether the objects/subjects are in front of a known background or
not;
Figure 4.4: -Visual art images
• Specific databases: if some of the image characteristics are not
constant throughout the collection, for example in a database
containing pictures of stamps, where illumination conditions and
distance between the camera and the stamp are unknown, but
where other parameters are known (dark background, only one
stamp per image, subject placed in the centre of the image, same
color depth, ...); another example of a database of this kind is a
collection of face shots.
• Monothematic databases: when all is known about the images is
that every one of them will portray a particular kind of object or
70
subject; a collection of picture of cats is a good example in this
case.
• General (heterogeneous) databases: when nothing is known
about the conditions under which the pictures were taken, or about
the portrayed subject; the Internet is a general database.[75]
Obviously, many other kinds of image databases exist, ranging
between the very specific and the general ones as presented
above. The more specific a database is the more in depth will be
the analysis of the images. Precise data about the images can be
extracted, either with highly specialized algorithms or by human
experts, since the fact that the images are contained in the
database already says many things about their content. With
images extracted from general databases, this work must be done
before the extraction of any precise data.
When querying by sketch, the user builds an image containing elements
which will be used to retrieve the sought image(s). This image (the
sketch) can contain edge information, color information, texture
information, region information, spatial constraints or predefined
elements with simple semantic meaning.
Queries produced this way are entirely visual, and are well-adapted to
databases in which only primitive features are associated to the images.
It also allows specifying clearly the position of the sought objects, as well
71
as spatial constraints between the elements in the sketch. Many current
systems use a sketch as query image. Among them the list can be QBIC,
PICASSO and SCORE. In this chapter, the different ways of [26]querying
by sketch, as well as extracting information from digital images so that it
can be compared to that coming from the sketch will be explored.
4.7.1 Producing a Sketch
There are many distinct ways to question a databank by delineation, row
from draft only the most influential margin to afflictive to sketch an image
that seem resembling the sought one. All the different ways of querying
an image database by sketch will be listed, along with their strengths and
weaknesses.
4.7.2 Drawing Edges
This techniques consists in drawing the contour of one (or more) of the
objects pictured in the image. The retrieval is then performed by trying to
find edges in the database images that match the ones in the sketch.
This matching can be done in a multitude of ways:[26] using classic
pattern matching techniques, comparing the direction of the edges in the
image to that of those in the sketch or, like in the ETM system, by
bending the contour produced by the user to make it match those found
72
in the images and then computing the energy that was necessary to
perform the deformation.
4.7.3 Predefined Elements
Querying using predefined elements consists in placing icons
representing entities (objects or concepts) on a canvas. Systems
supporting this kind of query are QBIC, Image Search (Plate 5) and
SCORE. Every icon corresponds to an entity that was recognized during
database population (for example: sky, sun, grass, face, water, human
face, bear, car, etc.), and information’s about the presence and the
position of these entities is stored with every database image. Database
images are then ranked according to the presence of the entities
appearing in the sketch and of their spatial relationships.
4.7.4 Geometric Shapes
Pictorial queries containing only geometric shapes are produced using a
simple paint tool (much like QBIC’s one [116], see Plate 6), and can be
used to specify the color of different regions of the target image, in a
rather imprecise way.
4.7.5 How Much Information do we need?
When developing a CBIR system allowing querying by sketch, one of the
questions that need to be answered is how much information is needed
73
in order to find the target images. In short, [27] we need to know if the set
of features extracted from the query should be almost identical to that
extracted from the database images, or if it can be a subset of it. In the
first case, one will say the sketch is complete, while in the second case,
one will call it incomplete.
It is clear that if the user is looking for groups of images then the
constraints will have to be looser, so the second solution should be
chosen. When looking for a particular target image, on the other hand,
both choices are possible.
If edge sketches and sketches built with predefined elements are by
definition incomplete sketches (i.e., not every element present in the
image descriptor must be in the sketch in order to consider the sketch
and the database image as similar) the same cannot be said of
geometric shape and freehand sketches.
As it will be seen in this section, incomplete sketches appear to be a
more flexible and easy to produce way to visually query an image
database.
74
4.7.6 Complete Sketches
A complete sketch generates a description which corresponds closely to
the one of its target images. In particular, if the sketch is composed of
edges, this means that the user is asked to draw every relevant edge and
not only those of some of the objects in the image. Intuitively, one can
say that such a way of querying the database will be useful only when
looking for a particular image.
The production of a sketch of this kind is bound to a series of problems:
1. time: drawing a complete sketch can take a large amount of time;
2. Precision: especially if the user is sketching from memory, certain
areas of the image will be drawn less accurately than others
because considered less relevant.
If some areas of the sketch are not faithful to the target image, chances
are that the latter will not achieve a good similarity score. Furthermore,
previous experiences showed that the areas considered as relevant are
in general smaller than those considered as not relevant, and that the
latter often correspond to the background. A sketch with large areas not
corresponding to the target image has few chances of resulting in a good
set [27] of retrieved images (Plate 42).
To retire this proposition, a saliency perception technique would be
required, and only ability of the sketch answering to bounding ability of
75
the databank images should be necessity to assess the similitude
between depict and database management system cast. This
procession, though, part of the use’s toil would come unprofitable.
4.7.7 Incomplete Sketches
If the user is allowed to produce a sketch which generates a subset of
the features used to represent the target images, he/she can obviously
concentrate on the parts of the image that are most relevant to him/her
(see Plate 43 for an example). This way, all the areas painted by the user
can be considered relevant (assuming that, if they were not the user
would not have painted them).
The images in the database will be ranked according to the similarity
between the painted regions in the sketch and visual information
contained in the images. This comparison can be done on a local basis
(like in PICASSO, in which the comparison is done on a region by region
basis) [104] or in a more global way (in their system, Jacobs et al.
compute a description for the whole image which is based on a wavelet
decomposition). Comparison on a local basis will in general require
extraction of region information from the database images (for example
with a segmentation of the database images into regions); the
comparison between regions will then be based on:
1. color: histograms, distribution, average, etc.;
76
2. spatial position;
3. shape;
4. size;
5. Texture information; etc.
Spatial relationships between the regions can also be part of the
similarity computation. While incomplete sketches can be used to retrieve
a particular target image, they are a more powerful retrieval method, as
they can be used to cluster images sharing given visual features. By
sketching only a few regions, the user can retrieve all the images
containing the same visual information (like all images with the sun in the
top left corner [28] and a textured green patch in the bottom right one).
By carefully choosing the image-to-sketch similarity function, it is possible
to look for images containing the depicted areas exactly as pictured, in
any part of the image, in any spatial arrangement, with different sizes,
and so on. It is easy to see, then, that querying by sketch is a very
flexible solution.
Incomplete outline also have questionable issues accompanying to the
integrity of pigment, largeness and design of the portrayed areas,
compared to the conformable areas of the aim image(s), as discernment
of pigment in the human is a difficult procedure, entangle material and
psychological elements. Notably, the understand pigment of an object
77
rely on its whiteness, the brightness, the superficial of the end and
reflective knowledge approaching from other appearance, just to relate
some of the external elements. When the user paints an object in
isolation, he/she will try to match the color he/she understands in the aim
image with the one he/she understand in the outline.
Despite these shortcomings, querying by incomplete sketch seems to be
the most powerful way to query an image database in a visual way,
allowing searches for both particular images and categories of images.
The method is intuitive, and production of the query image can be
extremely fast.
4.7.8 Extracting Comparable Information from Sketches and
Database Images
Sketches and digital images have different characteristics, yet a human
being will in general be able to say that a sketch represents a particular
image, or at least that an object contained in a sketch corresponds to an
area of the image. This means that some features are conserved when a
user paints a sketch of a particular image or of a part of it. This problem
does not apply to contour sketches and to sketches built using
predefined elements, since in the former the information extracted from
the sketch already is of the same kind as that extracted from the
database images, while in the latter there is no need to extract new
78
information from the sketch, since the [71] different icons correspond to
entities which are represented by feature values. Hence, the only kinds of
queries concerned are the ones where painting is needed.
4.7.9 Extracting the Information from Geometric and Freehand
Sketches
In the early stages of this work, it is observed that, even when given a
powerful image production tool like Adobe Photoshop, the users tend to
paint sketches composed of patches of uniform color to represent
homogeneous areas depicted in the images. Sketches produced this way
bear a strong similarity to digital images having undergone a coarse
region segmentation, or being reconstructed starting from the most
important elements of a wavelet decomposition, as Jacobs et al. point
out. Hence, information comparable to that found in this kind of sketches
can be extracted from the database images using one of these two
techniques. Coarse segmentation, though, seems to have a slight
advantage over wavelet decomposition, since region information can be
made invariant to translation, rotation and scaling, while information
coming from wavelet decompositions cannot.
79
4.8 Image Features For CBIR
Image features affect every aspect of a CBIR system, and so it is
important to carefully choose the right image features for any CBIR
system. Most of the CBIR systems explore low-level image features such
as color, texture, shape, motion, etc. because they can be computed
automatically.
Middle-level features like regions and blobs which can be generated
without human assistance are used in [71] object-level image retrieval.
The characteristics of some of these features, focusing mainly on how
they are extracted and compared are discussed. The CBIR systems
discussed will be query-by-example (QBE) style, unless otherwise
specified.
4.8.1 Color
Color is one of the most prominent perceptual features. Most commercial
CBIR systems include color as one of the features (e.g., QBIC of IBM,
Virage, etc.) The easy-to-compute color histogram is a popular and
widely used image feature. To use color histograms for image retrieval,
there are three basic steps: (i) Color space quantization — for a choice of
color space C (e.g., RGB, HSV, LXY, CIE, Munsell etc.), the division of C
into m levels, c1; c2; : : : ; cm; (ii) Histogram binning — a linear-time scan
80
of the image to count the number of color pixels in each color bin; and (iii)
Histogram matching — choosing a distance function D to measure the
similarity of two histograms. The choice of the color space C, the different
ways of dividing the color space (say, uniformly or non-uniformly), and
the choice o the distance function D. There is a brief discussion of two
CBIR systems.
4.8.2 Find it
The parameters in FINDIT, developed by Swain et. al. are (i) HVC (Hue-
Value-Chroma) color space, (ii) histogramming Hue-Chroma, and (iii) the
histogram intersection distance function D:
( ) ( )( )( ) ( )( )( )∑
∑=
== m
j j
m
j jj
Qh
QhIhQhIhD
1
1 ,min,
(4.1)
where h(I)j; h(Q)j are color counts of color j in image I and Q. Note that
this definition is not symmetric in I and Q; thus D is not a metric.
To retrieve answers for one query, the above distance function takes O
(mN) time, where m is the number of colors and N is image quantity in
the database. To save this expensive on-line computation, only
prominent colors can be used in the summation of Equation 3.1. This
decomposing step reduces the comparison to ( )BNmmO +log if B largest
81
number of color used. If B=o(m)then the retrieval could be faster, at the
expense of some false positives, QBIC. IBM’s QBIC is a CBIR system
that exploits multiple image features including color histogram, texture,
and shape. It allows QBE and drawing a picture. QBIC uses color
histogram in the following [78] way: (i) RGB color space, (ii) non-uniform
partitioning into 256 clusters based on the perceptual distance of colors
contained in the RGB space, and (iii) a quadratic distance function to
compare histograms:
( ) ( )( ) ( ) ( )( ) ( ) ( )( ) ( ) ( )( )∑∑= =
−−=−=m
i
m
j
Tiiij QhIhAQhIhQhIhaQhIhD
1 1
22 , (4.2)
The matrix A indicates the quantities of perceptual similarity between two
colors. Therefore D2 not only compares the corresponding color bins, but
also compares different color bins. For instance, this measure can be
made to indicate that orange is equivalent to red than to blue. Since D2
involves the whole matrix-vector multiplication, it is not decomposable as
histogram intersection. The computation of one query takes ( )Nm20 which
is not efficient. To summarize, the color histogram is suitable for
retrieving images based on overall color impression. Since, it does not
include any spatial information it has limited image discriminative power.
4.9 Summary
This chapter discusses the primary concept of visual art images with
respect to digital image processing and its possible utility in designing a
82
Content Based Image Retrieval (CBIR) system. The chapter also
presents various factors that need to be considered while developing a
framework for feature set pruning in CBIR modeling.
83
Chapter 5 Content Image Retrieval Approach
(COIR)-(COIR Scheme)
5.1 Introduction
The proposed discusses about the significant implementation of the
content-based image retrieval process considering the case study of
visual art images. At present, majority of the image search tool (e.g.
Google) uses annotation or text based tags on the images to perform
search. The proposed system differs from the existing one by
incorporating a characteristics where contents as well as unique features
are used for searching [40] the relevant images. Complexities of visual
art images are already discussed in the prior chapters.
The process of retrieval is extremely significant as it is dependent of
various study parameters e.g. color of visual art image, its quality of
texture, dimensionality, intensity etc. These parameters are the
conventional parameters used for the retrieval of the visual art images by
performing similarity match with massive dataset of visual art images that
are trained using Support vector Machine and Principle Component
Analysis.
84
Finally, the study also uses human judgment in the form of relevance
feedback which further strengthens the accuracy of the outcome of the
study. Another uniqueness of the study is use of unique process of
feature extraction using two different types of visual descriptors. The next
section will brief about the methodology used to accomplish the study.
5.2 Research Methodology
The proposed system adopts the mixed mode standards of research
methodology using empirical approach and analytical approach. The
adoption of empirical approach is required as the proposed study is
essentially based on standards of content-based image retrieval system,
which follows certain scientific methods of extraction of unique
information between the dataset and query image for performing the final
image retrieval process. Analytical method further performs analysis of
the empirical approach and ensures better validation approach and
standards adopted in the proposed work.
The design of the proposed system is carried out in Matlab considering
the standard dataset of visual art images. The reason for choosing
Matlab is only because of its potential supportability of complex
mathematical and its effective enrichment of signal processing [9]
designs. The research methodology is also illustrated with respect to its
adopted functional and non-functional dependencies:
85
5.3 Functional Dependences
The functional dependencies of the proposed study are as follows:
1. Similar feature extraction process in similar sequence has to be
used both at the time of designing the dataset as well as
performing the query of the image.
Figure 5.1: Schematic diagram of the COIR system
86
This section describes the various procedures that are adopted in the
due course of investigation process for the objective of accomplishing
research goals mentioned in previous section. Hence the (COIR) study
can be discussed under structured stages of research methodologies as
disused below:
Stage-I: Applying learning techniques on proposed system
The initial stage of the proposed study has introduced a novel technique
that introduces a framework for retrieving various forms of visual art
images. Before applying the learning techniques, it is important that
grouping of the images are done appropriately. As there are various
forms of the visual art images, hence, the proposed system performs a
categorization of the different types of images available into specific
groups. This forms of grouping also assists in ranking the visual art
images most appropriately in the later stages. The technique associates
retrieval of the images with the specific groups that are newly formulated.
The set of the image group that are newly formulated can be used
directly for eliminating the possibility of the redundant visual art images.
The system than performs fine tuning of the feature vectors in the form of
linear combinations of the resemblance match. For ensuring the
preciseness of proposed system, relevance feedback is introduced for
the purpose of upgrading the weights of the features. The weights of the
features are organized in the form of positive and negative ranges of
87
feedback. The proposed study uses supervised learning algorithm for the
purpose of better classification of the visual art images. Better
classification process will essentially result in narrowing [99] down the
similarity match outcomes. The concept of relevance feedback as well as
supervised learning technique may also result in generations of false
positive. However, the entire false positives are significantly eliminated in
the final stage of the study. The problem formulation in this part of the
study will consist of understanding the critical factor of precise clustering
mechanism, recognition rate, usage of colored image, massiveness in
the database size etc.
All these information were gathered in one location for understanding the
difference in the prior techniques with the existing one. Completion of this
stage of research work meets the first study objective.
Stage-2: Using k-means Clustering
The next consecutive study phase consists of analyzing the visual art
images using clustering mechanism. The completion of the prior research
stage exhibited that prime motive of CBIR techniques calls for specific
indexes representing a unique characteristics of an image. This stage
also invokes the visual art images to be segregated into a multiple blocks
of 4x4 dimensional in terms of pixels. It also considers a feature vector
for all the partitioned blocks of images, where each block of image
88
posses around 6 image pixels which are required to be abstracted. The
operations on color space conversion are followed after this stage. The
proposed system consists of LUV color space that essential consists of
luminance, hue, saturation, and chrominance. The color space are
applied on the visual art images considering [105]16 blocks of pixels
occupied in 4x4 block size. The system also uses a transformation
technique using 2-dimensional approach of Haar wavelets. This specific
form is subjected to the decomposition using the transformation
technique. Using Matlab, the transformation is carried out to support 8
level of decomposition.
The initial processing is carried out in one level of decomposition on the
4x4 block size. The decomposition is carried out on 4 frequency bands of
block size 2x2. Ultimately, a k-means clustering technique is applied on
the proposed system for the purpose of carrying out clustering the
feature vectors. This clustering is carried out over numerous classes of
images with every class equivalent to the single region over the image
that is segmented significantly.
The study in this stage consist of core 3 implementation steps:
1. Exploring the positional value of centroids.
2. Identify the spatial distance for all the existing objects from the
centroids.
89
3. Cluster the objects presents completely based on the decreasing
order to distance.
This stage formulates a novel algorithm that determines the empirical
value of number of regions i.e. k, which is required for carrying out image
segmentation. The process of segmentation is accomplished again
based on the heterogeneity of the digital contents present in an image.
The significant steps in this process are as follows:
Perform feature vectors were segmented considering value of K
equivalent to 2.
Addition of the spatial distance is carried out for every feature vectors
positioned over clusters with respect to their distance. This information is
extracted and compared with the analytically evaluated critical value.
In case of occurrences of multiple varieties of image, there is a higher
feasibility that feature vectors presents in a single cluster may differ
significantly. In such condition, the addition value will be greater than the
critical value.
The segmentation is carried out for the feature vector if added value of all
clusters is found maximum than the critical value. In such stage, the
value of K is increased to 1.
90
Repetition of 2nd and 4th stage is carried out until the added value
becomes equivalent or less than critical value.
Stage-3: Using Block Truncation Coding:
This stage of the study has carried out CBIR technique over visual art
images using block truncation coding. This stage is a continuation of
systematic evaluation of proposed CBIR technique using conventional
technique. In this technique, block truncation coding is again one of the
conventional techniques. This part of the system also applies
conventional k-means technique for carrying out clustering over visual art
images. This stage of the implementation imposes a technique that
extracts the features of the colors and then the respective moments of
colors are evaluated. This can also be adopted for differentiating images
which is based on color features.
Descriptive statistical techniques were adopted for evaluation of
moments using mean, standard-deviation, [45] and skewness.
Interestingly, block truncation coding is a compression technique usually
applicable over non-colored images. However, the proposed systems
uses block truncation coding for extracting the valuable feature
information, makes the image lightweight, and carry out CBIR operation.
Although color images were used in the experiments, but the system in
this stage uses a significant quantizer in order to minimize the quantity of
91
grey levels for every blocks. Interestingly, the system doesn’t change the
value of mean and standard deviation. The significant steps for carrying
out block truncation coding are as follows:
Stage-4 Using Contribution Approach
This study phase is completely different from the other prior study stages.
The prior study stages uses k-means clustering mainly, which is not
extended in this stage. Inspite, a novel clustering algorithm is designed
92
with both inter and intra clustering operation. This stage is called as
contribution-based approach that we use for carrying out CBIR technique
on visual art images.
In this study phase, a new form of clustering technique has been
designed considering the idea of data points in one cluster that is termed
as contribution. The technique is essentially applied to extract the
relevant images matching in context with the clustering operations.
The behavior of the proposed clustering technique is almost an
enhanced version of k-means clustering technique that has been
implemented in the prior study stages. The parameter contribution is
used to understand the level of quality that it can incorporate on one
cluster. This parameter is the applied in order to get the better group of
particular clusters when certain set of data points are already defined.
The system can be said to implement a squared-error based logic in
order to get the outcome. A simple mathematical equation is designed for
carrying out CBIR technique of [29] visual art images along with condition
setting for positive and negative contribution data points. If the
contribution point is found to be positive, it would reflect the quality
influence on the particular cluster under study. While if it shows the
negative contribution point, it would reflect the adverse influence on the
particular cluster under study. This part of the study stage ensure that if a
93
negative data point of the contribution is found, the data point is instantly
shifted to the respective cluster where its contribution is highest.
However, in case of positive data point in contribution, the system
considers multi-objective optimization criteria by optimizing both
heterogeneous as well as homogeneous cluster dispersion.
94
It can be seen that δnew and φnew are the updated values of δ and φ after
the data point Dp is shifted to the cluster CLp[76].
5.4 COIR Model : COIR
The prime purpose of proposed system is to perform a categorization of
the visual art images using the novel design of the CBIR techniques. Just
like conventional CBIR technique, the overall operation of the proposed
system is also similar. The uniqueness lies in implementation over visual
art images, which was never done before. The training of the proposed
95
system is done using supervised learning algorithm using Support Vector
Machine and Principle [76] Component Analysis. The system also
performs the similarity fusion considering the concept of statistical
techniques. Support vector machines have the capability to perform
classification on both linear as well as non-linear problems. The proposed
system uses multiple features of visual art images for the purpose of
subjecting it to the training characteristics.
The system considers training the visual art image one by one using
supervised learning technique. For better ranking of the similarity feature,
the proposed system performs group wise training as well as query
system. The system also filters out all the redundant information from the
database itself resulting in highest accuracy of the queried image that is
untrained. Once the features are extracted using dual mode of feature
extraction policy using visual descriptor e.g. color layout descriptor and
edge histogram descriptor. All the features are ultimately integrated using
similarity matching technique for understanding level of effectiveness in
accuracy level. The study was carried out on dataset of 1000 visual art
images extracted from Yale University Art Gallery. Extensive study and
analysis were performed using this dataset.
96
5.5 Summary
This chapter discusses the COIR system that introduces modeling of
COIR learning based framework for pruning feature set in CBIR system
for visual art image. Supported by architectural description, the chapter
also highlights the research methodology adopted to design the COIR
model and evaluation techniques applied.
97
Chapter 6 Implementation of COIR
6.1 Introduction
In present day, the machine learning technique has evolved up
drastically giving rise to sophisticated system design. The areas of image
retrieval process have found that usages of low-level features are
absolutely not useful and it should adapt itself to novel CBIR techniques
that give better reliability factor. Although training is an integral part of the
CBIR technique, but more important process in the training is type of the
image feature where the training algorithm should be implemented. The
proposed model uses an empirical methodology for the purpose of
extracting multiple forms of [10] image features for the purpose of
subjecting it to the supervised learning algorithm. The system uses
Support Vector Machine for this reason.
While developing the proposed system, various challenges of the visual
art images were considered. It was already known that color quality in
visual art images differs significantly from the natural images. Hence, the
challenge scenario surfaced up stating that it is quite difficult to analyze
visual art images for following research challenges:
98
• The Yale dataset used for the experiment consists of around 1000
visual art images, where all the images form a very abstract
meaning with confusive object representation.
• Grouping of the massive number of images requires significant
characteristics which was very hard to find from visual art images.
• All the images are of different size and shape with different scales
of color intensities.
All these problems will essentially lead to the false positives over the
system while passing a queried image as input. Another significant
research problem will be how to use the feature extraction method and
which one should be used. In the proposed system, visual descriptors
were used for the purpose of feature extraction. The significance of the
usage of the visual descriptors is that it can extract multiple formats of
features e.g. edges, color, etc. For better effectiveness of the study, the
implementation of the feature extraction is done carried out using color
layout descriptor and edge histogram. Once the feature extraction is
carried out, the features were trained using supervised learning approach
using Support Vector Machine and Principle [106] Component Analysis.
Hence, the database is built up. Once the entire images were trained, the
system uses some of the untrained images (considered as queried
image) for checking the effectiveness in image retrieval process. The
effectiveness is calculated from the accuracy and precision scale
99
obtained. This chapter will essentially discuss elaborately how the
proposed system is implemented as well as designed for the perfection.
6.1.1 Image Feature Representation
The first stage of the proposed system is to perform an effective
representation of an image. An effective representation of an image is
highly essential in order to process the complex quality of an image just
like visual art images. The system digitizes the visual art images and
stores the image information in the format of a matrix. This step is highly
essential as quantity of visual art images are high in number and this
step is also followed by grouping and feature extraction. Therefore, the
significance of the image representation is quite high as the entire
classification process to be maintained during the image retrieval process
highly depends on it. The inclinations of the dependencies are totally in
the representation of feature vectors.
The system carried out the supervised training process, where the
preliminary input for the system is considered to [107] be the set of
feature vectors. An annotation process of each visual art images are
maintained in the dataset and the labeling of the concept is carried out by
uniform partitioning of all the visual art images. Mathematically, the
concept probabilities of the proposed system can be represented as
100
( ) iLXiyPik kiP ≤== 1,, (6.1)
In the above eq(1), P is the probability factor for all the image Ij
positioned in { x1j , . . . , xkj , . . . , xlj }, where xkj∈ Rd is a term that jointly
signifies feature extracted from the information contained in text and
color of visual art images. The color features are elementary color
constituents while text features are considered by the name of groups of
visual art images during training process. Ultimately, the grouping will
assist to narrow down the search space while performing similarity
matching of the trained image with the queried input image. Depending
on the signified encoding scheme, the function to represent weights in
supervised training can be empirically drawn as,
[ ]TLjijConcept
j wwwf ,....,......,1= (6.2)
where each wij denotes the weight of a concept ci , 1 ≤ i ≤ L in image Ij,
depending on its information content. The proposed system also uses a
scheme that is equivalent to term frequency as well as inverse term
frequency. The variable wij is mathematically represented considering the
product of global and local weights.
When there are presences of various forms of visual art images, there
are also multiple modalities along with multiple views and variable
illumination condition that are present in the considered dataset. Most
101
importantly, the visual representation of visual art images is anticipated to
be highly flexible enough for catering up the massive varieties of
[11]multiple instances of images. Hence, it is quite important that
grouping of such massive number of visual art images are done in order
to reduce the search space. Hence, the proposed system performs
grouping of the all the images while performing the training operation
using unsupervised approach and also when it is a time to feed the query
image.
The proposed system uses a highly efficient design of invariant
characteristics from an image and performs extraction of features using
non-conventional visual descriptor. The brief discussions on visual
descriptors used in the design of proposed technique are discussed as
following:
• Color layout Descriptor: This type of visual descriptor is designed
to extract the significant information about the spatial distribution of
the colors from an image. After performing digitization of the visual
art images, this visual descriptor performs blocking of the image (to
be trained as well as queried image too) followed by quantization
with conventional Discrete Cosine Transformation (DCT). The
features are extracted [79] by allowing the visual art images to get
partitioned into specific number of blocks (See Fig.6.1) followed by
color space representation. The processed image is then
102
subjected to transformation using conventional Discrete Cosine
Transformation followed by reading the pixels in zig zag manner
for faster processing.
Figure 6.1: Partitioning in Visual art image using CLD
• Edge Histogram Descriptor: Just like color, edge is considered
as a prominent feature for processing in the proposed system. The
design of the proposed system is carried out considering 5
different types of edges e.g. 4 edges of directional type and 1 with
non-directional type. The directional edge considers the direction in
terms of vertical, horizontal, 45 degree, and 135 degree. The block
consisting of an image is further partitioned to 4 sub-blocks to
obtained their mean value.
The feature vector extracted from the adoption of visual descriptors (color
layout and edge histogram descriptors) will be subjected for the
supervised training process. All the feature vectors are stored in matrix
format and is manually annotated for all the considered categories of
103
visual art images. The system consider a variable M for the categories to
[41], [5], [12], [7], [59], [72], [60], [31], [80], [46] the opted for while
training visual art images. Consider that M is a set of labels empirically
represented as {ω1, . . . , ωi, . . . , ωM }, where the variable ωi represents
the category of a global image. Hence, if a feature vector x is considered
that the probability of each category can be mathematically represented
as
( ) MmforXwypp mm ≤≤== 1, (6.3)
The above equation (3) represents a maximum probability for choosing
the final groups of the visual art images.
6.1.2 Classifier Combination
The proposed system has used visual descriptors that are used for
extracting the significant feature information from the visual art images.
The image data is basically represented by the visual descriptor and
highly assists in efficient classification. The system performs dual
techniques for applying feature extraction process e.g. appending the
visual features to an elongated integrated vector form and deploying the
multiple classifier on single feature vector. The sequencing of the visual
features plays an important role in classification process. The proposed
system adopts support vectors from multiple classes and then performs
fusion of the various support vectors accomplished from the processing.
The resultant support vector is finally responsible for classifying the
104
feature vectors of all the trained images which assist in similarity
matching process.
6.1.3 Image Filtering
The proposed has adopted the usage of the group forecasting the query
as well as the trained image for image filtering to reduce the search
space is utilized. The output of the previous classification approach to
form M-dimensional category vector of an image Ij as follows:
[ ]TjMjmj pppjP ,...,,...,1= (6.4)
In the above equation (4), the elements represent the probability factor of
the value of class confidence that belongs to an image Ij to a specific
group ωm with respect to the visual features. The indexing mechanism in
offline takes place by storing the trained image with specific indexes on
each group of images so that retrieval process consumes less time to
search in problem space. When the system is searched based on an
unknown query image, similar feature extraction and category prediction
stages are performed online. The category vector of a query image Iq
based on (4) and the similar vectors of the database images are chosen
to identify the best target image similarity match with the trained image.
The proposed system considers n<M as the nearest classes of the query
and the trained image. Normally, the value of the nearest classes n is
105
maintained with smaller number to the cumulative quantity of the M
classes for the purpose of neglecting the distant classes. The filtering
process is followed along with this. The condition of selection of the
trained image from the trained dataset is presence of at least one
common group between the trained and queried image itself. Hence, not
all the trained images in the dataset is search saving lots of
computational time and memory.
6.1.4 Similarity Fusion
Similarity fusion is the adopted technique used in the proposed system to
carry out internal search for the factors matching between both queried
image as well as set of trained images. This phenomenon is also called
to be used for comparative analysis. There are multiple availability of the
feature descriptors in the area of CBIR technique. The proposed system
designs up a technique where fusion of two similar features can be used
for performing image retrieval process. The proposed system uses linear
blending of various similar feature vectors with well-defined weights. The
formulation of the similarity fusion of the queried Iq and objective image Ij
can be mathematically shown as ,
( ) ( )∑=F
jqFF
jq IISIISim ,, α (6.5)
The similarity matching function can also be said to a set of following
elements (featureCLD, key point, featureEHD) and Euclidean matching
function. A specific form of weight is hard coded on the multiple image
106
representation schemes in the proposed study. The individual feature
vector is computed for the individual queried image and training is carried
out using supervised learning approach. All the resultants are the
integrated considering the min-max rules.
As visual art images are highly confusive images in terms of colors, there
is a probability that it may lead to false positives. Hence, the proposed
system will use relevance feedback as a second level of implementation
to further confirm the relevancy of the outcome. The relevance feedback
is incorporated by the interface by allowing the user to select the image
by the user based on its similarity observed visually. The system then
performs ranking of the images to be shown as an outcome. Hence, the
outcome is shown with respect to ranking value, where lower ranks
represents lower similarity and higher rank represents higher similarity.
The feature weights are then updated. For all the returned images, the
user will be required to provide the feedback. The effectiveness of the
relevance feedback is mathematically expressed as
( ) ( )kpk
iRankE
K
t ∗= ∑ =
2/1 (6.6)
In the above equation (6),
The normalized values of the above formulation are considered while
computing weights in the consecutive iterations at the time of retrieving
the images.
107
( ) ( ) ( )∑∑ ==F
jFF
qFF
jqFF
Fjq ffSIISIISim ˆ,ˆˆ,ˆ, αα
Where 1ˆ =∑ FF α
The significant action required for updating the weight factor as well as
performing matching the similarity between the queried image and
trained images can be shown as following algorithm.
6.1.5 Algorithm using COIR
Evaluate the first K-number of images using similarity fusion depending
on the similar weighting of image features.
1. Extract the user feedback
2. Estimate the novel feature vector as the mean vector
corresponding to relevant images.
3. Perform ranking based on sorting of weights in increasing order.
4. Consider all first k-images in each ranks.
5. Normalize the weight factor within 0-1
6. Consider updated weights as normalized score.
108
6.2 Summary
This chapter discusses about the experimental test bed created along
with the discussion on the input of visual art images that has been
considered for evaluating the COIR model. The chapter also presents
various algorithm details responsible for the design of the COIR system.
109
Chapter 7 Results And Discussions
The previous chapter has discussed about the technique that was
adopted for implementation of the proposed COIR system with an aid of
algorithms and all the tools adopted for accomplishing the study. This
chapter will discuss about the outcomes being accomplished from the
implementation being undertaken in the study. The chapter will illustrate
the results and discuss the outcomes abstracted from the analysis.
7.1 Results Accomplished From COIR Model
As seen in the implementation technique, the proposed COIR system is
analyzed over the common form of the machine that is expected to have
wide availability. The minimum system specification to accomplish the
final results is just i3 processor and minimum of 4 GB of RAM on
windows machine with Matlab. All the visual art images were trained
using supervised machine learning algorithms and then a query image is
passed over to testify its accuracy in the retrieval process. The system
also uses two different forms of visual descriptor for the purpose of
extractions of features. These features underwent training using Support
Vector Machine and were used for the purpose of similarity check for the
queried visual art images. However, it includes various forms of
intermediate steps that is already discussed in previous chapter.
110
(a) (b)
(c) (d)
Figure 7.1: Visual Outcomes after applying Visual Descriptors
Fig.7.1 highlights the visual outcomes being abstracted after performing
extraction of features using visual descriptors. The proposed system has
111
deployed two forms of visual descriptors viz. Color Layout Descriptor and
Edge Histogram Descriptor.
Fig.7.1 (a) shows the queried visual art images, which also means that it
is untrained image and this image doesn’t exist by any form in the trained
dataset over the machine. The system considers the input image and
then applies Color Layout Descriptor for extracting the spatial distribution
of the significant color contents of the visual art images. As discussed in
the preliminary chapters that visual art images are quite different from
natural images with respect to the contents and mixtures of the colors
used in the images; therefore, color layout descriptor was found to
significantly eradicate this constraint. The process of partitioning using
Color Layout Descriptor is used for accomplishing 64 blocks of the input
visual art image, as shown in Fig.7.1 (b). Once the original input image is
partitioned by the size of 8x8 blocks of non-overlapping type, the system
accomplishes the coefficients of this visual descriptor as seen in
Fig.7.1(c).
In order to perform much precise extraction of feature, the proposed
COIR system also uses another visual descriptor called as Edge
Histogram Descriptor separately from Color Layout Descriptor. The
outcome of the Edge Histogram Descriptor can be seen in Fig.7.1 (d).
Hence, now the system has two forms of features for the same input
image (queried image), one accomplished from Color Layout Descriptor
and another accomplished from Edge Histogram Descriptor. Both the
112
forms of features are now stored in a matrix, where it undergoes
supervised training using Support Vector Machine and Principle
Component Analysis.
It should be noted that while performing training of the visual art images,
all the trained images were recorded in specific names of groups that will
make the extractions of the images much easier. The similar process is
maintained even for the queried image, where a specific group name has
to be checked. The group name is a specific string that describes the
generality meaning of the set of visual art images under them. For
effective analysis of the proposed retrieval system, the different forms of
the trained images undergo the training using Support Vector Machine
and Principle Component Analysis. Both the training and retrieval
procedure undergoes similar processing operation for feature extractions.
The next operation will be to carry out the process of checking the
similarity score of the queried input image with that of the trained dataset.
The queried image will undergo the process of feature extraction using
both Color Layout Descriptor and Edge Histogram Descriptor, which are
then matched with the trained features already present in the dataset for
matching the similarity. Successful similarity matching will result in
generation of set of similar images matching with the dataset. For the
purpose of effective analysis of the outcomes, the retrieved images are
indexed with ranking numbers in order to highlight the best matched
images in sequence. This operation also renders user-friendliness of the
113
proposed COIR system and thereby assists in analyzing retrieval of
similar trained images from the dataset.
Input Image 1st-Rank 2nd-Rank
3rd-Rank 4th-Rank 5th-Rank
6th-Rank 7th-Rank 8th-Rank
9th-Rank 10th-Rank 11th-Rank
Figure 7.2: Visual Outcomes of Retrieved images with Ranking
114
Fig.7.2 shows the input image that acts as query image. After the queried
image is subjected to the visual descriptor, it is matched for similarity with
the trained features in the dataset of the visual art images. The outcome
shows the retrieval process where the displays of the images are
arranged in the order of the ranking score.
A closer look into the outcome will show that the resultant images are
visually not compatible or even a near similar to the input image. This is
one of the biggest challenges in the outcome analysis. The ranking
system assists to filter the massive features existing in the datasets that
closely matches with queried image feature. However, as the features
are mainly of color and edge type, hence, it is quite possible that certain
false positive may exists in the outcomes. Hence, in order to overcome
this problem, the proposed COIR system applies relevance feedback
process.
The CBIR process used in the process study are highly enhanced to get
executed for complex visual images. The advantage of the system till
now is i) its capability of extracting features for large number and
complex forms of visual art images and ii) its capability of narrowing
down the search for similar images from the massive dataset of visual art
images. However, its limitations will be minor false positives as the
feature extraction is done using color and edge perspective, which may
be found similar and redundant in many cases of trained images. Hence,
relevance feedback aims to solve these limitations.
115
Relevance Feedback is the process that enables the user to visually or
analytically judge the similarity criteria of the retrieved data. The
proposed system adopts the use of explicit feedback mechanism from
the user for showcasing the relevance of the visual art image being
retrieved from the trials of query processing. It finally aims to remove the
redundant or false positive images in the retrieval process and thereby
acts as final stage of fine tuning the outcomes. The proposed system
performs relevance feedback only after the resultants of retrieved images
(as shown in Fig.7.2) are obtained. For this purpose the user selects the
ranked images manually either to retain it or to remove it from the list.
Fig.7.3 shows the process of Relevance Feedback adopted by the
proposed COIR system.
Figure 7.3: Outcome of Relevance Feedback Processing
116
While performing the operation of the relevance feedback, it is important
for the system to consider the rationality of the user to choose the precise
retrieved image that appropriately is found similar to queried image.
Fig.7.4 thereby shows that after the user selects their appropriate images
using the operation of the relevance feedback, the outcome of the study
is found with maximum number of images with highest similarity scores.
However, the deployment of the visual descriptors as well as the
supervised training methodology adopted in the proposed system assists
the model to minimize the false positives to large extent. Hence, even if
the user makes slight inappropriate selection during relevance feedback,
the feature matching is done precisely to ensure that only the best
matched features are being chosen while performing retrieval process.
Figure 7.4: Final Outcome of the COIR Retrieval System
117
7.2 Comparative Performance Analysis
The previous section has discussed the visual outcomes of the proposed
COIR system. However, it is quite essential that the outcomes of the
proposed system be compared with the existing system in order to
understand its effectiveness. Hence, choosing the existing system is
quite a non-trivial task as the existing system has to match with the
proposed research goal very closely although it is quite possible that
existing system adopts different research methodology. Hence, the
proposed study has shortlisted 3 significant researchers for this purpose.
The informative details of these researchers are as follows:
1 Narasimhan Approach: The author Narasimhan [73] has introduced
a technique of content-based image retrieval system, which is quite
similar to the proposed study. The technique of image retrieval is
accomplished by using the concept of contribution of the data point.
The author has presented an enhanced clustering mechanism that
always intends to optimize the inter and intra-clustering mechanism.
The significance of this technique is that it just requires few trials to
achieve similarity matches. The outcomes of the study were
evaluated with respect to recall and precision factor.
2 Silakari Approach: The author Silakari [91] has presented a study
of retrieval of image using block truncation coding. The study has
118
adopted an unsupervised mechanism of performing image clusters.
The study has also considered color as the features for extraction.
The dataset is built over the color features and block truncation
coding is applied. The system also deploys the conventional k-
clustering technique on the extracted dataset. The outcomes of the
study are found to be quite satisfactory in terms of retrieval system.
3 Biswas Approach: The author Biswas [81} has designed a system
that performs retrieval of the images that are found with higher
degree of equivalence with the queried image. The system was also
found to be experimented over a large scale of dataset of distinct
images. The presented design also uses discrete wavelet transforms
for decomposing the image into smaller signals (or coefficients). The
approach presented by this author typically applies segmentation
process for the purpose of extracting various forms of features from
the images. The outcomes claimed by the author shows better
similarity matches.
Hence, it can be seen that all the above three approaches are quite
simple and yet claim better outcomes when it comes to similarity
matches. Hence, these three existing works are considered for the
purpose of comparative analysis. It is also observed that all the above
techniques bears a strong resembles with the pruning technique in CBIR
119
approach. Table 7.1 summarizes the techniques adopted in each cases
of comparative performance analysis.
Table 7.1: Summary of Adopted Techniques for Comparative Analysis
Authors Techniques Adopted
Narasimhan [73] Contribution Based Clustering
Silakari [91] -Block Truncation Coding
-k-means clustering
Biswas [81] -k-means clustering
-Discrete Wavelet Transform
Proposed COIR Method -Visual Descriptors
-Support Vector Machine
-Principal Component Analysis
-Relevance Feedback
-Image Filtering
The process of comparative performance analysis is performed using
precision and recall rate.
• Precision: It is defined as cumulative quantity of extracted similar
images divided by cumulative quantity of extracted images.
• Recall: It is defined as cumulative quantity of extracted similar images
divided by cumulative quantity of similar images.
Moreover, the similar parameters were also adopted by the prior
researchers, which make the task of comparative analysis more
reliable and simple.
120
7.2.1 Outcomes of Individual Approaches
As discussed in the previous sections that it is imperative to perform
comparative analysis of the proposed system with the existing
approaches. However, before performing comparative performance
analysis, this section will discuss about the individual outcomes of the
existing study. As a recapitulation, it was seen that Narasimhan [73] has
used Contribution-based approach, Silakari [91] has used Block
Truncation Coding and k-means approach, and Biswas [81] have used
conventional k-means clustering and Discrete Wavelet Transformation.
The analysis of all the approaches was done over 804 visual art images
of multiple heterogeneous groups from the Yale University Art Gallery
dataset . The discussions of the individual outcomes of all the above
mentioned highlighted techniques are as follows:
7.2.1.1 Outcomes of Contribution-based Approach
The contribution-based approach when implemented over the visual art
images is found to extract the object information present in salient
foreground. Clustering is performed considering preliminary centroid
selected randomly. The system then determines the cluster located in the
proximity of the centroid. Inter-clustering and inter-clustering were
performed and checked for the images that belong to the specific
clusters. The images were then retrieved and their recall and precision
factor is determined. The visual outcomes of this approach is shown in
Fig.7.5.
121
Input-Query Image
Retrieved Image using Contribution-based approach
Figure 7.5: Visual Outcomes of Contribution-based approach
A closer look into the visual outcomes shows that majority of the
retrieved images are found to be equivalent to the input of queried image
of visual art. The analyses of the individual outcomes were performed on
22-categories of visual art images. Fig.7.5 shows the example of few
categorizes out of 22 total categories testified. The performance
parameters are selected as precision and recall factor in terms of
percentage.
The numerical outcomes achieved from the implementation of
contribution-based approach on 804 visual art images are highlighted in
Table 7.2.
122
Table 7.2: Result for Contribution based Feature Set Pruning
Contribution Based Approach
# Categories Recall (%) Precision (%) 1 image category-1 98 97.2 2 image category-2 97 95.3 3 image category-3 99 97.7 4 image category-4 97 97.6 5 image category-5 85 86.1 6 image category-6 89 90.1 7 image category-7 87 84.7 8 image category-8 95 97.5 9 image category-9 94 93.2 10 image category-10 96 96.4 11 image category-11 94 93.7 12 image category-12 87 86.3 13 image category-13 96 95.7 14 image category-14 93 98.5 15 image category-15 97 97.6 16 image category-16 92 98.7 17 image category-17 83 82.8 18 image category-18 85 90.7 19 image category-19 89 92.8 20 image category-20 99 98.7 21 image category-21 94 99.6 22 image category-22 97 98.3 Total 2043 2069.2
Average 92.86 94.05
123
7.2.1.2 Outcomes of Block-Truncation Coding and k-Means
Approach
The author Silakari [91] has used Block truncation coding for the purpose
of extracting the features. The blocking features of the block truncation
coding has allowed to increase the precision but however the recall rate
could not be further enhanced as compared to the prior technique of
Narasimhan [73]. Fig.7.6 shows the visual outcomes of the Block
Truncation Coding and k-Means approach and its retrieval performances.
Input-Query Image
Retrieved Image using Block Truncation Coding & K-Means approach
Figure 7.6: Visual Outcomes of BTC & K-Means approach
124
The numerical outcomes of this technique is highlighted in Table 7.3
Table 7.3: Numerical Outcomes of Block Truncation Coding Approach
BTC Based Approach
#. Category Recall (%) Precision(%) 1 image category-1 98 97.2 2 image category-2 97 95.3 3 image category-3 99 97.7 4 image category-4 97 97.6 5 image category-5 85 84.6 6 image category-6 89 87.7 7 image category-7 87 84.7 8 image category-8 95 94.8 9 image category-9 94 93.2 10 image category-10 96 96.4 11 image category-11 94 93.7 12 image category-12 87 86.3 13 image category-13 96 95.7 14 image category-14 93 98.5 15 image category-15 97 96.8 16 image category-16 92 91.6 17 image category-17 83 82.8 18 image category-18 85 84.9 19 image category-19 89 87.4 20 image category-20 99 98.7 21 image category-21 94 99.6 22 image category-22 97 96.9 Total 2043 2042.1 Average 92.86 92.82
125
7.2.1.3 Outcomes of K-Means Approach
The approach proposed by Biswas [81] discusses about using discrete
wavelet transform as well as conventional k-means algorithm for the
purpose of image retrieval process, however, k-means was the dominant
technique used in this part of the study. The visual outcomes
accomplished from the implementation of k-means clustering can be
seen in Fig.7.7
Input-Query Image
Retrieved Image using k-means clustering approach
Figure 7.7: Visual Outcomes K-Means approach
126
The numerical outcome of the k-means approach can be seen in Table
7.4
Table 7.4: Numerical Outcomes of K-Means Approach.
K-Mean Based # Category Recall (%) Precision (%) 1 image category-1 98 97.2 2 image category-2 97 95.3 3 image category-3 99 78.6 4 image category-4 97 97.6 5 image category-5 85 83.4 6 image category-6 89 87.7 7 image category-7 87 84.7 8 image category-8 95 94.8 9 image category-9 94 90.4 10 image category-10 94 96.4 11 image category-11 94 93.7 12 image category-12 80 80.7 13 image category-13 97 95.7 14 image category-14 93 98.5 15 image category-15 97 96.8 16 image category-16 87 91.6 17 image category-17 83 80.4 18 image category-18 84 84.9 19 image category-19 89 81.7 20 image category-20 99 98.7 21 image category-21 94 99.6 22 image category-22 84 96.9 Total 2016 2005.3 Average 91.63 91.15
127
Hence, it can be seen that the existing system offer score of recall factor
between 91-92% approximately, while precision factor is found between
91-94% approximately. The summary of the precision and recall for the
existing system can be seen in Fig. 7.8. The summary shows that
contribution-based system has better performance as compared to Block
Truncation Coding and conventional k-means algorithm. A closer look
into the outcome of existing system also shows that contribution-based
approach offers higher precision rate of 94%. However, the recall rate of
contribution-based approach and block truncation coding is found to be
quite similar. Recall rate of conventional k-means algorithm is also found
to be slightly lowered compared to contribution-based and block
truncation based coding approach.
Figure 7.8: Summary of the Precision and Recall for Existing System
128
In order to carry out comparative performance analysis of the proposed
study, it is essential that similar type of test bed and dataset is used on
proposed system and existing system. Hence, the specific algorithms
discussed by the researchers in existing system were implemented over
visual art images and outcomes were recorded. Table 7.5 summarizes
the outcomes of the study with respect to processing time of an
algorithms and accomplished accuracy
Table 7.5: Summary of Processing Time and Accuracy
Authors Techniques Adopted Processing
Time
Precision
Narasimhan[73] Contribution Based
Clustering
17 min 94%
Silakari [91] -Block Truncation Coding
-k-means clustering
10 min 92%
Biswas [81] -k-means clustering
-Discrete Wavelet
Transform
9 min 91%
Proposed COIR
Method
-Visual Descriptors
-Support Vector Machine
-Principal Component
Analysis
-Relevance Feedback
-Image Filtering
7 min 97%
129
Table 7.5 shows the outcomes of the proposed system as well as
existing system, where it can be seen that processing time of proposed
COIR system is only 7 min for massive dataset considered during study,
while other existing approaches fall back in this regards. The accuracy of
the proposed system is also found to be around 97%, while the accuracy
of existing system ranges between 91-94% in maximum. It was also
observed that processing time of the proposed system could be further
more minimized if either of the training technique viz. Support Vector
Machine or Principle Component Analysis were removed. However, it
was done as removal of either of the training algorithm results in slight
fall of accuracy score and hence decided to retained it.
Figure 7.9: Recall & Precision Analysis
130
Fig.7.9 highlights the comparative performance outcomes of proposed
system with the existing system considering recall and precision in the
form of percentage. The outcome shows that recall rate for Biswas [81] is
quite poor compared to Narasimhan [73] and Silakari [91] approach as
well as with the the proposed system too.
A closer look into the outcomes will interpret the reason for the patterns
of the outcomes. It was seen that Narasimhan [73] have mechanised a
clustering technque that is solely dependent on the datapoints residing
within that clusters. For better effectiveness, the author have also
segregated the concept of intra-clustering as well as inter-clustering
depending on the forms of the cluster. RGB colored histograms were
used to represent visual content of the input image. Their technique has
also perform quantization of the RGB color space and its respective
coordinates very uniformly.
The approach presented by Silakari [91] has used color moments for the
purpose of performing feature extraction. However, the authors in the
next half chooses to implement Block Truncation Coding alongside with
conventional clustering algorithm e.g. K–means. Adoption of two
dimensional discrete wavelet transformation was found in the work done
by Biswas [81] for the purpose of feature extraction. The author has also
deployed k-mean clustering algorithm for grouping up the extracted
131
features for better image retreiva system. The technique adopted was
quite simple and is supposed to give better outcomes in similary match,
but however, the results differs.
The analysis indicates that the techniques discussed by the Biswas [81]
has significantly enhanced the searching of visual art images, but the
cumulative outcomes of recall and precision factor is explored to be quite
minimal. Hence, the precision as well as recall rate for Narasimhan [73]
approach is quite high compared to Silakari [91] and Biswas [81]
approach.
The problems found in the outcomes of the existing system was
addressed in the proposed COIR system by introducing a novel
combination of the visual descriptor. Inspite of using a single visual
descriptor, the proposed COIR system chooses to use both Color Layout
Descriptor to extract color features and Edge Historgram Descriptor to
extract edge features. Adoption of visual description was also witnessed
in the study presented by Narasimhan [73], and hence, its outcome was
found better compared to Silakari [91] and Biswas [81]. The recall and
precision factor is further enhanced in the proposed system owing to the
adoption of supervised training mechanism, which is another uniqueness
of the proposed system. Interesting even after using dual features of
training (Support Vector Machine and Principle Component Analysis) and
132
dual usage of feature extraction method (Color Layout Descriptor and
Edge Histogram Descriptor), the overhead of the proposed system is
neglible and has no effect in processing speed, which is found to be quite
faster compared to all the existing system. Hence, the proposed system
doesn’t only introduce an efficient visual art reterival system but also
offers cost-effective system design that is found to be computationally
efficient.
7.3 Summary
This chapter has presented the elaborated discussion about the findings
of the proposed study. The chapter has discussed the rationale behind
selection of the existing research work that has been compared with the
proposed COIR system with respect to processing speed, recall rate, and
precision rate. The outcome of the proposed study was found with 97%
of accuracy rate, which is found to be highest as compared to the
existing study.
133
Chapter 8 Conclusion and Future Work
8.1 Conclusion
This thesis has investigated an exclusive case of Content-Based Image
Retrieval (CBIR) system. While at the time of reviewing the literatures, it
was found that CBIR technique has wide case of utility right from
commercial to medical applications. However, it was found that CBIR
technique has never been evaluated for visual art images. This thesis
has discussed about the complexities involved in processing visual art
images with respect to CBIR technique. This thesis has investigated a
practical technique for visual art image retrieval by a novel CBIR
algorithm based on feature extraction using learning algorithms. The
technique is based on defining meta-features which are calculated from
the spatial configurations of local image features in order to imitate
partitioned clustering. User relevance feedback has been integrated
within the approach to allow human judgment to influence the definition
of visual art image similarity when retrieving visual art images. Here is
the summary, the contributions and discussion on ideas for future work. A stated in Chapter 1, the problem addressed in this thesis can be posed
by the question how can a user perform feature set pruning using
Content based image retrieval system. To answer this question the
investigator has set out to investigate how to analyze a visual art image,
134
how to identify those visual art images that are most similar in
morphology, and how to organize their presentation to the best effect.
Since visual art image are composed of potentially complex elements, it
was realized that a means for representing these image sub-structures
would be important. In addition it was realized that being able to capture
human perceptual judgment in terms of what constitutes image similarity
would also be important, since human beings ultimately arbitrate in
disputes of whether a trademark is deemed novel or not.
The proposed study has emphasized on the usage of CBIR technique
considering case study of visual art map. In this thesis, we draw some
conclusions are drawn, and how this work can be useful for future work is
presented. The goal was not that of building a complete and operational
CBIR system instead to study new techniques for CBIR. As mentioned in
the thesis, it is found that there was room for improvement in three key
aspects of CBIR: query form, representation of database of visual art
image and queries and comparison between queries and database visual
art images. In particular, it was felt that there were some kinds of users
that were being neglected, as none of the systems existing at the time
could satisfy their needs. The attention was especially focused on users
wanting to retrieve a known image from a heterogeneous database in
which visual art image information had not been extracted manually.
Users may find themselves in such a situation and this can be from
135
various fields: publishing, advertisement, design and retail. The thesis
has discussed about the technique that was adopted for querying the
untrained visual art images to be checked for similarity with the trained
images. More precisely, attention was focused on incomplete queries
(i.e., pictorial examples that do not cover all the available canvas).
Furthermore, it was also noticed that most of the CBIR systems were
extracting information from database images as a whole (global
features). By extracting features from limited areas of the images (local
features) there was an improvement in the results obtained with global
features, as discussed in this thesis. While investigating local features,
different ways of extracting them was evaluated. Owing to the capabilities
and limitations of human potentials, the proposed system is intended for
involuntary retrieval of visual art images along with extraction of features.
In particular, the selection of the areas from which the features are
extracted via a segmentation method was studied. The experiences
showed that segmentation can produce relevant regions for feature
extraction, as reported in Chapter6.
Once regions are extracted from images, they must be represented by a
feature set. The choice was to represent regions as a combination of
size, position, shape and color. Also, the consequences that sketches
with different levels of detail would have on the retrieval results were
136
studied, resulting in an image representation composed of several
segmentation results. On the other hand, the different nature of the user-
produced queries led towards a descriptor containing a single
segmentation result. CBIR ultimately rests on the comparison of the
query with (a subset of) the database images. The representation of
query and database image to develop a complex and customizable
method for comparing a query and a database visual art image was
taken as an advantage. The solution is composed of a dissimilarity
measure between regions, which is then used to compute the
dissimilarity between segmentation results (by associating pairs of
regions that have a low dissimilarity score), which in turn is used to
compute the dissimilarity between the query and the database image.
According to this dissimilarity, it is then possible to rank all the database
images. The user can control (to a certain extent) how similarity is
computed.
When this work was initiated few techniques were available for partial
shape matching that supported multi-component retrieval. Furthermore,
many of the techniques for image database retrieval that did exist
required the exact image segmentation, which is difficult to achieve with
the level of reliability required for a real trademark retrieval application. In
addition, few researchers had attempted to apply principles derived from
human perception for shape retrieval that would allow perceptually
137
meaningful configurations of trademark image components to be
represented and classified. Finally, since global image features are not
suitable for retrieving occluded or connected components in an image, it
was realized that interest points, i.e. local features, as adopted by many
vision researchers for general image database retrieval would be
required to be analyzed and checked for comparing trademark images
reliably. Therefore, the main aim of this research has been to develop a
method for solving the partial matching and shape perception problem by
investigating the following discussions:
1. To identify how the interest points be used to distinguish visual art
image
2. To suggest which interest point techniques are most accurate
when applied to visual art image
3. To support on how can perceptual grouping methods serve to
group interest points and represent these within a shape descriptor
(i.e. a meta-feature vector)
4. To suggest on techniques related to exploit the shape descriptors
(meta-feature vectors) while retrieving abstract visual art image
8.2 CBIR Of Visual Art Images Using Contribution
This thesis described the COIR design and implementation of a
complete, fully functional, image search, and retrieval system with
relevance feedback and learning capabilities using SVM/PCA. The COIR
138
framework allows searching for images in large databases in a simple,
efficient, friendly, and effective way. It supports a variety of browsing and
searching modes, including a query-by-example mode, in which users
can configure the system to use a particular combination of feature(s)
and distance measurements. The extracted features were successfully
used for:
1. similarity search with selected image by one or more of the extracted
features;
2. search of images that satisfied user queries featuring contrasts’
characteristics;
3. Investigation on the possibilities to integrate such characteristics
within specialized resource discovery (searching for distinctive
feature of movements, artists or artists’ periods).
This work led to analyze different aspects of CBIR. In this section some
of the observations on the subject will be listed.
1. The first important observation is that effective CBIR by user-drawn
pictorial example in a heterogeneous database using only
automatically extracted local features is possible.
2. The present method is found to be effective even if the queries are
incomplete. Incomplete queries free the user from the burden of
having to produce an image representing the whole target image.
This way, it is possible to retrieve an image by painting only a part
of its content.
139
3. Associating several consecutive segmentation results to the
database images does indeed allow efficient retrieval with queries
containing various levels of detail: good retrieval results are
obtained with sketches ranging from extremely raw to rather
detailed.
4. Associating importance weights to sketch regions mirrors the fact
that not all areas of the query are equally important to the user.
Assigning weights proportionally to the regions sizes turns out to
be a rather accurate estimation of region importance.
This work shows that it is possible to develop efficient CBIR methods
with following characteristics:
1. the query should posses an incomplete user-produced pictorial
example of a target visual art image;
2. the retrieval is based on SVM based pruning process for extracting
features;
3. features are extracted from regions which are automatically formed
according to their visual homogeneity;
4. the same representation of image should be adopted for training
database as well as passing the query image.
5. the database images representation is flexible enough to be
compared to visual art images of different levels of detail;
140
6. the comparison of query and database images allows partial
matches.
7. The principal contributions are split among three areas: feature
extraction, image and query representation and comparison.
Feature Extraction: A technique for the automatic selection of visually
homogeneous areas for later extraction of image features which is based
on region segmentation followed by region merging is presented.
Image and Query Representation: A common representation for
database visual art images and sketches, which contains a set of
segmentation results, each of which is composed of a set of regions is
presented. In order to ensure the maximum flexibility, database images
can be represented by a variable number of segmentation results. Using
regions as the basic element of the representation allows the use of
incomplete sketches.
Comparison: A dissimilarity measure between database images and
query images is presented. This measure is based on a region
dissimilarity, which is then used to compute the dissimilarity between
segmentation results by associating pairs of regions that have low
dissimilarity. The dissimilarities between the outcome were used for
evaluating the dissimilarity between query and database image. The way
the dissimilarity is computed can be influenced by the user by setting a
141
set of weights, thus making it customizable to different drawing
techniques and needs. Results show that the measure handles equally
well complete and incomplete sketches.
Main contributions can be summarized as:
1. A new methodological approach to integration of colors'
characteristics for CBIR within digital art repositories.
2. Development of a color model appropriate for contrast characteristics
extraction as a combination of three other models; the model is
innovative because it is easy for human comprehension while also
allows for efficient conversion from RGB.
3. A formal description of harmonies and contrasts from the point of view
of three main characteristics of the color – hue, saturation and
luminance.
4. An architecture and implementation of experimental CBIR-system.
The implementation includes a mixture of newly developed and
existing open source components.
8.3 Possible Enhancement
The experiments were conducted using a database of about 1000 visual
art images. Obviously, in a real-life situation the amount of images
contained in the database would be of massive increase in numbers of
visual art images. It would be interesting, then, to verify how our
technique would perform in such a case. It would be extremely
142
interesting to verify if letting the user directly specify the importance
weights of the regions in the query image would lead to better retrieval
results. Similarly, the user may also be asked to give relative importance
weights for size, position, color and shape information for each region,
and these would then influence the way the difference between the
trained image and query images is computed. This kind of testing,
though, is not practical in the current state of our work, as no graphical
user interface (GUI) is available to the user.
1. Another interesting kind of testing consists in, instead of trying to
retrieve a target image, trying to retrieve all images containing a
particular shape, or a region of a given color and size, and so on.
This can be done by setting the weights for size, position, color or
shape information in the region dissimilarity measure to zero or low
values, so that the corresponding features are ignored.
2. The evaluation of results was uniquely based on the position of the
target image, because no information about the relevance of the
images in the database compared to the target images was
available. It would be interesting to generate such information and
verify how relevant the images retrieved at the top of the ranking are.
In order to do this, though, it should be kept in mind that relevance
must be evaluated only visually, and not semantically, since our
dissimilarity measure is only sensitive to visual similarity.
143
3. Testing reported in this work was performed on queries produced by
colleagues, friends and relatives of the author. Although these
people surely gave their best to follow the guidelines and produce
better queries, it would be extremely interesting to verify how the
method would perform if it was used in a real-life situation by people
with a real need to retrieve a particular image.
4. The sketches used in the experiments were produced using a mouse
as a drawing device, which is not a very natural way of drawing.
Tests should be performed using other drawing devices, such as a
graphic tablet.
5. Probably the area of this work where improvement would be most
effective is the region merging algorithm. New approaches should be
tested. Furthermore, it is believed that adaptive thresholds should be
used to decide when to stop the merging process, instead of the
fixed thresholds that are currently used.
6. Another weak point of this work is the region shape descriptor.
Representing region shapes as ellipses and comparing them by
computing the overlapping surface is not a very good approach.
Alternative techniques may prove difficult to find, though, because
they will need to be applicable to non-connex regions.
144
8.4 Future Work
The near plans for further research are focused on:
1. Analyzing the possibilities of using SIFT-descriptors as a ground for
defining upper-layer concepts
2. Focusing on the processes of throwing out redundant attributes in
order to achieve more clear and faster results
3. Applying already extracted as well as new developed attributes and
corresponding methods in the field of analysis Eastern
Iconographical painting schools (especially Bulgarian tradition) and
themes within the icons.
145
References
1] Agarwal.M., Mostafa.J,” Content-based Image Retrieval for
Alzheimer‟s Disease Detection” Content-Based Multimedia Indexing
(CBMI), 9th International Workshop, 2011
[2] Ahmed J. Afifi, Wesam M. Ashour “Content-Based Image
RetrievaUsing Invariant Color and Texture Features”, IEEE , Vol. 2,
No.2, ISSN: 978-1-4673-2181, 2012.
[3] Ali,M., Adnan.A., Saqib.M and Zahidullah,” Content Based Image
Retrieval CBIR) Using Materialized Views”, International Conference
on Computer [71] (Science and Information Technology
(ICCSIT'2011) Pattaya Dec. 2011
[4] Antani, S., Long, L.R., Thomas, G.R., Content-Based Image
Retrieval for Large Visual art Image Archives, MEDINFO 2004
[5] Asmita A. Desai, ―Review Paper on Sketch Based Image retrieval
System, Proceedings of Sixth IRAJ International Conference, ISBN:
978-93-82702-32-0, 6th October 2013, Pune, India.
[6] Asha Vellaikal and C.-C. Jay Kuo. Hierarchical clustering techniques
for image database organization and summarization. In C.-C. Jay
Kuo, Shih-Fu Chang, and Sethuraman Panchanathan, editors,
Multimedia Storage and Archiving Systems III (VV02) , Vol.3527 of
SPIE Proceedings, pp.68–79, Boston, Massachusetts, USA,
November 1998
146
[7] K. Ashok Kumar & Y.V.Bhaskar Reddy, “Content Based Image
Retrieval Using SVM Algorithm”, international Journal of Electrical
and Electronics Engineering (IJEEE) ISSN (PRINT): 2231 – 5284,
Vol.1, Issue 3, 2012
[8] Anil K. Jain and Aditya Vailaya. Image retrieval using color and
shape. Pattern Recognition , 29(8):1233–1244, August 1996
[9] Apostolos Marakakis, Nikolaos Galatsanos2, Aristidis Likas3
“Relevance Feedback for Content-Based Image Retrieval Using
Support Vector Machines and Feature Selection”, Springer-Verlag
Berlin Heidelberg ,ICANN s, Part I, LNCS 5768, pp. 942–951, 2009
[10] K. Arthi and Mr. J. Vijayaraghavan, ―Content Based Image
Retrieval Algorithm Using Color ModelsI, ―International Journal of
Advanced Research in Computer and Communication EngineeringI,
Vol.2, Issue 3, March 2013
[11] Ala‘a Al-zoubi, Mohammad Al-zoubi, ―Automatic Mapping of
MPEG-7 Description to relational DatabaseI, The International Arab
Journal of Information Technology, Vol. 9, No .6, November 2012
[12] B. Sz´ant´o, P. Pozsegovics, Z. V´amossy, Sz. Sergy´an,
ISketch4Match – Content-based Image Retrieval System Using
SketchesI, SAMI 2011• 9th IEEE International Symposium on
Applied Machine Intelligence and Informatics •pp.27-29, January
2011 • Smolenice, Slovakia.
147
[13] Balan, S., Devi, T. 2012., “Design and Development of an Algorithm
for Image Clustering In Textile Image Retrieval Using Color
Descriptors, International Journal of Computer Science, Engineering
and Applications (IJCSEA), Vol. 2(3), 2012
[14] Chadha, A., Mallik, S., Johar, R., Comparative Study and
Optimization of Feature-Extraction Techniques for Content based
Image Retrieval, International Journal of Computer Applications
(0975 – 8887) Vol.52, No.20, August 2012
[15] Cheng, S.C., Wu, T.L., Fast indexing method for image retrieval
using k-nearest neighbors searches by principal axis analysis,
Journal of Visual Communication in Image Retrieval, Elsevier, 2006
[16] Conci, A., Mathias, E, Castro, M. M., “Image mining by content”,
Expert Systems with Applications, Vol. 23, No. 4, pp. 377-383, 2002
[17] Chowdhury.M., Das.S and Kundu.M.K,” Interactive Content Based
Image Retrieval Using Ripplet Transform and Fuzzy Relevance
Feedback” PerMIn 2012, LNCS 7143, pp. 243–251, 2012, Springer-
Verlag Berlin Heidelberg 2012
[18] Cunningham, S.J., Bainbridge, D and Masoodian, M, “How people
describe their image information needs: a grounded theory analysis
of visual arts querie” Digital Libraries, 2004. Proceedings of the 2004
Joint ACM/IEEE Conference on.
[19] Constantinescu, L., Jinman, K., Dangan, F., Integration of Interactive
Visual art Image Data Visualisation to Internet-Based Personal
148
Health Records for Handheld Devices, e-Health Networking,
Applications and Services, Healthcom ,11th International
Conference, pp. 79-83, 2009
[20] Calvo, R.A., Crespo, J., Marmol, M.M., Intriago, M., Castellanos, A.I.,
A Cloud Computing Service for Managing Visual art Image
Collections, IEEE, 2012
[21] Chang, S.F., Sikora, T., Puri, A., Overview of the MPEG-7 Standard,
IEEE Transactions on Circuits and Systems for Video Technology,
Special Issue on MPEG-7, June 2001
[22] David A. White and Ramesh Jain. Algorithms and strategies for
similarity retrieval. Technical Report VCL-96-101, Visual Computing
Laboratory, University of California, San Diego, 9500 Gilman Drive,
Mail Code 0407, La Jolla, CA 92093-0407, July 1996.
[23] David McG. Squire, Wolfgang Muller, and Henning Muller.
Relevance feedback and term weighting schemes for content based
image retrieval. In Third International Conference On Visual
Information Systems, Amsterdam, The Netherlands, June 2–4 1999.
(to appear).
[24] Deniz, S., Herman, G.T., Kong, T.Y., A Technology for Retrieval of
Volume Images from Visual art Databases, IEEE, 2004
[25] Dr.N.D.Harini, Dr.D.Lalitha Bhaskari “Image Mining Issues and
Methods Related to Image Retrieval System”, International Journal
149
of Advanced Research in Computer Science, Vol.2 (4), pp. 224-
229, July-August 2011
[26] T. Dharani, I. Laurence Aroquiaraj IEEE Member “Content Based
Image Retrieval System using Feature Classification with Modified
KNN Algorithm”, arXiv preprint:130.4717, 2013
[27] T. Dharani , I. Laurence Aroquiaraj “A Survey on Content Based
Image Retrieval”, IEEE, -978-1-4673-5845-3/13, 2013
[28] D.Jeyabharathi, Dr.A. Suruliandi “Performance Analysis of Feature
Extraction and Classification Techniques in CBIR”, ICCPCT, 978-1-
4673-4922-2/13/, 2013
[29] D.T.V. Dharmaji Rao, BH. Anantha Laxmi, ―A Content Based
Retrieval System Using Graphs , International Journal of Advanced
and Innovative Research (IJAIR) - ISSN: 2278-7844
[30] Emrah Bostan, Ulugbek Kamilov, and Michael Unser,”
Reconstruction of visual art images and sparse stochastic
modeling”, pp.1-12, IEEE, 2012
[31] I. Felci Rajam, S. Valli:” A Survey on Content Based Image
Retrieval”[vol.10(2): 2475-2487]. (ISSN: 1097-8135).
http://www.lifesciencesite.com 343, Life Sci J , 2013
[32] V.K. Garg, “Pragmatic data mining: Novel paradigms for tackling key
challenges,” Project Report, Computer Science & Automation (CSA),
Indian Institute of Science, 2009
150
[33] Georgaki,A and Li.H, “Content Based Image Retrieval Using a
Bootstrapped SOM Network”, ISNN'06 Proceedings of the Third
international conference on Advances in Neural Networks – Vol. Part
II, pp.595-601, 2006
[34] Gonde,A.B., Maheshwari,R.P and Balasubramanian.R, “Complex
Wavelet Transform with Vocabulary Tree for Content Based Image
Retrieval” ACM 978-1-4503- 0060, 2010
[35] A., Gupta, S., Santini, and R., Jain, In search of information in visual
media, Communications of the ACM, Vol.40, issue-12, pp.34-42,
1997
[36] Hiremath.P.S and Pujari.J,” Content Based Image Retrieval based
on Color, Texture and Shape features using Image and its
complement” International Journal of Computer Science and
Security, Vol.(1) : Issue (4), 2007
[37] Herráez-Arevalillo.M., Domingo. J and Ferri. F.J, “Combining
similarity measures in content-based image retrieval”ACM Digital
Library, Journal pattern recognition letters archive volume 29 issue
16, pp. 2174-2181, December 2008
[38] Huu, Q. N., Thu, H. N. T., Quoc, T. N., “An efficient content based
image retrieval method for retrieving images”, International Journal
Of Innovative Computing Information And Control, vol. 8(4), pp.
2823-2836, 2012
151
[39] Herwindiati, D.E., Isa, S.M., The Robust Distance for Similarity
Measure Of Content Based Image Retrieval, Proceedings of the
World Congress on Engineering Vol. II, WCE 2009
[40] Hossein Nezamabadi-pour, Ehsanollah Kabir , “ Concept learning by
fuzzy k-NN classification and relevance feedback for efficient image
retrieval”, Elsevier- 5948–5954, 2009
[41] Huda Abdulaali Abdul Baqi, Ghazali Sulong, Soukaena Hassan
Hashem, ―A Sketch Based Image Retrieval: A review of LiteratureI,
Journal of Theoretical and Applied Information Technology, Vol.63,
No.1, ISSN: 1992-8645, 10th may 2014
[42] Jaswal.G.,Kaul.A and Parmar.R,” Content based Image Retrieval
using Color Space Approaches”, International Journal of
Engineering and Advanced Technology (IJEAT), ISSN: 2249 – 8958,
Vol.2, Issue 1, October 2012
[43] Jadhav, S. H., Ahmed, S. A., “A Content Based Image Retrieval
System using homogeneity Feature extraction from Recency-based
Retrieved Image Library”, IOSR Journal of Computer Engineering
(IOSRJCE), Vol. 7, Issue. 6, pp. 13-24, 2012
[44] Jyun, H.W., Hwung, S and Vincent, S, “Efficient relevance feedback
for content based image retrieval by mining user navigation pattern”
IEEE Transaction on Knowledge and Data Engineering, 2011
[45] Junaid Khan, ―Content Based Image Retrieval by ClusteringI,
International journal of Advance Research in Computer Science and
152
Management Studies (IJARCSMS) – Vol.2, Issue1, pp.708-714,
January 2014
[46] T. Jyothirmayi, Suresh Reddy, “An Algorithm for Better Decision
Tree”, (IJCSE) International Journal on Computer Science and
Engineering, Vol. 02, No. 09, pp. 2827-2830, 2010
[47] T., Kato, “Database architecture for content-based image retrieval” in
Image Storage and Retrieval Systems (Jambardino, A A and
Niblack, W R , eds), Proc SPIE 1662, pp.112-123, 1992
[48] Kekre, H.B., Mishra, D., Kariwala, A., A survey of CBIR techniques
and semantics, International Journal of Engineering Science and
Technology (IJEST), 2011
[49] Kekre H.B, Mishra D., Content Based Image Retrieval using Density
Distribution and Mean of Binary Patterns of Walsh Transformed
Color Images, Vol .3, No 2, 2011
[50] Kekre, H.B, Thepade, S.D, Banura, V.K., Augmentation of Colour
Averaging Based Image Retrieval Techniques using Even part of
Images and Amalgamation of feature Abstract vectors, published in
International Journal of Engineering Science and Technology, Vol
.2.(10), pp. 5238-5246, 2010
[51] Kekre H.B., Image Retrieval with Shape Features Extracted using
Gradient Operators and Slope Magnitude Technique with BTC,
Vol.6-8, Published in IJCA (0975 – 8887), 2010
153
[52] Kekre H.B, Mishra D, Density Distribution and Sector Mean with
Zero-cal and Highest-sal Components in Walsh Transform Sectors
as Feature Vectors for Image Retrieval, Vol.8, No.4, published in
International Journal of Computer Science and Information Security,
2010
[53] Kekre H.B, Mishra D, Performance Comparison of Four, Eight &
Twelve Walsh Transform Sectors Feature Vectors for Image
Retrieval from Image Databases, Vol.2(5), Published in International
Journal of Engineering Science and Technology , 2010
[54] Kekre H.B, Mishra D, CBIR using Upper Six FFT Sectors of Color
Images for Feature Vector Generation, Vol.2(2), Published in
International Journal of Engineering and Technology , 2010
[55] Kekre H.B, Mishra D, Performance Comparison of Density
Distribution and Sector mean of sal and cal functions in Walsh
Transform Sectors as Feature Vectors for Image Retrieval, Vol.4,
Issue 3, Published in International Journal of Image Processing,
2010
[56] Khaparde.A., .Deekshatulu.B.L., Madhavilatha.M., Farheen.Z and
Kumari. S.V, “Content Based Image Retrieval Using Independent
Component Analysis” IJCSNS International Journal of Computer
Science and Network Security, VOL.8, April 2008
[57] Kimberly Powell,” Visual art imaging ecosystem and the role of the
GPU”, pp.1291-1292, IEEE, 2009
154
[58] Komali, A., Babu, R. V., “An Efficient Content Based Image Retrieval
System for Color and Shape Using Optimized K-Means Algorithm”
International Journal of Application or Innovation in Engineering &
Management (IJAIEM), Vol.2, Issue.8, 2013
[59] V. Karpagam, and R. Rangarajan, “Improved content-based
classification and retrieval of images using support vector machine”,
CURRENT SCIENCE, VOL.105, NO. 9, 10 November 2013
[60] K. C. Sia and Irwin King. “Relevance feedback based on parameter
estimation of target distribution” In IEEE International Joint
Conference on Neural Networks, pp. 1974–1979, 2002
[61] Liguang, MA. Yanrong, C., Jianbang, H., Visual art Image Storage,
Retrieval and Visualization Based-on Open Source Project”,
Congress on Image and Signal Processing, 2008
[62] Liapis, S., Color and texture image retrieval using chromaticity
histograms and wavelet frames, IEEE Transaction on Multimedia,
2004
[63] Liu, D., Hua, A., Vu, K., and Yu, N, “Fast Query Point Movement
Technique in large content based image retrieval system” , IEEE
Transaction on knowledge and data engineering , 2010
[64] Lux, A., Chatzichristofis, S.A., LIRe: Lucene Image Retrieval - An
Extensible Java CBIR Library”, MM '08 Proceedings of the 16th
ACM international conference on Multimedia, 2008
155
[65] Muller, H., Kalpathy, J., Eggel, I., Overview of the CLEF 2010
medical image retrieval track, Citeseer, 2010
[66] Murthy, V. S. V. S., et al. "Content based image retrieval using
Hierarchical and K-means clustering techniques." International
Journal of Engineering Science and Technology vol.2.3, pp.209-212,
2010
[67] Murthy, V.S., Vamsidhar, J.N.V.R., Swarup Kumar, P. Sankara Rao,
“Content Based Image Retrieval using Hierarchical and K-Means
Clustering Techniques”, International Journal of Engineering
Science and Technology Vol.2(3), pp.209-212, 2010
[68] Murthy, V. S. V. S., Vamsidhar, E., Rao, P. S., Raju, G. S. V.
“Application of hierarchical and K-means techniques in Content
based image retrieval”, International Journal of Engineering Science
and Technology, vol.2(5), pp.749-755, 2010
[69] Malakar, A., Mukherjee, J., “Image Clustering using Color Moments,
Histogram, Edge and K-means Clustering”, International Journal of
Science and Research (IJSR), India Online ISSN: 2319-7064, 2013
[70] Mascio. T.D., Frigioni.D and Tarantino.L,” VISTO: a New CBIR
System for Vector Images”, Information Systems, Vol.35, Issue 7,
pp.709–734, November 2010
[71] Mit Patel, Keyur Bhrahmbhatt, Kanu Patel “Feature based Image
retrieval based on clustering, classification techniques using low
156
level image features”, IJAERD, Vol.1, Issue 5, e-ISSN: 2348 –
4470, May 2014
[72] Malik, F. and Baharudin, B. B., Feature analysis of quantized
histogram color features for content-based image retrieval based on
Laplacian filter. In International Conference on System Engineering
and Modeling. IACSIT Press, Singapore, vol. 34, 2012
[73] Narasimhan, H., Ramraj, P., Contribution-Based Clustering
Algorithm for Content-Based Image Retrieval, 5th International
Conference on Industrial and Information Systems, ICIIS 2010, Jul
29 - Aug 01, 2010
[74] Puyan, M., Vetri, L.J, Microwave Visual art Imaging Using the
Multiplicative Regularized Gauss–Newton Inversion, Antennas and
Wireless Propagation Letters, IEEE, Vol.8, pp.645-648, 2009
[75] Pooja Kamavisdar, Sonam Saluja, Sonu Agrawal, “A Survey on
Image Classification Approaches and Techniques”, International
Journal of Advanced Research in Computer and Communication
Engineering, Vol.2, Issue 1, January 2013.
[76] Prajakta Mahendra Ovhah, Prof. B.D. Phulpagar, ―A Survey On
Various Feature Extraction Techniques, Followed By Image
Retrieval System , International Journal of Emerging Trends and
Technology in Computer Science (IJETTCS) , Vol. 3, Issue 5,– ISSN
2278-6856, Sept-Oct 2014
157
[77] Park, S.B, Lee, J.W., Kim, S.K., Content-based image classification
using a neural network, Elsevier, 2004
[78] Pushpa B. PATIL, Manesh B. KOKARE” Relevance Feedback in
Content Based Image Retrieval: A Review”, Journal of Applied
Computer Science & Mathematics , Vol.1, No. 10 (5) /2011, 2011
[79] Priyanka B. Khairnar, Aarti A. Khairnar, Priyanka R. Khandre,
Priyanka S. Mahajan, ―Content Based Image Retrieval SystemI,
1st International Conference on Recent Trends in Engineering &
Technology, Special Issue of International Journal of electronics,
Communication & Soft Computing Science & Engineering, ISSN:
2277-9477, Mar-2012
[80] Patheja P.S., Waoo Akhilesh A. and Maurya Jay Prakash, “An
Enhanced Approach for Content Based Image Retrieval”,
International Science Congress Association, Research Journal of
Recent Sciences, ISSN 2277 – 2502 Vol. 1(ISC-2011), pp.415-418,
2012.
[81] Prasad, B. G., K. K. Biswas, and S. K. Gupta. "Region-based image
retrieval using integrated color, shape, and location index."
Computer vision and image understanding No.94.1, pp.193-233,
2004
[82] Rajendran, P., and Madheswaran, M., “An Improved Image Mining
Technique For Brain Tumour Classification Using Efficient classifier”,
158
International Journal of Computer Science and Information Security,
vol.6, No. 3, pp.107-116, 2009
[83] Rajshree S. Dubey, “Image Mining using Content Based Image
Retrieval System”, (IJCSE) International Journal on Computer
Science and Engineering Vol. 02, No. 07, pp. 2353-2356., 2010
[84] Ravishankar.K.C., Prasad.B.G., Gupta. S.K and Biswas. K.K,”
Dominant Color Region Based Indexing for CBIR” Image Analysis
and Processing, International Conference on IEEE, pp.887-892,
1999 proceedings
[85] Ren.F. H “Multi-image query content-based image retrieval” School
of Information Technology and Computer Science, October 2006
[86] Rao,C.S., Kumar.S.S and Mohan.B.C” Content based image
retrieval using exact legendre moments and support vector
machine”, IJMA, Vol.2, No.2, May 2010
[87] Reddy,P.V.N and Prasad K.S, “Local Derivative Patterns and their
Magnitudes for Content Based Image Retrieval” International
Journal of Computer Science & Informatics, Vol.1, Issue 2, 2011
[88] Remenyi, A. and Bandi, I. and Valcz, G. and Bodgdanov, P. and
Kozlovszky, Miklós, Visual art image processing with GPGPU using
CUDA. In: MIPRO 2011. 34th international convention. Proceedings.
Vol.1. MEET and GVS. Opatija, 2011
[89] Ravindran, U., Shakila, T, “Content Based Image Retrieval For
Histology Image Collection Using Visual Pattern Mining”,
159
International Journal of Scientific & Engineering Research, Vol.4,
Issue 4, 2013
[90] Raghatate, K. S., Janwe, J., “Content Based Image Retrieval With
Relevance Feedback Using Clustering”, International Journal of
Recent Advances in Engineering & Technology (IJRAET) ISSN,
Vol.1, Issue 2, 2013 [91] Silakari, S., Motwani, S and Maheshwari,
M, “Color Image Clustering using Block Truncation Algorithm”, IJCSI
International Journal of Computer Science Issues, Vol.4, No.2., 2009
[92] R., Swiniarski, A., Swiniarska, Comparison of feature extraction and
selection methods in mammogram recognition, Annals of the New
York Academy of Sciences, Vol.980, pp.116-24, 2002
[93] Setumin S, Sheikh U.U, Abu Bakar S.A.R. Car Plate Character
Extraction and Recognition Using Stroke Analysis, Published In
IEEE, 2010
[94] Soman.S., Ghorpade.M., Sonone.V and Chavan.S, “ Content Based
Image Retrieval Using Advanced Color and Texture Features”
International Conference in Computational Intelligence (ICCIA)
Proceedings published in International Journal of Computer
Applications® (IJCA), 2011
[95] Sudhir.R,”An Efficient CBIR Technique with YUV Color Space and
Texture Features”, Computer Engineering and Intelligent Systems,
ISSN 2222-1719 (Paper) ISSN 2222- 2863 (Online), Vol. 2, No.6,
2011
160
[96] Sharma.N.S, Rawat.P and Singh.J.S.,” Efficient CBIR Using Color
Histogram Processing” Signal & Image Processing : An International
Journal(SIPIJ) Vol.2, No.1, March 2011
[97] Sharma.P., Dubey.D,” Modified Color Averaging Technique for
Content based Image Retrieval”, International Journal of Computer
Applications (0975 –8887), Vol. 51, No.20, August 2012
[98] Sravanthi.A., Reddy.B.H,” Sketch4Match – Contentbased Image
Retrieval System Using Sketches” International Journal of
Engineering Research & Technology (IJERT), ISSN: 2278-0181
Vol.1 Issue 7, September 2012
[99] Stan Sclaroff, Leonid Taycher, and Marco La Cascia. ImageRover: a
content-based browser for the world wide web. In IEEE Workshop
on Content-Based Access of Image and Video Libraries, pp. 2–9,
San Juan, Puerto Rico, June 1997
[100] Samathal, S., Mohanraj, N., “BTC with K means classifier using
color image clustering”. Journal of Computer Application, 5, 2010
[101] Si Yong Yeo, “Implicit Active Contours for N-Dimensional Visual art
Image Segmentation”, 2012 IEEE International Conference on
Systems, Man, and Cybernetics, pp.14-17, COEX, Seoul,
Korea,2012
[102] S. Singaravelan, D. Murugan, K. Rajalakshmi, G. Santhiya “
Refining CBIR Using Rule Based KNN”, Journal of Remote Sensing
Technology, Vol.1 Issue.1, pp.13-19, May 2013
161
[103] Tsai.Y.H,” Salient Points Reduction for Content-Based Image
Retrieval” World Academy of Science, Engineering and Technology,
25, 2009
[104] Sultan Aljahdali , Aasif Ansari , Nisar Hundewale “Classification of
Image Database using SVM with Gabor Magnitude”, IEEE, 978-1-
4673- 1520-3/12/ ,2012
[105] Surekha Karpe, Sneha Shree, Saurabh Kurkute, Abuzer Shah, Mrs.
Archana Chaugule, ― Sketch4Match: Content Based Image
Recognition using Colored SketchesI, International Journal of
Engineering Trends and Technology (IJETT)- Vol. 9, No.10, March
2014
[106] Swati V. Sakhare and Vrushali G. Nasre, ―Design of Feature
Extraction in Content Based Image Retrieval(CBIR) using Color and
TextureI, I International Journal of Computer Science & InformaticsI,
Vol.1, Issue 2, 2011.
[107] Sikora Thomas, “The MPEG-7 Visual Standard for Content
Description—An OverviewI, IEEE Transaction On Circuits and
System for Video Technology, Vol.11, No.6 pp. 696-702, June 2001
[108] Thierry Pun and David McG. Squire. Statistical structuring of
pictorial databases for content-based image retrieval systems.
Pattern Recognition Letters, 17:1299–1310, 1996.
162
[109] Tagare, H.D., Jaff e, C.C., and Duncan, J. Medical image
databases: A content-based retrieval approach. American Medical
Informatics Association, 4:184–198, 1997
[110] Thomas M. Lehmann, Berthold Wein, Jörg Dahmen, Jörg Bredno,
Frank Vogelsang, and Michael Kohnen,” Content- Based Image
Retrieval in Medical Applications: A Novel Multi- Step Approach”, In
Proceedings of SPIE: Storage and Retrieval for Media Databases,
Vol.3972 (2000), pp.312-320 Key: citeulike:2783702, 2000
[111] Tonge, Vanita G. "Content based image retrieval by K-Means
clustering algorithm." Int J Eng Sci Tech, Vol. 2, pp. 209-212, 2011
[112] Thomas M. Lehmann, Mark O. Gu¨lda, Thomas Deselaersb, Daniel
Keysersb, Henning Schubertc, Klaus Spitzera, Hermann Neyb,
Berthold B. Wein, Automatic categorization of medical images for
content-based retrieval and data mining, Computerized Medical
Imaging and Graphics 29 pp.143– 155, 2005
[113] Udupa.J.K, Raya.S.P, Barreit.W.A, A PC-based 3D imaging system
for visual art data”, Visualization in Visual art Computing,
Proceedings of the First Conference, pp.294-303, 2009
[114] R. Xu and D. Wunsch, “Survey of clustering algorithms,” IEEE
Transactions on Neural Networks, Vol.16, Issue 3, pp.645– 678,
May 2005
[115] Yong Rui, Thomas S. Huang, Michael Ortega, Sharad Mehrotra,”
Relevance Feedback A Power Tool for Interactive Content Based
163
Image Retrieval”, IEEE, Transactions on circuits and video
technology
[116] Zhang Yi, Pu Yuanyuan*, Xu Dan, Scientific understanding of
visual art, Proceedings of the 31st Chinese Control Conference
pp.25-27, July 2012, Hefei, China
164
List of Publications
[1] “Survey on Content Based Image Retrieval System and Gap
Analysis for Visual Art Image Retrieval System “ International
journal of Computer Science Issues,(IJCSI) Vol.10, Issues 3, No.1 ,
, ISSN(Print) 1694-0814 ! ISSN(online) 1694-0784. Pp 23-31, May
2013.
[2] “Efficient Modeling of Visual Art Color Image Clustering “ international
journal of computer Application (IJCA) (0975-8887), Vol .91 N0.11,
pp 27-32, April 2014.
[3]” Scalable Learning Based FrameWork for Pruning CBIR System
Using Visual Art Image” international journal of Engineering
Research & Technology (IJERT) , ISSN: 2278-0181, Vol.2, Issue
8,pp 2372-2377, August 2013.
