DE Conferentie 2005 Egon v.d. Broek

Multilevel Access to our Cultural Heritage (MACH)


People who participated in the project

• Egon L. van den Broek (AI, VU Amsterdam /

NICI, RU Nijmegen)• Thijs Kok (ICIS, RU Nijmegen)

• Eduard Hoenkamp (NICI, RU Nijmegen)• Theo E. Schouten (ICIS, RU Nijmegen)• Peter J. Petiet (AI, VU Amsterdam)• Louis G. Vuurpijl (NICI, RU Nijmegen)


In addition…The people who worked on the foundation of M4ART

• Frans Gremmen (RTOG, RU Nijmegen)• Maarten A. Hendriks (freelance)• Peter Kisters (freelance)• Harco C. Kuppens (ICIS, RU Nijmegen)• Eva M. van Rikxoort (ISI, UMC Utrecht)• Charles M. de Weert (Soc. Fac., RU Nijmegen)


NWO research line ToKeN

• Two projects within the ToKeN research line:– Eidetic– VindIT

• A few of the aims of the projects– Access and enhancement of our cultural heritage – Combining Computer Science and Cognitive Science– Combining (text-based) Information Retrieval (IR) and Content-Based

Image Retrieval (CBIR)– Launching a CBIR system for the National Gallery of Arts (Het

Rijksmuseum)


Key notions of this talk

• Aim: Access and enhancement of knowledge• Domain: Cultural heritage; more specific: The National

Gallery of Arts (Het Rijksmuseum)• Characteristics:

– Digital photographs of objects– Annotations of these objects; i.e., descriptions of the artist, the

material, the content/theme of the object etc. • For … both laypersons (i.e., non-experts) and experts• How: (text-based) Information Retrieval (IR) and Content-

Based Image Retrieval (CBIR)• Additionally, facilitating:

– The user’s understanding of CBIR– Comparison of IR and CBIR


Current state-of-the-art in museum access through the Internet

Four examples:

Virtual Catalogue for Art History

The National Gallery of Art (Het Rijksmuseum)

The hermitage museum (powered by IBM)


Why this aim in this domain?

• Attract people to museums through Internet• Make museums better accessible for laypersons• Make museums better accessible for experts• Providing more information through new methods of access• Gaining more understanding of CBIR through using it

• Future aims:– Interaction between users and the system through agent technology– Extracting information of artists directly from their art

Multilevel Access to our Cultural Heritage (MACH)In

form

ati

on

Re

trie

va

l (e

.g.,

as

wit

h G

oo

gle

)


Color-Based Image Retrieval:3 methods of querying

• Browsing the catalogue

• Providing an image from the Internet, by way of an URL

• Uploading a local image from your desktop


From simple to complex user interfaces

M4ART can be described using 2 dimensions:

1) Retrieval method; either IR or CBIR

2) Complexity of querying;

- straightforward queries

- specified/combined queries


Complex IR querying (1)

• Exclusion of terms

• Boolean search, using AND and/or OR operators

• Utilizing specific tags (fields) of the annotations

Multilevel Access to our Cultural Heritage (MACH)C

om

ple

x IR

qu

ery

ing

(2

)


Complex CBIR querying (1)

• Color space

• Quantization scheme; i.e., how any colors do you want to distinguish?

• Distance measure

• Optionally: include texture

Multilevel Access to our Cultural Heritage (MACH)C

om

ple

x C

BIR

qu

ery

ing

(2

)


Gaining understanding …

• Showing the features …

• Compare IR and CBIR techniques

Multilevel Access to our Cultural Heritage (MACH)S

ho

win

g t

he

fe

atu

res

Multilevel Access to our Cultural Heritage (MACH)D

ista

nc

e m

atr

ix


Advanced schemes incorporated

• 11 color categories quantization scheme

• Parallel-Sequential Texture Analysis


Color spaces (and their quantizations)

• RGB• HSV • YUV/YIQ• CIE XYZ• CIE LUV• 11 colors ...


Its all only in 11 color categories

• Most famous: Berlin & Kay (1969) with their book

“Basic color terms: Their universals and evolution”

• But already 15 years before them Brown and Lenneberg (1954) presented their work in the paper: “A study in language and cognition”

• More recently:– Derefeldt, Swartling, Berggrund, and Bodrogi (2004)– Moroney (2004)– Regier, Kay, and Cook (2005)


The practical use of fundamental research: Color categories

• People perceive in color categories

• People talk in color categories

• Human memory uses color categories

• Robust to inter and intra personal variability

• Computationally inexpensive


THE 11 colors

• Red• Green• Blue

• Yellow• Purple• Pink• Orange• Brown

BlackGrayWhite


Texture defined …

• Cross and Jain (1983): There is no universally accepted definition for texture

• Bovik, Clarke, and Geisler (1990): an exact definition of texture either as a surface property or as an image property has never been adequately formulated.

• Gonzales and Woods (2002) state: No formal definition of texture exists, intuitively this descriptor provides measures of properties such as smoothness, coarseness and regularity.


Texture Analysis

• Palm (2004) noted: “The integration of color and texture is still exceptional.” Others: Drimbarean and Whelan (2001) and Mäenpää and Pietikäinen (2004)

• Techniques applied in literature, so far:– Global color analysis (e.g., color histograms)– Local gray-scale texture analysis (e.g., co-occurrence matrix)– Local color-based texture analysis (e.g., color correlogram)– Global color analysis combined with local gray-scale texture analysis (e.g.,

parallel approach)

• What is used in M4ART: Parallel-sequential texture analysis– Global and local color analysis in parallel: the color histogram and the color

correlogram combined.


Gray versus Color


Parallel-Sequential texture analysis*

* More info? Broek, E. L. van den and Rikxoort, E. M. van (2005). Parallel-sequential texture analysis. Lecture Notes in Computer Science (Advances

in Pattern Recognition), 3687, 532-541.


Back to C-BAR: Some last remarks

• Technical specifications

• Freely accessible

• Questions?


Technical specifications

• W3C HTML 4.01 validated

• W3C CSS validated

• Running on an Apache HTTP server

• PHP 5.1 powered

• Running on a MySQL database


Freely accessible

• All feedback = welcome!

• How to get there …

Use Google query on ‘M4ART’ or go through:

http://cai.nici.ru.nl/M4ART/

http://cai.nici.ru.nl/M4ART/


Thank you for your attention!

Questions?

DE Conferentie 2005 Egon v.d. Broek

Education

Transcript of DE Conferentie 2005 Egon v.d. Broek