Di Mascio Bozen 22

Tania Di Mascio, Eurac Research Bozen (Italy), 22 december

Cross Language Information TechnologiesFree University of Bozen, Italy

Multi-media Retrieval Systemsbased on

User Centered Design Methodology:from the Content-based to the Semantic-based approach

dott. ing.Tania Di MascioEURAC RESEARCH BOZEN


Outline

IntroductionDefinitionsOpen issues

Multi-media Retrieval Systems [20,21]Content Based Image Retrieval Systems for Vector Images [4,7,9,22]Ontology Based Text Retrieval Systems for the Semantic Web [4,7,9,22]Ontology Based Multimedia Retrieval Systems for LIS [22, 23]

Human Computer Interaction [1,3]Information Visualization

Vector Images Visualization [2,9,22]Ontology Visualization [5,6]

Usability EvaluationUsability Evaluation for software system [10,13]Usability Evaluation for web site [11]

Publications

Outline


IntroductionIntroduction definitions

Retrieval SystemsMulti-media Information

Text

Sounds

Combinations (e.g. paint)

ImagesContent-Based

Ontology Based Text Retrieval Systems

Logical-Based

Ontology-Based

Content Based Image Retrieval Systems

Ontology Based Multimedia Retrieval Systems


Open IssuesIntroduction open issues

Content Based Image

Retrieval Systems

vector images

Ontology Based Text

Retrieval Systems

Semantic web

Ontology Based Text Retrieval Systems for Semantic Web [8, 12]

Content Based Image Retrieval Systems for Vector images [4, 7, 9, 22]

Multi-media Information Retrieval Systems [20, 21]

Retrieval Issue

Ontology Based Multimedia

Retrieval Systems

LIS

Ontology Based Multimedia Retrieval Systems for LIS [22, 23]


Open IssuesIntroduction open issues

Ontology Visualization [5, 6]Vector images Visualization [2, 9, 22]

usability

visualization

Usability Evaluation for software Web systems [11]Usability Evaluation for software systems [10, 13]

Human Computer Interaction [1, 3]

Interaction Issue


OutlineOutline

Introduction

Multi-media Retrieval Systems [20,21]Content Based Image Retrieval Systems for Vector Images [4,7,9,22]Ontology Based Text Retrieval Systems for the Semantic Web [4,7,9,22]Ontology Based Multimedia Retrieval Systems LIS [22, 23]



Usability EvaluationUsability Evaluation for software system [10,13]Usability Evaluation for web site [11]

Publications


Content-Based Image Retrieval SystemsMultimedia Retrieval Systems Content Based Image Retrieval Systems

DB Images

Query Image

IndexingRepository

Query Image Indexed

RankingModule

SimilarityComputation

Module

Compairison Process

FeatureExtraction

Module

Feature Representation

Module

Users

Query InterfaceModule

Query ResutModule

Users Interface

Feature Process

Engine System


Content-Based Image Retrieval SystemsMultimedia Retrieval Systems Content Based Image Retrieval Systems Retrieval Issue

DB Images

Query Image

IndexingRepository

Query Image indexed

RankingModule

SimilarityComputation

Module

Compairison Process

FeatureExtraction

Module

Feature Representation

Module

Feature Process

CBIR Engine

Mathematic and Physic approach to solve the Retrieval Issue


Center of MassCalculation

Process

DiscretizingProcess

Content-Based Image Retrieval Systems

Feature Extraction Module

System Origin Moving

Process

CMn

ii

n

iii

xm

xm=

∑

∑

=

=

0

0

*

mi * yii= 0

n

∑

mii= 0

n

∑= yCM

Multimedia Retrieval Systems Content Based Image Retrieval Systems Retrieval Issue



Feature Representation Module

( ) ( ) jCMji

iCMji yyxxm −⋅−= ∑

;;

Regular Moments

Hu Moments

Zernike Moments

Bamieh Moments

Feature Descriptor




Similarity Computation Module

Similarity Measures

Euclidean Distance

Cross Correlation

Discrimination Cost

( )∑=

−=N

iIPiE iiFFwD

1

2

∑ ∑

∑

= =

==N

J

N

kIkPj

N

iIPi

kj

ii

FwFw

FFwD

1 1

22

0

( ) ( )( )

2

11

,min,max

, ∑= ⎥

⎥⎦

⎤

⎢⎢⎣

⎡−=

N

j ijvj

ijvj

xpxp

viD




Tuning InterfaceMultimedia Retrieval Systems Content Based Image Retrieval Systems Interaction Issue



Search Interface

VISTO (Visual Image Search TOol)

Multimedia Retrieval Systems Content Based Image Retrieval Systems Interaction Issue


SewasieUser Profile

OLAPTool

QueryTool

NegotiationTool

Ontology EditorTool

Sewasie Interaction Layer

DataMetaData

QueryAgent

BrokeringAgentInternetInternet

…SINode

Repository

Sewasie Core System

SINodeRepository

SINodeRepository

…..

Ontology-Based Text Retrieval SystemsMultimedia Retrieval Systems Ontology Based Text Retrieval Systems Retrieval Issue


Ontology-Based Text Retrieval SystemsMultimedia Retrieval Systems Ontology Based Text Retrieval Systems Retrieval Issue

the User perspective

describes the domain of the data in the information system;defines a vocabulary which is:

richer than the logical schema of data,closer than the user’s rich vocabulary.

the System perspective

enables only the generation of precise and unambiguous query expressions.

The intelligence of the Sewasie Query tool is driven by an ontology.


OLAPVisualization

Tool

NegotiationCommunication

Tool

Ontology EditorTool

DataMetaData

Retrieval Data

Single Item

SewasieUser Profile

Query BuildingModule

QueryResult

Module

Query Tool

Ontology-Based Text Retrieval SystemsMultimedia Retrieval Systems Ontology Based Text Retrieval Systems Interaction Issue


Ontology-Based Text Retrieval Systems

Query Tool InterfaceMultimedia Retrieval Systems Ontology Based Text Retrieval Systems Interaction Issue

the User perspective

to support the user in formulating a precise query in order to best captures users information needs;

complete ignorance

the System perspective

to generate a query ready to be executed by some evaluation engine associated to the information system;

conjunctive query


Ontology-Based Text Retrieval Systems

Query Tool Interface

Information Domains;Query start;Query Compose;Query result;

Multimedia Retrieval Systems Ontology Based Text Retrieval Systems Interaction Issue


Ontology-Based Multimedia Retrieval Systems

Multimedia Retrieval Systems Ontology Based Multimedia Retrieval Systems e-LIS



Multimedia Retrieval Systems Ontology Based Multimedia Retrieval Systems Retrieval Issues

ES1:(Con Ori Loc Mov not )

1) String

2) ImageES2:


“static” Interface “dynamic” Interface

based on

constraint visualisation

Users can select parameters

Users are guided in parameters choosing i.e.


Multimedia Retrieval Systems Ontology Based Multimedia Retrieval Systems Interaction Issue


OutlineOutline

IntroduzioneDefinizioneProblemi Aperti

Multimedia Retrieval Systems [20,21]Content Based Image Retrieval Systems for Vector Images [4,7,9,22]Ontology Based Text Retrieval Systems for the Semantic Web [4,7,9,22]Ontology Based Multimedia Retrieval Systems [22, 23]



Usability EvaluationUsability Evaluation for Software system [10,13]Usability Evaluation for Web site [11]

Publications


Vector Images Visualization

Human Computer Interaction Information Visualization Ontology Visualization e-LIS

Movement Parameters as animation SVG


Vector Images Visualization

Human Computer Interaction Information Visualization Vector Images Visualization VISTO

Result as Similarity Ranking List


OutlineOutline

IntroduzioneDefinizioneProblemi Aperti

Multimedia Retrieval Systems [20,21]Content Based Image Retrieval Systems for Vector Images [4,7,9,22]Ontology Based Text Retrieval Systems for the Semantic Web [4,7,9,22]Ontology Based Multimedia Retrieval Systems [22, 23]




Publications


Context analysis

Usability goals

I prototype

II prototype

Final prototype

Product

•survey evaluation

•survey evaluation •observational evaluation

(Think Aloud and Verbal Protocols)


(Think Aloud and Verbal Protocols)•controlled experiment•survey evaluation •observational evaluation

(Think Aloud and Verbal Protocols) •controlled experiment


(Think Aloud and Verbal Protocols)•controlled experiment

User Centered Design MethodologyHuman Computer Interaction Usability Evaluation


Users

Engineers

Analysing Users

TeachingUsers

Designingthe experiment

Performingthe experiment

Analysing the usability

• User-Based Criteria • A common Think Aloud Protocol method schema

User Centered Design MethodologyHuman Computer Interaction Usability Evaluation


Ontology-Based Text Retrieval SystemsHuman Computer Interaction Usability Evaluation Software System

Usability tests of the Query tool aims at perform:

The effectiveness: it is evaluated by relating the goals or sub-goals of using the Query tool to the accuracy and completeness with which these goals can be achieved.

The efficiency: it is evaluated by relating the time spent to show query result.

The satisfaction: it is evaluated by relating the comfort and acceptability of the Query tool interface.


For each query A complexity tree

P1

P1

P2

P3

P3P1

P2P1

l=1

l=2

l=3

l=4

Level

C1

C1C1

C1C1

C2

C2

Ontology-Based Text Retrieval SystemsHuman Computer Interaction Usability Evaluation Software Systems

Users

Engineers

Analysing User

s

Teaching

Users

Designingthe

experiment

Performingthe

experiment

Analysing the usabilit

y


P1

P1

P2

P3

P3P1

P2P1

C1

C1C1

C1C1

C2

C2

C1 = 0,1 if edge = add proprierty

C2 = 0,2 if edge = add concept

Pn = n for each node of level

( )∑∑ ==

levelpernodnum

n nlelnum

l lnflpcl ___

12;1_

1 ),(*)(* If l < lmax

If l=lmax⎩⎨⎧

=1

___),(

levelsucnumevrlnf

Users

Engineers

Analysing User

s

Teaching

Users

Designingthe

experiment

Performingthe

experiment


yOntology-Based Text Retrieval SystemsHuman Computer Interaction Usability Evaluation Software Systems


For example:

“Supplier located in Europe which is a Multinational selling on the Italian Market”

The complexity is: 1*[0.1*1]*2+2*[0.2*1+0.1*2]*0.5+3*[0.1*1]*1 = 0.9

Supplier

located_in

EuropeMultinational

selling_on

Italian_market

Which is a 0,1

1

21

1

0,1

0,2 l=1

l=2

l=3

Users

Engineers

Analysing User

s

Teaching

Users

Designingthe

experiment

Performingthe

experiment


yOntology-Based Text Retrieval SystemsHuman Computer Interaction Usability Evaluation Software Systems


DIEL web-site SystemsHuman Computer Interaction Usability Evaluation DIEL Web Systems

21,6%35,3%Non Computer Skilled

5,2%11,7%Computer Skilled

17,02%29,7%Non Domain Expert

20,07%33,4%Domain Expert

BOXTRADEffectiveness measure (*)


Outline

Introduzione

Multimedia Retrieval Systems [20,21]Content Based Image Retrieval Systems for Vector Images [4,7,9,22]Ontology Based Text Retrieval Systems for the Semantic Web [4,7,9,22]




Publications

Outline


PublicationsPublicationsPublications

Libri Editati[1] L. Tarantino and T. Di Mascio. Atti di HCItaly Symposium 2005, September 13 2005, Rome (Italy). Finito di stampare in Luglio 2005. Studio Grafico Pierpaolo Ceccarelli.

Riviste Internazionali[2] D. Frigioni, L. Tarantino, T. Di Mascio. On Formal Aspects of Zooming in Geographic Maps. Informatica, Vol. 26, n. 3, pp. 309–319, 2002. Slovene Society Informatika.[3] L. Tarantino and T. Di Mascio. Advances Visual Interfaces: the Focus is on the User. The Knowledge Engineering Review, Vol. 18:2, pp. 175–181, 2003. Cambridge University Press, printed in the United Kingdom.

Conferenze Internazionali[4] T. Di Mascio and L. Tarantino. Main Features of a CBIR prototype supporting cartoon production. In Proceedings of the 10th Human Computer Interaction International Conference (HCI2003), June 22–27 2003, Crete (Greece), Vol. 1, pp. 921–925, 2003. Lawrence Erlbaum Associates.[5] T. Di Mascio. Assisting User of Semantic Web: the Sewasie Proposal. In Proceedings of the 10th Human Computer Interaction International Conference (HCI2003), June 22–27 2003, Crete (Greece), Adjunct Proceedings, pp. 157–159, 2003. Lawrence Erlbaum Associates.[6] T. Catarci, T. Di Mascio, E. Franconi, G. Santucci and S. Tessaris. An Ontology-based Visual Interface for query generation support. In Proceedings of International Conference on Ontologies, Databases and Applications of SEmantics (ODBASE2003), November 03–07 2003, Catania (Italy), Adjunct Proceedings, pp. 32–34, 2003. Springer-Verlag.[7] T. Di Mascio, M. Francesconi, D. Frigioni, and L. Tarantino. Tuning a CBIR System for vector images: the interface support. In Proceedings of the Advanced Visual Interface (AVI2004), May 25–28 2004, Gallipoli (Italy), pp. 425–428, 2004. ACM Press.[8] T. Catarci, P. Dongilli, T. Di Mascio, E. Franconi, G. Santucci and S. Tessaris. An Ontology based visual tool for query formulation support. In Proceedings of European Conference on Artificial Intelligence (ECAI2004), August 24–27 2004, Valencia (Spain), pp. 308–312, 2004. IOS Press.[9] T. Di Mascio, D. Frigioni, and L. Tarantino. A visual environment for tuning a contentbased nvector image retrieval engine. In Proceedings of the 11th Human Computer Interaction International Conference (HCI2005), June 22–27 2005, Las Vegas (USA), Adjunct Proceedings, 2005. Lawrence Erlbaum Associates.[10] T. Catarci, T. Di Mascio, P. Dongilli, E. Franconi, G. Santucci, and S. Tessaris. Usability evaluation in the SEWASIE (SEmantic Webs and AgentS in Integrated Economies) project. In Proceedings of the 11th Human Computer Interaction International Conference (HCI2005), June 22–27 2005, Las Vegas (USA), Vol. 2, 2005. Lawrence Erlbaum Associates. [11] T. Di Mascio, I. Salvatore, and L. Tarantino. Navigation by zooming in boxes: preliminary evaluation. In Proceedings of the Advanced Visual Interface (AVI2006), May 23–26 2006, Venice (Italy), pp. 262–265. ACM Press.


PublicationsPublicationsPublications

[12] R. Gennari and T. Di Mascio. The e-LIS Project: an Ontology based Approach to an Electronic Dictionary for Italian Sign Language. In Proceedings of the second Workshop on Formal Ontology (FOMI2006), December 14–15 2006, Trento (Italy). IOS Press.

Conferenze Nazionali[13] T. Catarci, T. Di Mascio, P. Dongilli, E. Franconi, G. Santucci, and S. Tessaris. An Ontology- Based Query Manager: Usability Evaluation Proceedings of the 4th Italian Symposium on Human Computer Interaction (HCItaly2005), September 13 2003, Roma (Italy), pp.95–100.

Rapporti Tecnici di Progetto[14] E. Franconi, S. Tessaris, T. Catarci, T. Di Mascio, G. Santucci, and G. Vetere. Specification of the tool for the design, management and storage of the Semantic Information related to the data stored. Technical Report D6.2 of Sewasie project (IST FP5 Programme-IST-2001-34825), February 2003.[15] S. Tessaris, R. Bernardi, T. Catarci, T. Di Mascio, E. Franconi, G. Santucci, and G. Vetere. Specification of the tool for end-user query management. Technical Report D6.3 of Sewasie project (IST FP5 Programme-IST-2001-34825), March 2003.[16] T. Di Mascio, T. Catarci, G. Santucci, E. Franconi, S. Tessaris, and G. Vetere. Evaluation of existing tools, description of the user centered design. Technical Report D6.1 of Sewasie project (IST FP5 Programme-IST-2001-34825), May 2003.[17] D. Montanari, A. Tavernari, T. Di Mascio, A. Jertila, S. Tessaris, H. Roberts, A. Becks and A. Brunner. First tests and analysys of test results. Technical Report D9.3 of Sewasie project (IST FP5 Programme-IST-2001-34825), May 2004.[18] A. Tavernari, D. Montanari, T. Di Mascio, A. Jertila, R. Offermanns, A. Becks and C. Seeling. Second tests and analysys of test results. Technical Report D9.4 of Sewasie project (IST FP5 Programme-IST-2001-34825), February 2005.[19] T. Di Mascio, T. Catarci, G. Santucci, D. Montanari, A. Tavernari, A. Jertila, E. Franconi, P. Dongilli, J. Heguiabehere and A. Becks. Evaluation of the system usability. Technical Report D9.5 of Sewasie project (IST FP5 Programme-IST-2001-34825), March 2005.

Lavori sottomessi a Riviste Internazionali[20] T. Di Mascio, D. Frigioni, and L. Tarantino. A CBIR system for vector images: the tuning interface.[21] T. Di Mascio, S. Tessaris, P. Dongilli, T. Catarci, G. Santucci and E. Franconi. Browsing and Querying in the Sewasie Environment.

Lavori sottomessi Conferenze Internazionali[22] T. Di Mascio, L. Laura, V. Mirabella, and L. Tarantino. The interface of VISTO: a CBIR system for vector images.[23] G. Marcelli, T. Di Mascio MDSS Discovery Support System.


The EndThe End

Grazie!Grazie!

tania@[email protected]

Di Mascio Bozen 22

Documents

Transcript of Di Mascio Bozen 22