project I2RPIntelligent Information Retrieval and Presentation

in public historical multimedia databases

prof. dr. L. Schomaker

KI/RuG

2

ToKeN2000 grants for research between computer science, AI

and cognitive science

money from Min. of Econ. affairs and Min. of Education

demonstrating that the ‘human perspective’ has an added value

demonstrating that working systems and/or models can be implemented

3

ToKeN2000Project Title

EIDETIC Intelligent Content-based Image Retrieval

I2RP Intelligent IR and Presentation in public historical multimedia databases

DUMPERS Distributed User Modeling and Exploration in Personalized Recommender Systems

CHIME Cultural Heritage in an Interactive Multimedia Environment

AUTHENTIC Knowledge discovery and disclosure for visual art: authentication and dating of graphic art and paintings

ANITA Administrative Normative Information Transaction Agents

VINDIT Combining visual and textual information for IR

MIA Medical Information Agent

DIME Distributed Interactive Medical Exploratory for 3D Medical Images

TIMEBAYES Building and Using Temporal Bayesian Models in a CPR setting

NARRATOR Narrative disclosure of health-care knowledge

4

I2RP partners

CWI

Universiteit Leiden

Universiteit Maastricht

Rijksuniversiteit Groningen

Rijksmuseum Amsterdam

5

prof. L. Hardman CWI/TUE

prof. dr. H.J. van den Herik UM

prof. dr. G.A.M. Kempen UL

prof. dr. L.R.B. Schomaker RUG

dr. I. Sprinkhuizen-Kuyper UM

dr. J. van Ossenbruggen CWI

dr. N. Taatgen RUG

Supervisors

+ Rijksmuseum: dhr. K. Schoemaker

6

Researchers

drs. Stefano Bocconi OIO CWI

dr. Floris Wiesman postdoc IKAT

drs. J. Grob OIO RUG

drs. C. van Breugel UL

+ M.Sc. students

7

Intelligent Information Retrieval and Presentation

Information Retrieval: searching in weaklyorganized multimedial databases

Presentation: user and context-relatedrendering of retrieved results

“Intelligent”, i.e., making use of methodsfrom AI and Cognitive Science

Upper-left picture is the query

“boy in yellow raincoat”

…yields very counter-intuitive results

What was the user’s intention?

9

Human-machine communication

Grice’s Maxims of bi-directional cooperative dialog: quantity (adapt the size of your answer) quality (tell the useful truth) relation (react to what has been asked) manner (avoid ambiguities)

Current HMC violates most of these maxims

10

Starting points in I2RP Bidirectional cooperative dialog (Grice) (maxims of quantity, quality, relation, manner)

An example of ‘intelligent information retrieval and presentation’: car sales

Buyer: “I’m looking for a Volvo 850 Estate for less than 5000 Euro”

11

Starting points in I2RP Bidirectional cooperative dialog (Grice) (maxims of quantity, quality, relation, manner)

An example of ‘intelligent information retrieval and presentation’: car sales

Buyer: “I’m looking for a Volvo 850 Estate for less than 5000 Euro”Seller: “we don’t have it” (logical response)

12

Starting points in I2RP Bidirectional cooperative dialog (Grice) (maxims of quantity, quality, relation, manner)

An example of ‘intelligent information retrieval and presentation’: car sales

Buyer: “I’m looking for a Volvo 850 Estate for less than 5000 Euro”Seller: “we don’t have it” (logical response)

vsSeller: “we do have a Mitsubishi Station of 5500 Euro” (intelligent response)

13

Reasoning with world knowledge

(1) Volvo 850 Estate (3) Mitsubishi Station

(2) family car!

all cars

sports cars SUVs

14

Knowledge sources in I2RP

A bi-directional cooperative dialog (Grice)…

Requires: world knowledge semantic web, ontologies knowledge on humans user modeling, language

15

Project Partners

Optima: A user agent for object-based image search

Spreekbuis: A Dutch sentence generator

Cuypers: Automatic user-centric hypermedia generation

GO: Graphical Ontologies

16

Spreekbuis: a sentence generator for Dutch

UL (C. van Breugel/Arsenijevic)

Performance Grammar Workbench (PGW)

17

Optima: a user agent for object-based image search

KI/RuG, Taatgen/Grob/Schomaker

User modeling , learning in ACT-R

KI

RuG

18

Cuypers: user-centered hypermedia generator

CWI

Stefano Bocconi, AIO per 01-01-2002

using knowledge on graphical design and communication in the application domain

19

GO: Graphical Ontologies

IKAT/UM (Floris Wiesman)

‘Generic tool for searching (navigating), accessing, and editing ontologies’

MetaBrowser: a graphical browser for information retrieval

20

Goal of the meeting

a lot of mono-disciplinary research exists

… based on toy problems or artificial data

(TREC, multimedia retrieval benchmark dBs)

… barely looking at the user requirements

I2RP we can do it better!

21

System: application/experimentation

RenderingSemantics

User Modeling Speech/Language

Multimedia retrievalapplication

22

System: application/experimentation

Multimedia retrievalapplication

dB

UI

Optima/ACT-R

GO Cuypers

Spreekbuis

23

Dependencies

RenderingSemantics

User Modeling Speech/Language

dB

UI

Optima/ACT-R

GO Cuypers

Spreekbuis

24

Agenda

Group introduction

Bilateral discussions

Integration

Concrete goals: define Milestones Experimentation-platform specification Demonstrable output

25

Agenda bilateral 20-min. discussions

Room C001 UM + RuG UL + RuG UM + UL

Room C002 UL + CWI UM + CWI CWI + RuG