Ontologies of Information Structure and

Commonsense Psychology

Jerry R. Hobbs

USC/ISI

Marina del Rey, CA

Information Structure

Motivation

The best answer to a question is often a diagram, a graph, a map, a photograph, a video.

What is the Krebs cycle?

How has the average height of adult American males varied over the years?

How did the Native Americans get to America?

What does Silvio Berlusconi look like?

What happened on September 11, 2001?

Grounding Symbols in Cognition

cause(perceive(a,x), cognize(a,c))

objectstateevent

processabsence

...

concept(including

propositions)

Grounding Symbols in Cognition

cause(perceive(a,smoke), cognize(a,fire))cause(perceive(a,cloud), cognize(a,dog))cause(perceive(a,bell), cognize(a,food))

No necessary causal connection between x and c

This schema makes symbols possible

cause(perceive(a,x), cognize(a,concept-of(x)))

but other things as well

Grounding Symbols in Cognition

cause(present(b,x,a), perceive(a,x))

cause(perceive(a,x), cognize(a,c))

cause(car beeps, driver hears beep)

cause(driver hears beep, driver remembers seat belt)

Intention and Convention in Communication

Unintentional: fidget --> nervous “ouch” --> pain

Presenter intends concept, but not recognition of intent: my door is closed --> I’m not in

Recognizing Intent

know(b, cause(present(b,x,a), cognize(a,c)))

goal(b,cognize(a,c))

goal(b,g1) & know(b, cause(g2,g1)) & etc --> goal(b,g2)

So b has goal present(b,x,a), an executable action, so he does it.

a looks for causal explanation of present(b,x,a)and comes up with exactly this

Intention is recognized.

Gricean Nonnatural Meaning

If an agent b has a goal g1 and g2 tends to cause g1, then b may have as a goal that g2 cause g1.

If an agent b has as a goal that g2 cause g1, then b has the goal g2.

When a recognizes this plan, he will recognize not only b’s goal to have a cognize c, but also b’s intention that a do so by virtue of the causal relation between b’s presenting x and a’s cognizing c.

Mutual Belief and Convention

Mutual Belief:

mb(s,p) & member(a,s) --> believe(a,p) mb(s,p) --> mb(s,mb(s,p))

Structure of a communicative convention:

mb(s, cause(present(b,x,a), cognize(a,c))) where member(a,s) and member(b,s)

e.g. red flag with white diagonal in community of boaters means “diver below”

represent(x,c,s)

symbol content

Composition in Symbol Systems

Symbol System Content Domaincomposite

symbol

basicconcept/

proposition

atomicsymbol

complexconcept/

proposition

compositionoperation

compositionoperation

interpretation

interpretation

Speech and Text(within sentences)

Symbol System Content Domainsentences

“a man works”

basicpropositions

man(x), work(y)

words“a” “man”, “works”

complexproposition

man(x) & work(x)

pred-arg rels,conjunction

concatenation

interpretation

interpretation

Speech and Text(Discourse)

Symbol System Content Domain

discourse

sentencemeanings

sentences

augmenteddiscoursemeaning

coherence relations:causality, similarity,

figure-ground

concatenation

interpretation

interpretation

Tables

R

a b

Spatial arrangement ==> predicate-argument relations

R(a,b)

Beeps in Car

Atomic symbol: beep ==> something’s wrong

Composite symbols: beep ..... beep ..... beep ..... ==> fasten seat belt If car is running: beep beep beep beep ==> door is still open If car is off: beep beep beep beep ==> lights are still on

Maps

Underlying regions of single color/pattern

icons

internal structureof icons

labels

name of entity

names

entities

meaningfulregion

overlay iconson field

icon and labeladjacent

categoriesof entities

locationof entities

==>

==>

==>

==>

==>

==>

Process Diagrams

icons entities

adjacentgrouped icons

states

adjacent groupsw arrows between

statetransitions

(Futrelle, 1999)

Documents (Scott & Powers, 2003)

Title

Body

Adjacent Paragraphs(mod Page, Col breaks)

Diagram neardescription

Conveys content of body

Main detailed content

Read sequentially

Coreference

Similarly, Web pages, PowerPoint presentations, ...

Face-to-Face Conversation

Atomic elements:

Speech, prosody

Facial expression

Gaze direction

Body position

Gestures w hands and arms

Composition operators:

Temporal adjacency

Temporal synchrony

Need to determinemeaning/functionof various behaviors

Larger-Scale CommunicativePerformances

Lectures w PowerPoint slides

Plays

Demos

....

Coreference

Two noun phrases

Icon and label

Same icon in two state groups in diagram

Region of photo and noun phrase in caption and phrase in text

Iconic gesture and phrase in speech

Useful for image search by keywords

Modalities and Media (Hovy & Arens, 1990; Allwood, 2002)

Channels of perception: optical, acoustic, chemical, pressure, temperature, ...

Greatest opportunities for composition

Communication devices:

Primary: speech, gesture Secondary: writing, drawing, telephones, videotape, computer terminals, ....

Advantages and disadvantages of each e.g., visual: exploit 2-D structure to convey relations

Manifestations of Symbolic Entities

We group together classes of symbolic entities sharing same content and call them first class entities.

manifest(x1,x) & represent(x,c,s) --> represent(x1,c,s)

(defeasibly -- if P is in content of x then defeasibly it is in content of x1)

Hamletthe play

The performanceof Hamlet

A particularperformance

of Hamlet

A videotapeof that

performance

A copyof that

videotape

The text of Hamlet

An editionof Hamlet

A copy ofthat edition

(Pease & Niles, 2001)

CommonsensePsychology

(work with Andrew Gordon, USC/ICT)

Methodology

Agents plan, so to discover what agents know, investigate strategies.

Picked 10 planning domains: politics, warfare, personal relationships, artistic performance, sales, immunology, animal camoflage, ...

Interviewed experts to learn strategies

Resulted in 372 strategies

Rewrote strategies in controlled vocabulary -- 988 terms

Classified terms into 48 representational areas (space, time, ...); 18 general knowledge; 30 commonsense psychology

Enrich each representational area by text mining

Formalize

(Gordon, 2000)

Methodology

Agents plan, so to discover what agents know, investigate strategies.

Picked 10 planning domains: politics, warfare, personal relationships, artistic performance, sales, immunology, animal camoflage, ...

Interviewed experts to learn strategies

Resulted in 372 strategies

Rewrote strategies in controlled vocabulary -- 988 terms

Classified terms into 48 representational areas (space, time, ...); 18 general knowledge; 30 commonsense psychology

Enrich each representational area by text mining

Formalize

(Gordon, 2000)

Machiavelli Sun Tzu his wife

Theories So Far

Memory

Knowledge Management

Envisioning (Thinking)

Goals and Planning

Why is Memory Important?

We plan to remember actions/information at the appropriate time.

We are responsible for remembering. Why was Mary angry that John forgot her birthday? But forgetting is often a less serious breach than some other reason. Why didn’t you get me a present? I forgot it was your birthday. vs. I didn’t want to.

Naive Model of Memory

Focus of Attention

Memory

concept

concept

store

retrieve

If in memory,then it was stored

Accessibility

concept-1

concept-2

concept-3

concept-4

Concepts in memoryhave varying accessibility.

threshold

Concepts notretrievable

Associations and Accessibility

concept-1

concept-2

concept-3

concept-4

concept-1

concept-2

concept-3

concept-4

concept-0

Associations and Accessability

Thinking of concepts makes associated concepts more accessible.

This give agents partial control over memory retrieval.

Technique of memorization: Rich associations.

“Remember” and “Forget”

in memory above accessibility threshold --> remember

retrieve --> remember

cause self to retrieve --> remember

cause self to retrieve after some effort --> remember

forget concept <--> concept drops below accessibility threshold

Remembering for a Time

We store concepts in memory until we need them and then forget them.

Where did I park my car today? vs. Where did I park my car on January 4?

We use memory to satisfy knowledge prerequisites for planned actions.

Knowledge Management:Belief

Reify agents and propositions: believe(a,p)

Reasoning is possible inside belief: believe(a,p) & believe(a,p-->q) & etc --> believe(a,q)

Perception causes belief (seeing is believing)

Communication tends to cause belief

BDI: We act in ways that maximize satisfaction of our goals, given our beliefs

Graded Belief (Friedman & Halpern, 2001)

0 gb(a,p) 1

gb(a, p&q) min(gb(a,p), gb(a,q))

gb(a, p & [p-->q]) = gb(a,q)

gb(a,p) = 1 <--> believe(a,p)

The higher the graded belief, the more likely agent is to act on it

Knowledge Domain

Sentence = set of propositions + a claim

king(x,France) & bald(x)

Knowledge domain: Has a set of characteristic predicates Is a set of sentences all of whose claims have predicates that are in the characteristic set

Expert: Agent is defeasibly an expert in a knowledge domain if agent knows sampling of facts in the knowledge domain (tests, inference from displays of knowledge)

propositionalcontent claim

Mutual Belief

mb(s,p) & member(a,s) --> believe(a,p)

mb(s,p) --> mb(s,mb(s,p))

These rules are mutually believed

Can show that if a knows b is a member of s and a knows s mutually believes p, then a knows that b believes p

Inference of who knows what / who is an expert in what from membership in communities

Causal Complex

e1 e2

e3 e4e

....

s

causal-complex(s,e)

e1 s, ....

When every event or state in s happens or holds, then e happens or holds.

All eventualities in s are relevant.

causally-involved(ei,e)

causal complex

effect

Cause

In a causal complex, some eventualities are distinguished as causes.

power on

finger insocket

shock

What is presumable depends on task, context, knowledge base, ....

presumable

cause

Causes are the focus ofplanning, prediction,

explanation, interpretingdiscourse

(but not diagnosis)

Envisioning (Thinking)

e1

e2

e3

e4

e5

e6

e7

e8

e11

e10

e9

e’s are causally involved

Causal System

Envisioning

e4

e5

e6

e4

e5

e6

e7

e8

e1

e2

e3

e4

e5

e6

Contiguouscausal systems

Envisioning

e1

e2

e3

e4

e5

e6

e7

e8

e11

e10

e9

envisionedcausal system

slice

Agent has this in focus

Envisioned Causal System

e1

e2

e3

e4

e5

e6

e7

e8

e11

e10

e9

ExplanationPrediction

A sequence of envisioned contiguous causal systems

Correspondence with Reality

If the events and states in the ECS are believed, the ECS is the “current world understanding”

Need an account of how graded belief is increased or decreased as predictions and explanations are verified or falsified.

Goals and Planning

Causal Knowledge:

(e1,x)[p’(e1,x) --> (e2)[q’(e2,x) & cause(e1,e2)]] or, p causes q

(e1,x)[p’(e1,x) --> (e2)[q’(e2,x) & cause(e1,e2)]] or, p enables q

where enable(e1,e2) <--> cause(~e1,~e2)

Planning Axioms:

(a,e1,e2)[goal(a,e2) & cause(e1,e2) & etc --> goal(a,e1)]

(a,e1,e2)[goal(a,e2) & enable(e1,e2) --> goal(a,e1)]

subgoal(a,e1,e2)

Goals and Planning

Goals can be ...

competitive adversarial auxilliary .....

Collective Goals

Groups can have goals:

All agents in group mutually believe the group has the goal

All agents have the individual goal that the group achieves its goal

Must bottom out in individual agents’ actions

Organizations are such collective plans made concrete; an agent’s role in an organization is the actions the agent carries out as a subgoal in the collective plan

Where Do Goals Come From?

A False Mystery

Stipulate: goal(a, thrive(a))

All else is causal knowledge/beliefs about what causes thriving

Goal Themes

goal-theme(s,t) <--> ( a,e) [member(a,s) & member(e,t) & etc ---> goal(a,e)]

From group membership, we can infer beliefs and goals and thus behavior (defeasibly)e.g., he’s a puritan / hedonist / geek / ....

group ofagents

set of possibleeventualities

Summary

Ontologies are important for communicating the contents and capabilities of Web sites and Web resources

Most of this information is now in the form of natural language

We need ontologies that are capable of expressing the full range of content in Web sites, forming the basis of a deeper lexical semantics

I have presented first cuts at some of the most basic ontologies needed: services, events, time, space, information, human psychology