Using Non-Taxonomic Knowledge to Improve Semantic Matching Peter Yeh July 22, 2003.

Using Non-Taxonomic Knowledge to Improve Semantic Matching

Peter Yeh

July 22, 2003

Talk Outline

• Introduction

• Analysis of Existing Techniques

• Our Approach

• Initial Evaluation

• Proposed Work

Introduction

• Many AI tasks require determining whether two knowledge representations encode the same knowledge.

Information Retrieval

• Match queries with documents.

Q: “A car with a bumper made of gold.”

Car

Bumper Gold

has-part

material

Car

Gold

material

Car Acme

Produce agentobject

Car

Bumper Gold

has-part

material

A: “Acme makes a car made of Gold.”

Knowledge Acquisition• Match new knowledge with existing knowledge.

KB

KB: Are you trying to encode a conversion?

Microbe Pollution

agent object

Destroy Createcauses

Food

result

Microbe Pollution

agent object

Destroy Createcauses

Food

result

New Knowledge

Conversion

next-event

agent object

Destroy Create

resultagent

subevent subevent

EntityEntity Entity

Existing Knowledge

Rule-based Classification• Match rule antecedents with working memory. For example,

Course of Action (COA) critiquing.

Attack Delay

Military-Unit agentagent

objectobject

causes

Military-Unit

IF

THEN<good, Enemy-Maneuver-Engagement>

Attack Block Delay

Advance

Artillery-Unit

Armor-Unit

agentagent

object

object

object

agent

agent

causes causes

Pattern COA

Attack Delay


objectobject

causes

Military-Unit

“This COA has a rating of good for enemy maneuver engagement.”

The Core Problem

• Solving this matching problem is hard because multiple encodings of the same knowledge rarely match exactly.

• Representations don’t match exactly because: – Expressive Ontology. – Knowledge is encoded by different sources.– Knowledge being encoded is complex.

Types of Mismatches

• Informal examination of a knowledge-base containing: – Patterns. – COAs.

• Knowledge-base was built by two Subject Matter Experts (SMEs) participating in DARPA’s RKF project.

• Looked for cases of mismatch.

Types of Mismatches (cont.)

• Taxonomic Differences

“an armored brigade engaging an armored battalion.”


• Taxonomic Differences• Equivalent Alternatives

“One military unit attacking another unit.”


• Taxonomic Differences• Equivalent Alternatives• Omissions

“Mechanized infantry brigade engaging mechanized infantry

battalion.”


• Taxonomic Differences• Equivalent Alternatives• Omissions• Granularity

“Support attack occurs before main attack.”

Analysis of Existing Techniques

• Analogy

• Inexact Matching

• Semantic Matching

• Conceptual Indexing

• Ontology Merging

Analogy

• Analogy: mapping of knowledge from a base domain to a target domain.

• Structure Mapping Engine (Forbus et. al. 89): – Maps relational knowledge (mappable systems).– Systematicity Principle used to select best analogy.

• Analogy based on common generalizations (Leishman 92)– Maps both relational knowledge and object

attributes.– Prefers minimal common generalization.

Analogy: Structure Mapping Engine

Attack Delay


objectobject

causes

Military-Unit

agentagent

objectobject

causes

Attack Block Delay

Advance

Artillery-Unit

Armor-Unit

agentagent

object

object

object

agent

agent

causes causes

agentagent

object

object

object

agent

agent

causes causes

agent

object

object

agent

causes

agent

object

object

agent

causes

agent

object

object

agent

causes

agent

object

object

agent

causes

Attack Block

Artillery-Unit

Armor-Unit

agent

object

object

agent

causes

Block Delay

Artillery-Unit

Armor-Unit

agent

object

object

agent

causesAttack Delay


objectobject

causes

Military-Unit

Inexact Matching

• Inexact Matching: tries to address mismatches between representations

• Graph Editing (Tsai et. al. 83, Shapiro and Haralick 81, Messmer et. al. 93, Wolverton et. al. 2003)– Uses edit distance parameters. – Similarity based on shortest sequence of edits.

• Partial Matching – Does not require representations to be isomorphic.– Similarity based on amount of structural overlap.– Minimal Common Supergraph (Bunke et. al. 2000) and

Maximal Common Subgraph (Bunke and Shearer 98).

Inexact Matching: MCS

Attack Delay


objectobject

causes

Military-Unit

Attack Block Delay

Advance

Artillery-Unit

Armor-Unit

agentagent

object

object

object

agent

agent

causes causes

Block Delay

Artillery-Unit

Armor-Unit

agent

object

object

agent

causes

Attack Block

Artillery-Unit

Armor-Unit

agent

object

object

agent

causes

Attack Delay


objectobject

causes

Military-Unit

Semantic Matching• Semantic Matching: uses knowledge to match

representations.• Projection:

– Uses taxonomic knowledge. – Ontoseek (Guarino et. al. 99) and ELEN (Huibers et. al. 96).

• Projection+: Projection alone is too restrictive -projection (Genest and Chein 97).– Common generalization, graph splitting, regular expressions

(Fargues 92, Buche et. al. 2000, Martin et. al. 2001).

• Semantic Overlap– Maximal Joins and Generalizations (Myaeng 92, Poole et.

al. 95).– Shared Semantic Structures (Zhong et. al. 2002).

Semantic Matching: Semantic Overlap

Attack Delay


objectobject

causes

Military-Unit

Attack Block Delay

Advance

Artillery-Unit

Armor-Unit

agentagent

object

object

object

agent

agent

causes causesAttack Block Delay

Advance

Artillery-Unit

Armor-Unit

agentagent

object

object

object

agent

agent

causes causesAttack Delay

Artillery-Unit

Armor-Unit

agentagent

objectobject

Attack Delay

Artillery-Unit

Armor-Unit

agentagent

objectobject

Attack Delay


objectobject

causes

Military-Unit

Conceptual Indexing

• Conceptual indexing: how to organize and index knowledge.

• Requires so form matching.• Generalization hierarchy (Bournard et. al. 95,

Ellis 92, Levinson 82, Woods 97). – Knowledge indexed by common generalizations. – Generalizations organized hierarchically by

subsumption relationships. – Retrieve Most Specific Subsumer (MSS) of a query.

• Match procedure is similar to Projection - suffers the same problems.

Ontology Merging and Translation

• Ontology Merging: merge multiple ontologies built by different sources– Chimaera (McGuinness et. al. 2000)– SMART (Noy and Musen 99).

• Ontology Translation: translates a representation from one language to another– Ontomorph (Chalupsky 2000).

• Goals are different but share some of the same problems.

Our Approach

• The goal of this research is to solve the matching problem.

• We believe existing semantic approaches can be extended with additional knowledge to significantly improve matching.

• What kinds of additional knowledge?– Transformations

• Handle mismatches.• Improve matching.

– Not taxonomic knowledge.

Our Approach (cont.)

• Generality and domain-independence.– Want additional knowledge (e.g. Transformations) to

be useful across domains.

• We believe domain-independence is possible given a reusable domain-neutral upper ontology.– Contains a small set of general concepts.

– SMEs use this upper ontology to build KBs on specialized topics (e.g. chemistry, biology, battle space planning).

– No training in logic or knowledge representation.

Illustration of Our Framework

Transformations

Ontology

KB

KE

SME/KE KB can be viewed as a domain-specific matcher (e.g. match symptoms to diseases).

Domain-independent KB for the task of matching.

Our Prototype

• Extend semantic matchers with transformations.• Apply transformations in a forward-chaining

manner.• Use existing techniques for reasoning with

Conceptual Graphs (Corbett et. al. 99, Salvat et. al. 96, Willems 95): – Projection.– Unification.– Graph rules.

• Two caveats because existing techniques lead to promiscuous matches.

Transformations that Retains Semantics

Buy object

agent

origin

Car

Person: Y

Person: X

Buy object

agent

origin

Car

Person: Y

Person: X

Car Like

Person: X

object

agent

Projection

Car Likeobject

agent

Driving-License possesses

Buy object

agent

origin

Car

Person: John

Person: Bob


Buy object

agent

origin

Car

Person: John

Person: BobCar Sell

Person

object

agent

recipientPerson

Transformations that Retains Semantics

Buy object

agent

origin

Car

Person: Y

Person: X


Buy object

agent

origin

Car

Person: John

Person: BobCar Sell

Person

object

agent

recipientPerson

Buy object

agent

origin

Car

Person: Y

Person: X

Car Sell

Person: Y

object

agent


Buy object

agent

origin

Car

Person: John

Person: Bob

Sellobject

agent

Car

Person: John

Sellobject

agent

Car Sell

Person: Y

object

agent

Rule Applicability

Buy object

agent

origin

Car

Person: Y

Person: X

Car Sell

Person: Y

object

agent


Buy object

agent

origin

Car

Person: John

Person: Bob

Sellobject

agent


Buy object

agent

origin

Car

Person: John

Person: Bob

Buy object

agent

origin

Car

Person: Y

Person: X

Buy object

agent

origin

Car

Person

Person


Rule Applicability


Buy object

agent

origin

Car

Person: John

Person: BobCar Sell

Person

object

agent

recipientPerson

Buy object

agent

origin

Car

Person: Y

Person: X

Car Sell

Person: Y

object

agent


Buy object

agent

origin

Car

Person: John

Person: Bob

Sellobject

agent

Buy object

agent

origin

Car

Person: Y

Person: XCar Sell

Person: Y

object

agent

Enumerating Transformations

• Transformations derived from our domain-neutral upper ontology.

• Enumerated all ways that a relation can be legally used to encode information in a conceptual graph.

• Considered whether the same information can be expressed differently.

• Enumeration was possible because:– Small upper ontology. – Each concept had well-defined semantics.

Transformations Enumerated

• We were able to enumerate about 300 transformations.

• Resulting transformations fall into three general categories:– Transitivity – Part Ascension – Transfers Through

Transformations Enumerated (cont.)

relation Transitive Part Ascension Transfers Through

causes X - subevent, resulting-state

caused-by X subevent-of resulting-from

defeats - - -

defeated-by - subevent-of caused-by

enables X - causes, resulting-state, subevent

enabled-by X subevent-of caused-by, resulting-from

inhibits - subevent-of resulting-state

inhibited-by - subevent-of caused-by, resulting-from

by-means-of X - -

means-by-which X - -

prevents - subevent-of -

prevented-by - subevent-of caused-by, resulting-from

resulting-state - - causes

resulting-from - - -

Example: Our Approach

Attack Delay


objectobject

causes

Military-Unit

Attack Block Delay

Advance

Artillery-Unit

Armor-Unit

agentagent

object

object

object

agent

agent

causes causes

agent

agent

object

objectMilitary-Unit

Delaycauses

Attack

Military-Unit

Attack

Attack

Military-UnitDelay

Military-Unit

Delay

1:

2:

3:

4:

5:

agent

agent

object

object

causes

Attack

Attack

Attack

Delay

Delay

A:

B:

C:

D:

E:

Advance agent

Block

Artillery-Unit

Artillery-Unit

Artillery-UnitBlock

Armor-Unit

Armor-Unit

object

agent

Block Armor-Unit

Block Delaycauses

Armor-Unit

F:

G:

H:

I:

l1 = l1 = {(1,A)}

M = { }{(1,A)}


{(1, A)}, {(3,C)}, {(4,D)}, {(5,E)} }

M = {

A

B

C D

E

F

G

H

I

Attack Block Delay

Advance

Artillery-Unit

Armor-Unit

agentagent

object

object

object

agent

agent

causes causes

Attack Delay


objectobject

causes

Military-Unit1

4

2

3

5

A

B

C D

E

F

G

H

I

Attack Block Delay

Advance

Artillery-Unit

Armor-Unit

agentagent

object

object

object

agent

agent

causes causes

Attack Delay


objectobject

causes

Military-Unit1

4

2

3

5

A

B

C D

E

F

G

H

I

Attack Block Delay

Advance

Artillery-Unit

Armor-Unit

agentagent

object

object

object

agent

agent

causes causes

Attack Delay


objectobject

causes

Military-Unit1

4

2

3

5

A

B

C D

E

F

G

H

I

Attack Block Delay

Advance

Artillery-Unit

Armor-Unit

agentagent

object

object

object

agent

agent

causes causes

Attack Delay


objectobject

causes

Military-Unit1

4

2

3

5

A

B

C D

E

F

G

H

I

Attack Block Delay

Advance

Artillery-Unit

Armor-Unit

agentagent

object

object

object

agent

agent

causes causes

Attack Delay


objectobject

causes

Military-Unit1

4

2

3

5


Attack Block Delay

Advance

Artillery-Unit

Armor-Unit

agentagent

object

object

object

agent

agent

causes causes

Attack Delay


objectobject

causes

Military-Unit

Transformations

Action Action Actioncauses causes

Action Actioncauses

Action Action Actioncauses causesAction Actioncauses


Transformations

Action Action Actioncauses causes

Action Actioncauses

Attack Block Delay

Advance

Artillery-Unit

Armor-Unit

agentagent

object

object

object

agent

agent

causes causes

Attack Delay


objectobject

causes

Military-Unit

causes

Attack Block Delay

Advance

Artillery-Unit

Armor-Unit

agentagent

object

object

object

agent

agent

causes causes

Attack Delay


objectobject

causes

Military-Unit

causes

Initial Evaluation

• Used our matcher in an application in the domain of battle space planning (DARPA's RKF Project).

• The task is to analyze COAs.• Battle space ontology built by extending our upper

ontology.• Two military analysts used this ontology to build

KBs containing: – Patterns. – COAs.

• Our matcher matched the patterns to COAs.

Example Output

Experiment 1

• Evaluates our first hypothesis.– How significant is the improvement?

• Compared our matcher to: – Maximal Common Subgraph (MCS). – Semantic Search Lite (SSL).

• Methodology:– 300 domain-neutral transformations; 80 domain-specific

transformations.– Matched the patterns to the COAs. – A pattern matches a COA if the match score meets or

exceeds a pre-specified threshold. – Used metrics of precision and recall.

Experiment 1: Precision

Experiment 1: Recall

Experiment 2

• Initial evaluation of our second hypothesis.– Assesses the domain independence of using

transformations.

• Limited - conducted in only one domain, but can still offer some insight.

• Methodology: – Divided transformations into 2 groups (domain-neutral

vs. domain-specific). – Used domain-neutral transformations to construct DN – Used domain-specific transformations to construct DS – Everything else is the same as Experiment 1.

Experiment 2: Precision

Experiment 2: Recall

Proposed Work

• More Comprehensive Evaluation.

• Use background knowledge.

• Incorporate indexing to make matching more efficient.

Comprehensive Evaluation

• Evaluate our approach in several applications in four domains.

• Four data sets:– Chemistry (Halo). – Biology (RKF). – Battle Space Planning (RKF). – Office Procedures (EPCA).

• Three Applications:– Elaboration: Chemistry and Office Procedures.– Question Answering: Biology and Battle Space.– Plan Evaluation: Battle Space and Office Procedures.

Background Knowledge

• Background Knowledge.

• Can be used to normalize new knowledge at acquisition time via a join (Mineau et. al. 93).

• Idea can be applied to matching. – Increase similarity.

• Two problems:– When should a join be performed?

– How to better control the join?

Ontology

Block Move

object

object

prevents

Military-Unit

Block

Background Knowledge

• Background Knowledge.

• Can be used to normalize new knowledge at acquisition time via a join (Mineau et. al. 93).

• Idea can be applied to matching. – Increase similarity.

• Two problems:– When should a join be performed?

– How to better control the join?

Block Move

object

object

prevents

Military-Unit

Attack Block

object

Military-Unit

causes

object

Attack Move

object

object

prevents

Military-Unit

Block Move

object

object

prevents

Military-Unit

Attack

object

Military-Unit

Attack

Military-Unitobject

Move

objectMilitary-Unit

Move

objectMilitary-Unit

Indexing

• Need indexing to make matching more efficient.

• A common technique is a generalization hierarchy– Overhead for storage can be expensive.– Finding the MSS can also be expensive.

• We intend to study:– How to index knowledge by content?– Other index structures that are more

parsimonious.

Using Non-Taxonomic Knowledge to Improve Semantic Matching Peter Yeh July 22, 2003.

Documents

Transcript of Using Non-Taxonomic Knowledge to Improve Semantic Matching Peter Yeh July 22, 2003.