17.4 Rule-Based Expert Systems (1/9)ocw.snu.ac.kr/sites/default/files/NOTE/2991.pdf ·...

35
17.4 Rule 17.4 Rule-Based Expert Systems (1/9) Based Expert Systems (1/9) l Expert Systems ¨ One of the most successful applications of AI reasoning technique using facts and rules ¨ “AI Programs that achieve expert-level competence in solving problems by bringing to bear a body of knowledge [Feigenbaum, McCorduck & Nii 1988]” (c) 2000-2002 SNU CSE Biointelligence Lab 1 McCorduck & Nii 1988]” l Expert systems vs. knowledge-based systems l Rule-based expert systems ¨ Often based on reasoning with propositional logic Horn clauses.

Transcript of 17.4 Rule-Based Expert Systems (1/9)ocw.snu.ac.kr/sites/default/files/NOTE/2991.pdf ·...

Page 1: 17.4 Rule-Based Expert Systems (1/9)ocw.snu.ac.kr/sites/default/files/NOTE/2991.pdf · 2018-01-30 · 17.4 Rule-Based Expert Systems (3/9) lKnowledge acquisition subsystem ¨Checks

17.4 Rule17.4 Rule--Based Expert Systems (1/9)Based Expert Systems (1/9)l Expert Systems

¨ One of the most successful applications of AI reasoning technique using facts and rules

¨ “AI Programs that achieve expert-level competence in solving problems by bringing to bear a body of knowledge [Feigenbaum, McCorduck & Nii 1988]”

l Expert systems vs. knowledge-based systemsl Rule-based expert systems

¨ Often based on reasoning with propositional logic Horn clauses.

(c) 2000-2002 SNU CSE Biointelligence Lab 1

l Expert Systems¨ One of the most successful applications of AI reasoning technique

using facts and rules¨ “AI Programs that achieve expert-level competence in solving

problems by bringing to bear a body of knowledge [Feigenbaum, McCorduck & Nii 1988]”

l Expert systems vs. knowledge-based systemsl Rule-based expert systems

¨ Often based on reasoning with propositional logic Horn clauses.

Page 2: 17.4 Rule-Based Expert Systems (1/9)ocw.snu.ac.kr/sites/default/files/NOTE/2991.pdf · 2018-01-30 · 17.4 Rule-Based Expert Systems (3/9) lKnowledge acquisition subsystem ¨Checks

17.4 Rule17.4 Rule--Based Expert Systems (2/9)Based Expert Systems (2/9)

l Structure of Expert Systems¨ Knowledge Base

< Consists of predicate-calculus facts and rules about subject at hand.

¨ Inference Engine< Consists of all the processes that

manipulate the knowledge base to deduce information requested by the user.

¨ Explanation subsystem< Analyzes the structure of the

reasoning performed by the system and explains it to the user.

(c) 2000-2002 SNU CSE Biointelligence Lab 2

l Structure of Expert Systems¨ Knowledge Base

< Consists of predicate-calculus facts and rules about subject at hand.

¨ Inference Engine< Consists of all the processes that

manipulate the knowledge base to deduce information requested by the user.

¨ Explanation subsystem< Analyzes the structure of the

reasoning performed by the system and explains it to the user.

Page 3: 17.4 Rule-Based Expert Systems (1/9)ocw.snu.ac.kr/sites/default/files/NOTE/2991.pdf · 2018-01-30 · 17.4 Rule-Based Expert Systems (3/9) lKnowledge acquisition subsystem ¨Checks

17.4 Rule17.4 Rule--Based Expert Systems (3/9)Based Expert Systems (3/9)

l Knowledge acquisition subsystem¨ Checks the growing knowledge base for possible inconsistencies

and incomplete information.

l User interface¨ Consists of some kind of natural language processing system or

graphical user interfaces with menus.l “Knowledge engineer”

¨ Usually a computer scientist with AI training.¨ Works with an expert in the field of application in order to

represent the relevant knowledge of the expert in a forms of that can be entered into the knowledge base.

(c) 2000-2002 SNU CSE Biointelligence Lab 3

l Knowledge acquisition subsystem¨ Checks the growing knowledge base for possible inconsistencies

and incomplete information.

l User interface¨ Consists of some kind of natural language processing system or

graphical user interfaces with menus.l “Knowledge engineer”

¨ Usually a computer scientist with AI training.¨ Works with an expert in the field of application in order to

represent the relevant knowledge of the expert in a forms of that can be entered into the knowledge base.

Page 4: 17.4 Rule-Based Expert Systems (1/9)ocw.snu.ac.kr/sites/default/files/NOTE/2991.pdf · 2018-01-30 · 17.4 Rule-Based Expert Systems (3/9) lKnowledge acquisition subsystem ¨Checks

17.4 Rule17.4 Rule--Based Expert Systems (4/9)Based Expert Systems (4/9)

sheet.) balanceexcellent an hasapplicant (The BALexpenses.)his/her exceeds income sapplicant' (The INC

rating.)credit good a hasapplicant (TheRATING amount.)loan n thegrater tha

lysufficient is collateral on the appraisal (The APP.)reputation financial good a hasapplicant (The REP

payments.)loan the toable isapplicant (The PYMTry.)satisfacto isloan for the collateral (The COLLAT

approved.) be shouldloan (TheOK

Example: loan officer in a bank“Decide whether or not to grant a personal loan to an individual.”

Facts

(c) 2000-2002 SNU CSE Biointelligence Lab 4

OKREPBAL 5.PYMTINC 4.

REPRATING 3.COLLATAPP 2.

OKREPPYMTCOLLAT 1.

ÉÙÉ

ÉÉ

ÉÙÙ

sheet.) balanceexcellent an hasapplicant (The BALexpenses.)his/her exceeds income sapplicant' (The INC

rating.)credit good a hasapplicant (TheRATING amount.)loan n thegrater tha

lysufficient is collateral on the appraisal (The APP.)reputation financial good a hasapplicant (The REP

payments.)loan the toable isapplicant (The PYMTry.)satisfacto isloan for the collateral (The COLLAT

approved.) be shouldloan (TheOK

Rules

Page 5: 17.4 Rule-Based Expert Systems (1/9)ocw.snu.ac.kr/sites/default/files/NOTE/2991.pdf · 2018-01-30 · 17.4 Rule-Based Expert Systems (3/9) lKnowledge acquisition subsystem ¨Checks

17.4 Rule17.4 Rule--Based Expert Systems (5/9)Based Expert Systems (5/9)

l To prove OK¨ The inference engine searches fro AND/OR proof tree using either

backward or forward chaining.

l AND/OR proof tree¨ Root node: OK¨ Leaf node: facts¨ The root and leaves will be connected through the rules.

l Using the preceding rule in a backward-chaining¨ The user’s goal, to establish OK, can be done either by proving

both BAL and REP or by proving each of COLLAT, PYMT, and REP.

¨ Applying the other rules, as shown, results in other sets of nodes to be proved.

(c) 2000-2002 SNU CSE Biointelligence Lab 5

l To prove OK¨ The inference engine searches fro AND/OR proof tree using either

backward or forward chaining.

l AND/OR proof tree¨ Root node: OK¨ Leaf node: facts¨ The root and leaves will be connected through the rules.

l Using the preceding rule in a backward-chaining¨ The user’s goal, to establish OK, can be done either by proving

both BAL and REP or by proving each of COLLAT, PYMT, and REP.

¨ Applying the other rules, as shown, results in other sets of nodes to be proved.

Page 6: 17.4 Rule-Based Expert Systems (1/9)ocw.snu.ac.kr/sites/default/files/NOTE/2991.pdf · 2018-01-30 · 17.4 Rule-Based Expert Systems (3/9) lKnowledge acquisition subsystem ¨Checks

17.4 Rule17.4 Rule--Based Expert Systems (6/9)Based Expert Systems (6/9)

l By backward-chaining

(c) 2000-2002 SNU CSE Biointelligence Lab 6

Page 7: 17.4 Rule-Based Expert Systems (1/9)ocw.snu.ac.kr/sites/default/files/NOTE/2991.pdf · 2018-01-30 · 17.4 Rule-Based Expert Systems (3/9) lKnowledge acquisition subsystem ¨Checks

17.4 Rule17.4 Rule--Based Expert Systems (7/9)Based Expert Systems (7/9)

l Consulting system¨ Attempt to answer a user’s query by asking questions about the

truth of propositions that they might know about.¨ Backward-chaining through the rule is used to get to askable

questions.¨ If a user were to “volunteer” information, bottom-up, forward

chaining through the rules could be used in an attempt to connect to the proof tree already built.

¨ The ability to give explanations for a conclusion< Very important for acceptance of expert system advice.

¨ Proof tree< Used to guide the explanation-generation process.

(c) 2000-2002 SNU CSE Biointelligence Lab 7

l Consulting system¨ Attempt to answer a user’s query by asking questions about the

truth of propositions that they might know about.¨ Backward-chaining through the rule is used to get to askable

questions.¨ If a user were to “volunteer” information, bottom-up, forward

chaining through the rules could be used in an attempt to connect to the proof tree already built.

¨ The ability to give explanations for a conclusion< Very important for acceptance of expert system advice.

¨ Proof tree< Used to guide the explanation-generation process.

Page 8: 17.4 Rule-Based Expert Systems (1/9)ocw.snu.ac.kr/sites/default/files/NOTE/2991.pdf · 2018-01-30 · 17.4 Rule-Based Expert Systems (3/9) lKnowledge acquisition subsystem ¨Checks

17.4 Rule17.4 Rule--Based Expert Systems (8/9)Based Expert Systems (8/9)l In many applications, the system has access only to

uncertain rules, and the user not be able to answer questions with certainty.

l MYCIN [Shortliffe 1976]: Diagnose bacterial infections.

(.5). a-group-cusstreptococ ; (.75) pos-coag-ccusstaphylocoisinfection thecausing bemight which smears)or cultureson seen an those(orther th organism that theevidence is There :Then

bacterial isinfection theof typeThe 4)and culture, theofstain on theseen not wereOrganisms 3)

and infection,esoft tissuor skin serious of evidence have doespatient The 2)

and ,meningitis is therapy requireswhich infection The 1) : If300 Rule

(c) 2000-2002 SNU CSE Biointelligence Lab 8

(.5). a-group-cusstreptococ ; (.75) pos-coag-ccusstaphylocoisinfection thecausing bemight which smears)or cultureson seen an those(orther th organism that theevidence is There :Then

bacterial isinfection theof typeThe 4)and culture, theofstain on theseen not wereOrganisms 3)

and infection,esoft tissuor skin serious of evidence have doespatient The 2)

and ,meningitis is therapy requireswhich infection The 1) : If300 Rule

Page 9: 17.4 Rule-Based Expert Systems (1/9)ocw.snu.ac.kr/sites/default/files/NOTE/2991.pdf · 2018-01-30 · 17.4 Rule-Based Expert Systems (3/9) lKnowledge acquisition subsystem ¨Checks

17.4 Rule17.4 Rule--Based Expert Systems (9/9)Based Expert Systems (9/9)

¨ PROSPECTOR [Duda, Gaschnig & Hart 1979, Campbell, et al. 1982]<Reason about ore deposits.

¨ The numbers (.75 and .5 in MYCIN, and 5, 0.7 in PROSPECTOR) are ways to represent the certainty or strength of a rule.

¨ The numbers are used by these systems in computing the certainty of conclusions.

deposit.copperporphyry afor favorablet environmen regional a 0.7) (5, is

e then thersystem,fault going- thoroughintrusive,-pre a is thereIf

(c) 2000-2002 SNU CSE Biointelligence Lab 9

¨ PROSPECTOR [Duda, Gaschnig & Hart 1979, Campbell, et al. 1982]<Reason about ore deposits.

¨ The numbers (.75 and .5 in MYCIN, and 5, 0.7 in PROSPECTOR) are ways to represent the certainty or strength of a rule.

¨ The numbers are used by these systems in computing the certainty of conclusions.

deposit.copperporphyry afor favorablet environmen regional a 0.7) (5, is

e then thersystem,fault going- thoroughintrusive,-pre a is thereIf

Page 10: 17.4 Rule-Based Expert Systems (1/9)ocw.snu.ac.kr/sites/default/files/NOTE/2991.pdf · 2018-01-30 · 17.4 Rule-Based Expert Systems (3/9) lKnowledge acquisition subsystem ¨Checks

17.5 Rule Learning17.5 Rule Learning

l Inductive rule learning¨Creates new rules about a domain, not derivable from

any previous rules.¨ Ex) Neural networks

l Deductive rule learning¨ Enhances the efficiency of a system’s performance by

deducting additional rules from previously known domain rules and facts.

¨ Ex) EBG (explanation-based generalization)

(c) 2000-2002 SNU CSE Biointelligence Lab 10

l Inductive rule learning¨Creates new rules about a domain, not derivable from

any previous rules.¨ Ex) Neural networks

l Deductive rule learning¨ Enhances the efficiency of a system’s performance by

deducting additional rules from previously known domain rules and facts.

¨ Ex) EBG (explanation-based generalization)

Page 11: 17.4 Rule-Based Expert Systems (1/9)ocw.snu.ac.kr/sites/default/files/NOTE/2991.pdf · 2018-01-30 · 17.4 Rule-Based Expert Systems (3/9) lKnowledge acquisition subsystem ¨Checks

17.5.1 Learning Propositional Calculus 17.5.1 Learning Propositional Calculus Rules (1/9)Rules (1/9)

l Train rules from given training set¨ Seek a set of rules that covers only positive instances

< Positive instance: OK = 1< Negative instance: OK = 0

¨ From training set, we desire to induce rules of the form

¨ We can make some rule more specific by adding an atom to its antecedent to make it cover fewer instances.< Cover: If the antecedent of a rule has value True for an instance in the

training set, we say that the rule covers that instance.¨ Adding a rule makes the system using these rules more general.¨ Searching for a set of rules can be computationally difficult.

< èhere, we use “greedy” method which is called separate and conquer.

(c) 2000-2002 SNU CSE Biointelligence Lab 11

l Train rules from given training set¨ Seek a set of rules that covers only positive instances

< Positive instance: OK = 1< Negative instance: OK = 0

¨ From training set, we desire to induce rules of the form

¨ We can make some rule more specific by adding an atom to its antecedent to make it cover fewer instances.< Cover: If the antecedent of a rule has value True for an instance in the

training set, we say that the rule covers that instance.¨ Adding a rule makes the system using these rules more general.¨ Searching for a set of rules can be computationally difficult.

< èhere, we use “greedy” method which is called separate and conquer.

OKn ÉÙÙ aaa L21 }BAL INC, RATING, APP,{ where Îia

Page 12: 17.4 Rule-Based Expert Systems (1/9)ocw.snu.ac.kr/sites/default/files/NOTE/2991.pdf · 2018-01-30 · 17.4 Rule-Based Expert Systems (3/9) lKnowledge acquisition subsystem ¨Checks

l Separate and conquer¨ First attempt to find a single rule that covers only

positive instances<Start with a rule that covers all instances<Gradually make it more specific by adding atoms to its

antecedent.

¨Gradually add rules until the entire set of rules covers all and only the positive instances.

¨ Trained rules can be simplified using pruning.<Operations and noise-tolerant modifications help minimize the

risk of overfitting.

17.5.1 Learning Propositional Calculus 17.5.1 Learning Propositional Calculus Rules (2/9)Rules (2/9)

(c) 2000-2002 SNU CSE Biointelligence Lab 12

l Separate and conquer¨ First attempt to find a single rule that covers only

positive instances<Start with a rule that covers all instances<Gradually make it more specific by adding atoms to its

antecedent.

¨Gradually add rules until the entire set of rules covers all and only the positive instances.

¨ Trained rules can be simplified using pruning.<Operations and noise-tolerant modifications help minimize the

risk of overfitting.

Page 13: 17.4 Rule-Based Expert Systems (1/9)ocw.snu.ac.kr/sites/default/files/NOTE/2991.pdf · 2018-01-30 · 17.4 Rule-Based Expert Systems (3/9) lKnowledge acquisition subsystem ¨Checks

l Example: loan officer in a bank¨ Start with the provisional rule .

<Which cover all instances.

¨Add an atom it cover fewer negative instances-working toward covering only positive ones.

¨Decide, which item should we added ?<From by

17.5.1 Learning Propositional Calculus 17.5.1 Learning Propositional Calculus Rules (3/9)Rules (3/9)

OKT É

(c) 2000-2002 SNU CSE Biointelligence Lab 13

l Example: loan officer in a bank¨ Start with the provisional rule .

<Which cover all instances.

¨Add an atom it cover fewer negative instances-working toward covering only positive ones.

¨Decide, which item should we added ?<From by}BAL INC, RATING, APP,{

aa annr /+=

.antecedent the to ofaddition after the rule theof antecedent the

by covered instance positive ofnumber totoalthe:.antecedent the to

ofaddition after the rule theof antecedent by the covered instance ofnumber totoalthe:

a

a

a

a

+n

n

Page 14: 17.4 Rule-Based Expert Systems (1/9)ocw.snu.ac.kr/sites/default/files/NOTE/2991.pdf · 2018-01-30 · 17.4 Rule-Based Expert Systems (3/9) lKnowledge acquisition subsystem ¨Checks

l Select that yielding the largest value of .

17.5.1 Learning Propositional Calculus 17.5.1 Learning Propositional Calculus Rules (4/9)Rules (4/9)

a ar

75.04/35.06/3

667.06/45.06/3

======

==

BAL

INC

RATING

APP

rrrr

OKBALÉ

So, we select BAL, yielding the provisional rule.

(c) 2000-2002 SNU CSE Biointelligence Lab 14

75.04/35.06/3

667.06/45.06/3

======

==

BAL

INC

RATING

APP

rrrr

Page 15: 17.4 Rule-Based Expert Systems (1/9)ocw.snu.ac.kr/sites/default/files/NOTE/2991.pdf · 2018-01-30 · 17.4 Rule-Based Expert Systems (3/9) lKnowledge acquisition subsystem ¨Checks

17.5.1 Learning Propositional Calculus 17.5.1 Learning Propositional Calculus Rules (5/9)Rules (5/9)l Rule covers the positive instances 3,4, and 7,

but also covers the negative instance 1.¨ So, select another atom to make this rule more specific.

l We have already decided that the first component in the antecedent is BAL, so we have to consider it.

OKBALÉ

(c) 2000-2002 SNU CSE Biointelligence Lab 15

l Rule covers the positive instances 3,4, and 7, but also covers the negative instance 1.¨ So, select another atom to make this rule more specific.

l We have already decided that the first component in the antecedent is BAL, so we have to consider it.

0.12/20.13/3

667.03/2

====

==

INC

RATING

APP

rrr

OKRATINGBAL ÉÙWe select RATING because is based on a larger sample.RATINGr

Page 16: 17.4 Rule-Based Expert Systems (1/9)ocw.snu.ac.kr/sites/default/files/NOTE/2991.pdf · 2018-01-30 · 17.4 Rule-Based Expert Systems (3/9) lKnowledge acquisition subsystem ¨Checks

17.5.1 Learning Propositional Calculus 17.5.1 Learning Propositional Calculus Rules (6/9)Rules (6/9)

We need more rules which cover positive instance 6. To learn the next rule, eliminate from the table all of the positive instances already covered by the first rule.

(c) 2000-2002 SNU CSE Biointelligence Lab 16

Page 17: 17.4 Rule-Based Expert Systems (1/9)ocw.snu.ac.kr/sites/default/files/NOTE/2991.pdf · 2018-01-30 · 17.4 Rule-Based Expert Systems (3/9) lKnowledge acquisition subsystem ¨Checks

l Begin the process all over again with reduced table¨ Start with the rule .

¨ Finally we get which covers only positive instances with first rule, so we are finished.

17.5.1 Learning Propositional Calculus 17.5.1 Learning Propositional Calculus Rules (7/9)Rules (7/9)

OKT É

0.01/025.04/1

0.03/025.04/1

======

==

BAL

INC

RATING

APP

rrrr APP=INC=0.25, arbitrarily select APP.

OKAPP É

(c) 2000-2002 SNU CSE Biointelligence Lab 17

l Begin the process all over again with reduced table¨ Start with the rule .

¨ Finally we get which covers only positive instances with first rule, so we are finished.

0.01/025.04/1

0.03/025.04/1

======

==

BAL

INC

RATING

APP

rrrr

This rule covers negative instances 1, 8, and 9 è we need another atom to the antecedent.

0.01/05.02/1

5.02/1

======

BAL

INC

RATING

rrr Select RATING, and we get

OKRATINGAPP ÉÙThis rule covers negative example 9.

OKINCRATINGAPP ÉÙÙ

Page 18: 17.4 Rule-Based Expert Systems (1/9)ocw.snu.ac.kr/sites/default/files/NOTE/2991.pdf · 2018-01-30 · 17.4 Rule-Based Expert Systems (3/9) lKnowledge acquisition subsystem ¨Checks

l Pseudocode of this rule learning process.¨Generic Separate-and-conquer algorithm (GSCA)

17.5.1 Learning Propositional Calculus 17.5.1 Learning Propositional Calculus Rules (8/9)Rules (8/9)

(c) 2000-2002 SNU CSE Biointelligence Lab 18

Page 19: 17.4 Rule-Based Expert Systems (1/9)ocw.snu.ac.kr/sites/default/files/NOTE/2991.pdf · 2018-01-30 · 17.4 Rule-Based Expert Systems (3/9) lKnowledge acquisition subsystem ¨Checks

17.5.1 Learning Propositional Calculus 17.5.1 Learning Propositional Calculus Rules (9/9)Rules (9/9)

(c) 2000-2002 SNU CSE Biointelligence Lab 19

Page 20: 17.4 Rule-Based Expert Systems (1/9)ocw.snu.ac.kr/sites/default/files/NOTE/2991.pdf · 2018-01-30 · 17.4 Rule-Based Expert Systems (3/9) lKnowledge acquisition subsystem ¨Checks

17.5.2 Learning First17.5.2 Learning First--Order Logic Order Logic Rules (1/10)Rules (1/10)l Inductive logic programming (ILP)

¨ Concentrate on methods for inductive learning of Horn clauses in first order predicate calculus (FOPC) and thus PROLOG program.

¨ FOIL [Quinlan, 1990]

l The Objective of ILP¨ To learn a program, ,consisting of Horn clauses, ,each of which is

of the form where, the are atomic formulas that unify with ground atomic facts.< : should evaluate to True when its variables are bound to

some set of values known to be in the relation we are trying to learn (positive instance: training set).

< : should evaluate to False when its variables are bound to some set of values known not to be in the relation (negative instance).

(c) 2000-2002 SNU CSE Biointelligence Lab 20

l Inductive logic programming (ILP)¨ Concentrate on methods for inductive learning of Horn clauses in

first order predicate calculus (FOPC) and thus PROLOG program.¨ FOIL [Quinlan, 1990]

l The Objective of ILP¨ To learn a program, ,consisting of Horn clauses, ,each of which is

of the form where, the are atomic formulas that unify with ground atomic facts.< : should evaluate to True when its variables are bound to

some set of values known to be in the relation we are trying to learn (positive instance: training set).

< : should evaluate to False when its variables are bound to some set of values known not to be in the relation (negative instance).

p r

iaaar ,,, : 21 L- ia

+X

-X

p

p

Page 21: 17.4 Rule-Based Expert Systems (1/9)ocw.snu.ac.kr/sites/default/files/NOTE/2991.pdf · 2018-01-30 · 17.4 Rule-Based Expert Systems (3/9) lKnowledge acquisition subsystem ¨Checks

l We want to cover the positives instances and not cover negative ones.

l Background knowledge¨ The ground atomic facts with which the are to unify.¨ They are given-as either subsidiary PROLOG programs,

which can be run and evaluated, or explicitly in the form of a list of facts.

l Example: A delivery robot navigating around in a building finds through experience, that it is easy to go between certain pairs of locations and not so easy to go between certain other pairs.

17.5.2 Learning First17.5.2 Learning First--Order Logic Order Logic Rules (2/10)Rules (2/10)

p

a

(c) 2000-2002 SNU CSE Biointelligence Lab 21

l We want to cover the positives instances and not cover negative ones.

l Background knowledge¨ The ground atomic facts with which the are to unify.¨ They are given-as either subsidiary PROLOG programs,

which can be run and evaluated, or explicitly in the form of a list of facts.

l Example: A delivery robot navigating around in a building finds through experience, that it is easy to go between certain pairs of locations and not so easy to go between certain other pairs.

Page 22: 17.4 Rule-Based Expert Systems (1/9)ocw.snu.ac.kr/sites/default/files/NOTE/2991.pdf · 2018-01-30 · 17.4 Rule-Based Expert Systems (3/9) lKnowledge acquisition subsystem ¨Checks

l A, B, C: junctionsl All of the other locations:

shopsl Junction(x)

¨Whether junction or not.l Shop(x,y)

¨Whether shop or not which is connected to junction x.

17.5.2 Learning First17.5.2 Learning First--Order Logic Order Logic Rules (3/10)Rules (3/10)

(c) 2000-2002 SNU CSE Biointelligence Lab 22

l A, B, C: junctionsl All of the other locations:

shopsl Junction(x)

¨Whether junction or not.l Shop(x,y)

¨Whether shop or not which is connected to junction x.

Page 23: 17.4 Rule-Based Expert Systems (1/9)ocw.snu.ac.kr/sites/default/files/NOTE/2991.pdf · 2018-01-30 · 17.4 Rule-Based Expert Systems (3/9) lKnowledge acquisition subsystem ¨Checks

17.5.2 Learning First17.5.2 Learning First--Order Logic Order Logic Rules (4/10)Rules (4/10)

}CC2,,CC1,,C2C,,C1C,,BB2,,B1B,,B2B,,B1B,,AA2,,AA1,,A2A,,A1A,

,BC, ,AC, ,AB, ,CB, ,CA, ,BA,{Ξ :Easy of instances Positive

><><><><><><><><><><><><

><><><><><><

+

¨ We want a learning program to learn a program, Easy(x,y) that covers the positive instances in but not the negative ones.

¨ Easy(x,y) can use the background subexpressions Junction(x) and Shop(x,y).

¨ Training set

X

(c) 2000-2002 SNU CSE Biointelligence Lab 23

}CC2,,CC1,,C2C,,C1C,,BB2,,B1B,,B2B,,B1B,,AA2,,AA1,,A2A,,A1A,

,BC, ,AC, ,AB, ,CB, ,CA, ,BA,{Ξ :Easy of instances Positive

><><><><><><><><><><><><

><><><><><><

+

}CB2,,CB1,,CA2,,CA1,,BA2,,BA1,,BC2,,BC1,,AC2,,AC1,,AB2,,AB1,

,B2C,,B1C,,A2C,,A1C,,A2B,,A1B,,C2B, ,C1B, ,C2A, ,C1A, ,B2A, ,B1A,{

Ξ :Easy of instances Negative

><><><><><><><><><><><><><><><><><><><><><><><><

-

Page 24: 17.4 Rule-Based Expert Systems (1/9)ocw.snu.ac.kr/sites/default/files/NOTE/2991.pdf · 2018-01-30 · 17.4 Rule-Based Expert Systems (3/9) lKnowledge acquisition subsystem ¨Checks

¨ For all of the locations named in , only the following pairs give a value True for Shop:

¨ The following PROLOG program covers all of the positiveinstances of the training set and none of the negative ones

Easy(x, y) :- Junction(x), Junction(y)

:- Shop(x, y)

:- Shop(y, x)

17.5.2 Learning First17.5.2 Learning First--Order Logic Order Logic Rules (5/10)Rules (5/10)

X

}CC2,,CC1,,BB2,,BB1,,AA2,,AA1,{ ><><><><><><

(c) 2000-2002 SNU CSE Biointelligence Lab 24

¨ For all of the locations named in , only the following pairs give a value True for Shop:

¨ The following PROLOG program covers all of the positiveinstances of the training set and none of the negative ones

Easy(x, y) :- Junction(x), Junction(y)

:- Shop(x, y)

:- Shop(y, x)

Page 25: 17.4 Rule-Based Expert Systems (1/9)ocw.snu.ac.kr/sites/default/files/NOTE/2991.pdf · 2018-01-30 · 17.4 Rule-Based Expert Systems (3/9) lKnowledge acquisition subsystem ¨Checks

l Learning process: generalized separate and conquer algorithm (GSCA)¨ Start with a program having a single rule with no body¨Add literals to the body until the rule covers only (or

mainly) positive instances¨Add rules in the same way until the program covers all (or

most) and only (with few exceptions) positive instances.

17.5.2 Learning First17.5.2 Learning First--Order Logic Order Logic Rules (6/10)Rules (6/10)

(c) 2000-2002 SNU CSE Biointelligence Lab 25

l Learning process: generalized separate and conquer algorithm (GSCA)¨ Start with a program having a single rule with no body¨Add literals to the body until the rule covers only (or

mainly) positive instances¨Add rules in the same way until the program covers all (or

most) and only (with few exceptions) positive instances.

Page 26: 17.4 Rule-Based Expert Systems (1/9)ocw.snu.ac.kr/sites/default/files/NOTE/2991.pdf · 2018-01-30 · 17.4 Rule-Based Expert Systems (3/9) lKnowledge acquisition subsystem ¨Checks

¨ Practical ILP systems restrict the literals in various ways.

¨ Typical allowed additions are<Literals used in the background knowledge<Literals whose arguments are a subset of those in the head of

the clause.<Literals that introduce a new distinct variable different from

those in the head of the clause.<A literal that equates a variable in the head of the clause with

another such variable or with a term mentioned in the background knowledge.

<A literal that is the same (except for its arguments) as that in the head of the clause.

17.5.2 Learning First17.5.2 Learning First--Order Logic Order Logic Rules (7/10)Rules (7/10)

(c) 2000-2002 SNU CSE Biointelligence Lab 26

¨ Practical ILP systems restrict the literals in various ways.

¨ Typical allowed additions are<Literals used in the background knowledge<Literals whose arguments are a subset of those in the head of

the clause.<Literals that introduce a new distinct variable different from

those in the head of the clause.<A literal that equates a variable in the head of the clause with

another such variable or with a term mentioned in the background knowledge.

<A literal that is the same (except for its arguments) as that in the head of the clause.

Page 27: 17.4 Rule-Based Expert Systems (1/9)ocw.snu.ac.kr/sites/default/files/NOTE/2991.pdf · 2018-01-30 · 17.4 Rule-Based Expert Systems (3/9) lKnowledge acquisition subsystem ¨Checks

17.5.2 Learning First17.5.2 Learning First--Order Logic Order Logic Rules (8/10)Rules (8/10)

l The literals that we might consider adding to a clause are

l ILP version of GSCA¨ First, initialize first clause as Easy(x, y) :-¨ Add Junction(x), so Easy(x, y) :- Junction(x) covers the following

instances

¨ Include more literal ‘Junction(y)’ è Easy(x, y) :- Junction(x), Junction(y)

Junction(x), Junction(y), Junction(z)Shop(x,y), Shop(y,x), Shop(x,z)Shop(z,y), (x=y)

(c) 2000-2002 SNU CSE Biointelligence Lab 27

l The literals that we might consider adding to a clause are

l ILP version of GSCA¨ First, initialize first clause as Easy(x, y) :-¨ Add Junction(x), so Easy(x, y) :- Junction(x) covers the following

instances

¨ Include more literal ‘Junction(y)’ è Easy(x, y) :- Junction(x), Junction(y)

}C2C,,C1C,,B2B,,B1B,,A2A,,A1A,,BC, ,AC, ,AB, ,CB, ,CA, ,BA,{

><><><><><><><><><><><><

}C2B,,C1B,,A2B,,A1B,,B2C,,B1C,,A2C, ,A1C, ,C2A, ,C1A, ,B2A, ,B1A,{

><><><><><><><><><><><><

çPositive instances

çNegative instances

}BC, ,AC, ,AB, ,CB, ,CA, ,BA,{ ><><><><><><

Page 28: 17.4 Rule-Based Expert Systems (1/9)ocw.snu.ac.kr/sites/default/files/NOTE/2991.pdf · 2018-01-30 · 17.4 Rule-Based Expert Systems (3/9) lKnowledge acquisition subsystem ¨Checks

¨But program does not cover the following positive instances.

¨Remove the positive instance covered by Easy(x,y):-Junction(x), Junction(y) from to form the to be used in next pass through inner loop.< : all negative instance in + the positive instance that are not

covered yet.

¨ Inner loop create another initial clause “Easy(x,y) :-”<Add literal Shop(x,y) : Easy(x,y) :- Shop(x,y) è cover no negative

instances, so we are finished with another pass through inner loop.<Covered positive instance by this rule ( remove this from )

17.5.2 Learning First17.5.2 Learning First--Order Logic Order Logic Rules (9/10)Rules (9/10)

p

}CC2, ,CC1, ,C2C, ,C1C, ,BB2, ,BB1,,B2B,,B1B,,AA2,,AA1,,A2A,,A1A,{

><><><><><><><><><><><><

X CURX

(c) 2000-2002 SNU CSE Biointelligence Lab 28

¨But program does not cover the following positive instances.

¨Remove the positive instance covered by Easy(x,y):-Junction(x), Junction(y) from to form the to be used in next pass through inner loop.< : all negative instance in + the positive instance that are not

covered yet.

¨ Inner loop create another initial clause “Easy(x,y) :-”<Add literal Shop(x,y) : Easy(x,y) :- Shop(x,y) è cover no negative

instances, so we are finished with another pass through inner loop.<Covered positive instance by this rule ( remove this from )

X CURX

XCURX

}CC2, ,CC1,,BB2, ,BB1,,AA2,,AA1,{ ><><><><><><CURX

Page 29: 17.4 Rule-Based Expert Systems (1/9)ocw.snu.ac.kr/sites/default/files/NOTE/2991.pdf · 2018-01-30 · 17.4 Rule-Based Expert Systems (3/9) lKnowledge acquisition subsystem ¨Checks

¨Now we have Easy(x,y) :- Junction(x), Junction(y):- Shop(x, y)

¨ To cover following instance

¨Add Shop(y, x)¨ Then we have Easy(x,y) :- Junction(x), Junction(y)

:- Shop(x, y):- Shop(y, x)

¨ This cover only positive instances.

17.5.2 Learning First17.5.2 Learning First--Order Logic Order Logic Rules (10/10)Rules (10/10)

}C2C, ,C1C,,B2B,,B1B,,A2A,,A1A,{ ><><><><><><

(c) 2000-2002 SNU CSE Biointelligence Lab 29

¨Now we have Easy(x,y) :- Junction(x), Junction(y):- Shop(x, y)

¨ To cover following instance

¨Add Shop(y, x)¨ Then we have Easy(x,y) :- Junction(x), Junction(y)

:- Shop(x, y):- Shop(y, x)

¨ This cover only positive instances.

Page 30: 17.4 Rule-Based Expert Systems (1/9)ocw.snu.ac.kr/sites/default/files/NOTE/2991.pdf · 2018-01-30 · 17.4 Rule-Based Expert Systems (3/9) lKnowledge acquisition subsystem ¨Checks

17.5.3 Explanation17.5.3 Explanation--Based Based Generalization (1/2)Generalization (1/2)

l Example: “Block world”¨General knowledge of the “Block world”.

<Rules

<Fact

<We want to proof“ ”

<Proof is very simple è

(c) 2000-2002 SNU CSE Biointelligence Lab 30

l Example: “Block world”¨General knowledge of the “Block world”.

<Rules

<Fact

<We want to proof“ ”

<Proof is very simple è

Page 31: 17.4 Rule-Based Expert Systems (1/9)ocw.snu.ac.kr/sites/default/files/NOTE/2991.pdf · 2018-01-30 · 17.4 Rule-Based Expert Systems (3/9) lKnowledge acquisition subsystem ¨Checks

17.5.3 Explanation17.5.3 Explanation--Based Based Generalization (2/2)Generalization (2/2)

l Explanation: Set of facts used in the proof¨ Ex) explanation for “ ” is “ ”¨ From this explanation, we can make “ ”

<Replacing constant ‘A’ by variable ‘x’, then, we have “Green(x)”<Then we can proof “ ”, as like the case of “ ”<èExplanation Based Generalization

l Explanation-based generalization (EBG): Generalizing the explanation by replacing constant by variable¨More rules might slow down the reasoning process, so EBG

must be used with care-possibly by keeping information about the utility of the learned rules.

(c) 2000-2002 SNU CSE Biointelligence Lab 31

l Explanation: Set of facts used in the proof¨ Ex) explanation for “ ” is “ ”¨ From this explanation, we can make “ ”

<Replacing constant ‘A’ by variable ‘x’, then, we have “Green(x)”<Then we can proof “ ”, as like the case of “ ”<èExplanation Based Generalization

l Explanation-based generalization (EBG): Generalizing the explanation by replacing constant by variable¨More rules might slow down the reasoning process, so EBG

must be used with care-possibly by keeping information about the utility of the learned rules.

Page 32: 17.4 Rule-Based Expert Systems (1/9)ocw.snu.ac.kr/sites/default/files/NOTE/2991.pdf · 2018-01-30 · 17.4 Rule-Based Expert Systems (3/9) lKnowledge acquisition subsystem ¨Checks

Additional Readings (1/4)Additional Readings (1/4)

l [Levesque & Brachman 1987]¨Balance between logical expression and logical inference

l [Ullman 1989]¨DATALOG

l [Selman & Kautz 1991]¨Approximate theory: Horn greatest-lower-bound, Horn

least-upper-boundl [Kautz, Kearns, & Selman 1993]

¨Characteristic model

(c) 2000-2002 SNU CSE Biointelligence Lab 32

l [Levesque & Brachman 1987]¨Balance between logical expression and logical inference

l [Ullman 1989]¨DATALOG

l [Selman & Kautz 1991]¨Approximate theory: Horn greatest-lower-bound, Horn

least-upper-boundl [Kautz, Kearns, & Selman 1993]

¨Characteristic model

Page 33: 17.4 Rule-Based Expert Systems (1/9)ocw.snu.ac.kr/sites/default/files/NOTE/2991.pdf · 2018-01-30 · 17.4 Rule-Based Expert Systems (3/9) lKnowledge acquisition subsystem ¨Checks

Additional Readings (2/4)Additional Readings (2/4)

l [Roussel 1975, Colmerauer 1973]¨ PROLOG interpreter

l [Warren, Pereira, & Pereira 1977]¨Development of efficient interpreter

l [Davis 1980]¨AO* algorithm searching AND/OR graphs

l [Selman & Levesque 1990]¨Determination of minimum ATMS label: NP-complete

problem

(c) 2000-2002 SNU CSE Biointelligence Lab 33

l [Roussel 1975, Colmerauer 1973]¨ PROLOG interpreter

l [Warren, Pereira, & Pereira 1977]¨Development of efficient interpreter

l [Davis 1980]¨AO* algorithm searching AND/OR graphs

l [Selman & Levesque 1990]¨Determination of minimum ATMS label: NP-complete

problem

Page 34: 17.4 Rule-Based Expert Systems (1/9)ocw.snu.ac.kr/sites/default/files/NOTE/2991.pdf · 2018-01-30 · 17.4 Rule-Based Expert Systems (3/9) lKnowledge acquisition subsystem ¨Checks

Additional Readings (3/4)Additional Readings (3/4)

l [Kautz, Kearns & Selman 1993]¨ TMS calculation based on characteristic model

l [Doyle 1979, de Kleer 1986a, de Kleer 1986b, de Kleer 1986c, Forbus & de Kleer 1993, Shoham 1994]¨Other results for TMS

l [Bobrow, Mittal & Stefik 1986], [Stefik 1995]¨Construction of expert system

(c) 2000-2002 SNU CSE Biointelligence Lab 34

l [Kautz, Kearns & Selman 1993]¨ TMS calculation based on characteristic model

l [Doyle 1979, de Kleer 1986a, de Kleer 1986b, de Kleer 1986c, Forbus & de Kleer 1993, Shoham 1994]¨Other results for TMS

l [Bobrow, Mittal & Stefik 1986], [Stefik 1995]¨Construction of expert system

Page 35: 17.4 Rule-Based Expert Systems (1/9)ocw.snu.ac.kr/sites/default/files/NOTE/2991.pdf · 2018-01-30 · 17.4 Rule-Based Expert Systems (3/9) lKnowledge acquisition subsystem ¨Checks

Additional Readings (4/4)Additional Readings (4/4)

l [McDermott 19982]¨ Examples of expert systems

l [Leonard-Barton 1987]¨History and usage of DEC’s expert system

l [Kautz & Selman 1992], [Muggleton & Buntine 1988]¨ Predicate finding

l [Muggleton, King & Sternberg 1992]¨ Protein secondary structure prediction by GOLEM

(c) 2000-2002 SNU CSE Biointelligence Lab 35

l [McDermott 19982]¨ Examples of expert systems

l [Leonard-Barton 1987]¨History and usage of DEC’s expert system

l [Kautz & Selman 1992], [Muggleton & Buntine 1988]¨ Predicate finding

l [Muggleton, King & Sternberg 1992]¨ Protein secondary structure prediction by GOLEM