© 2002-9 Franz J. Kurfess ES Implementation 1 CPE/CSC 481: Knowledge-Based Systems Dr. Franz J....

© 2002-9 Franz J. Kurfess ES Implementation 1

CPE/CSC 481: Knowledge-Based Systems

CPE/CSC 481: Knowledge-Based Systems

Dr. Franz J. Kurfess

Computer Science Department

Cal Poly


Usage of the SlidesUsage of the Slides these slides are intended for the students of my

CPE/CSC 481 “Knowledge-Based Systems” class at Cal Poly SLO if you want to use them outside of my class, please let me know

([email protected]) I usually put together a subset for each quarter as a

“Custom Show” to view these, go to “Slide Show => Custom Shows”, select the

respective quarter, and click on “Show” To print them, I suggest to use the “Handout” option

4, 6, or 9 per page works fine Black & White should be fine; there are few diagrams where

color is important


Course OverviewCourse Overview Introduction Knowledge Representation

Semantic Nets, Frames, Logic

Reasoning and Inference Predicate Logic, Inference

Methods, Resolution

Reasoning with Uncertainty Probability, Bayesian Decision

Making

Expert System Design ES Life Cycle

CLIPS Overview Concepts, Notation, Usage

Pattern Matching Variables, Functions,

Expressions, Constraints

Expert System Implementation Salience, Rete Algorithm

Expert System Examples Conclusions and Outlook


Overview Implementation of Rule-Based Systems


Motivation Objectives Chapter Introduction

Important Concepts Performance Aspects

Pattern Matching Basic Idea Unification Pattern Matching in Rule-

Based Systems

Rete Algorithm Overview Rete Network Assert and Retract Optimizations Improvements

Rule Formulation General vs. Specific Rules Simple vs. Complex Rules Loading and Saving Facts

Important Concepts and Terms

Chapter Summary


LogisticsLogistics Introductions Course Materials

textbooks (see below) lecture notes

PowerPoint Slides will be available on my Web page handouts Web page

http://www.csc.calpoly.edu/~fkurfess

Term Project Lab and Homework Assignments Exams Grading


Bridge-InBridge-In


Pre-TestPre-Test


MotivationMotivation

pattern matching and unification are powerful operations to determine the similarity and consistency of complex structures they are at the core of many rule-based and predicate logic

mechanisms their application goes beyond rule-based systems

study concepts and methods that are critical for the functionality and performance of rule-based systems pattern matching and the Rete algorithm use and formulation of rules


ObjectivesObjectives comprehend the mechanics of pattern matching in rule-based

systems basic concepts and techniques Rete algorithm

understand the effects of matching and rule formulation on the performance of rule-based systems

learn to write rule-based programs and implement systems in an efficient way

analyze and evaluate the performance of rule-based programs and systems identify bottlenecks formulate and implement strategies for performance improvements




due to their more declarative nature, it can be difficult to evaluate and predict the performance of rule-based systems time to complete a task memory usage disk space usage

pattern matching can be used to eliminate unsuitable rules and facts but it can also introduce substantial overhead


Chapter IntroductionChapter Introduction

Important Concepts entities with internal structure

data structures, objects, components terms, sentences, graphs diagrams, images concepts, hierarchies

Performance Aspects somewhat different from conventional programs

less control over the runtime behavior pattern matching can do a lot of the work


Pattern MatchingPattern Matching determines if two or more complex entities (patterns)

are compatible with each other patterns can be (almost) anything that has a structure

pictures: mugshot vs. person drawings: diagrams of systems expressions: words,sentences of a language, strings

graphs are often used as the underlying representation the structure of the graphs must be compatible

usually either identical, or one is a sub-graph of the other the individual parts must be compatible

nodes must have identical or compatible values variables are very valuable

links must indicate compatible relationships compatibility may be dependent on the domain or task

[Giarratano & Riley 1998, Friedmann-Hill 2003, Gonzalez & Dankel, 2004]


?????

Pattern Matching ExamplePattern Matching Example

images Do both images refer to the same individual? Do they have other commonalities?

test

BuckyBucky likes fish

BuckyBucky likes fish

Bucky and SatchelSatchel likesBucky


Pattern Matching ExamplePattern Matching Example

shapes

??

?? ?????

?????


Pattern Matching ExamplesPattern Matching Examples

constants and variables

“Hans” “Franz”

?“Josef” “Joseph”

first_name “Joseph”

?last_name “Joseph”


Pattern Matching ExamplesPattern Matching Examples

terms composed of constants, variables, functions

?father(X) “Joseph”

? father(Y)father(X)

father(X) mother(X)

??father(father(X)) grandfather(X)


UnificationUnification formal specification for finding substitutions that make logical

expressions identical the unification algorithm takes two sentences and returns a unifier for

them (if one exists)

Unify(p,q) = if Subst(,p) = Subst(,q)

if there is more than one such substitution, the most general unifier is returned

used in logic programming, automated theorem proving possibly complex operation

quadratic in the size of the expressions “occur check” sometimes omitted

determines if a variable is contained in the term against which it is unified


Pattern Matching in Rule-Based Systems

Pattern Matching in Rule-Based Systems

used to match rules with appropriate facts in working memory rules for which facts can be found are satisfied the combination of a rule with the facts that satisfy it is

used to form activation records one of the activation records is selected for execution


Simplistic Rule-BasedPattern Matching

Simplistic Rule-BasedPattern Matching

go through the list of rules, and check the antecedent (LHS) of each rule against the facts in working memory create an activation record for each rule with a matching

set of facts repeat after each rule firing

very inefficient roughly (number of rules) * (number of facts)(number of patterns)

the actual performance depends on the formulation of the rules and the contents of the working memory


Rete AlgorithmRete Algorithm in most cases, the set of rules in a rule-based system is

relatively constant the facts (contents of working memory) change frequently

most of the contents of working memory, however, don’t change every time

optimization of the matching algorithm remember previous results change only those matches that rely on facts that changed

the Rete algorithm performs an improved matching of rules and facts invented by Charles Forgy in the early 80s basis for many rule-based expert system shells

[ Friedmann-Hill 2003, Giarratano & Riley 1998, Gonzalez & Dankel, 2004]


Rete NetworkRete Network the name comes from the latin word rete

stands for net consists of a network of interconnected nodes

each node represents one or more tests on the LHS of a rule input nodes are at the top, output nodes at the bottom

pattern nodes have one input, and check the names of facts join nodes have two inputs, and combine facts terminal node at the bottom of the network represent individual rules

a rule is satisfied if there is a combination of facts that passes all the test nodes from the top to the output node at the bottom that represents the rule

the Rete network effectively is the working memory for a rule-based system


Rete Network Example 1Rete Network Example 1

(deftemplate x (slot a))

(deftemplate y (slot b))

(defrule example-1

(x (a ?v1))

(y (b ?v1))

==> )

?= x ?= y

Left.0.a ?= Right.b

example-1

?v1 ?v1

?v1 = ?v1


Rete Left and Right MemoriesRete Left and Right Memories left (alpha) memory

contains the left input of a join node right (beta) memory

contains the right input of a join node notation:

Left.p.q ?= Right.r compare the contents of slot q in

fact p from the left memory with slot r in the fact from the right memory

(deftemplate x (slot a))(deftemplate y (slot b))

(defrule example-1(x (a ?v1))(y (b ?v1))==> )

?= x ?= y

Left.0.a ?= Right.b

example-1

?v1 ?v1

?v1 = ?v1


Running the NetworkRunning the Network only facts x and y are considered all facts where x.a == y.b pass the

join network all {x, y} tuples are fowarded to the

next node compare the contents of slot q in

fact p from the left memory with slot r in the fact from the right memory

(deftemplate x (slot a))(deftemplate y (slot b))


?= x ?= y

Left.0.a ?= Right.b

example-1

?v1 ?v1

?v1 = ?v1


Rete Network Example 2Rete Network Example 2 shares some facts with

Example 1(deftemplate x (slot a))


(deftemplate z (slot c))

(defrule example-2

(x (a ?v2))

(y (b ?v2))

(z)

==> )

?= x ?= y ?= z

Left.0.a ?= Right.b

example-2

?v2 ?v2

?v2 = ?v2

?v2


Rete Network Example 2 with AssertRete Network Example 2 with Assert additional fact asserted

(deftemplate x (slot a))


(deftemplate z (slot c))

(defrule example-2

(x (a ?v2))

(y (b ?v2))

(z)

==> )

(assert (z (c 17))

?= x ?= y ?= z

Left.0.a ?= Right.b

example-2

?v2 ?v2

?v2 = ?v2

?v2

?v2 = 17

17


Assert and Retract with ReteAssert and Retract with Rete asserting additional facts imposes some more constraints on the network retracting facts indicates that some previously computed activation

records are not valid anymore, and should be discarded in addition to the actual facts, tags are sent through the networks

ADD to add facts (i.e. for assert) REMOVE to remove facts (i.e. for retract) CLEAR to flush the network memories (i.e. for reset) UPDATE to populate the join nodes of newly added rules

already existing join nodes can neglect these tokens


Rete Network OptimizationRete Network Optimization networks with shared facts can

be combined(deftemplate x (slot a))(deftemplate y (slot b))(deftemplate z (slot c))


(defrule example-2(x (a ?v2))(y (b ?v2))(z)==> )

?= x ?= y ?= z

Left.0.a ?= Right.b

example-1 example-2


Further OptimizationsFurther Optimizations

sophisticated data structures to optimize the network hash table to presort the tokens before running the join

node tests

fine-tuning via parameters frequently trade-off between memory usage and time


Special Cases for Pattern MatchingSpecial Cases for Pattern Matching additional enhancements of the Rete network can be

used to implement specific methods backward chaining

requires a signal indicating to the network that a particular fact is needed

not conditional element indicates the absence of a fact requires special join nodes and special fields in the tokens passing

through the network test conditional element

uses a special join node that ignores its right input the result of the function is passed on


Exploring the Rete Network in JessExploring the Rete Network in Jess

(watch compilations) function diagnostic output when rules are compiledexample-1: +1+1+1+2+t

+1 one-input (pattern) node added to the Rete network +2 two-input (pattern) node added +t terminal node added

(view) function graphical viewer of the Rete network in Jess

(matches <rule-name>) function displays the contents of the left and right memories of the join nodes

for a rule useful for examining unexpected rule behavior


Rule FormulationRule Formulation

Pattern Order General vs. Specific Rules Simple vs. Complex Rules Loading and Saving Facts

[Giarratano & Riley 1998]


Pattern OrderPattern Order

since Rete saves information about rules and facts, it can be critical to order patterns in the right way otherwise a potentially huge number of partial matches can

be generated


Example Pattern OrderExample Pattern Order(deffacts information

(find-match a c e g) f1(item a) f2(item b) f3(item c) f4(item d) f5(item e) f6(item f) f7(item g)) f8

(defrule match-1(find-match ?x ?y ?z ?w) P1(item ?x) P2(item ?y) P3(item ?z) P4(item ?w) P5==>(assert (found-match ?x ?y ?z ?w))

(deffacts information(find-match a c e g)(item a)(item b)(item c)(item d)(item e)(item f)(item g))

(defrule match-2(item ?x)(item ?y)(item ?z)(item ?w)(find-match ?x ?y ?z ?w)==>(assert (found-match ?x ?y ?z ?w))



Pattern MatchesPattern Matches full matchesP1: f1

P2: f2,f3,f4,f5,f6,f7,f8

P3: f2,f3,f4,f5,f6,f7,f8

P4: f2,f3,f4,f5,f6,f7,f8

P5: f2,f3,f4,f5,f6,f7,f8

partial matchesP1: [f1]

P1-2: [f1,f2]

P1-3: [f1,f2,f4]

P1-4: [f1,f2,f4,f6]

P1-5: [f1,f2,f4,f6,f8]

Total: 29 full, 5 partial matches

full matchesP1: f2,f3,f4,f5,f6,f7,f8P2: f2,f3,f4,f5,f6,f7,f8P3: f2,f3,f4,f5,f6,f7,f8P4: f2,f3,f4,f5,f6,f7,f8P5: f1

partial matchesP1: [f2,f3,f4,f5,f6,f7,f8]P1-2: [f2,f2],[f2,f3],[f2,f4],

[f2,f5], [f2,f6],[f2,f7],[f2,f8],

[f3,f2],[f3,f3],[f3,f4],[f3,f5], [f3,f6],[f3,f7],[f3,f8],

...P1-3, P1-4: ...P1-5: [f2,f4,f6,f8, f1]

Total: 29 full, 2801 partial matches


Adding another FactAdding another Fact

what is the effect on the two cases if another fact(item h)

is added? no significant changes for match-1

in particular, no additional partial matches major changes for match-2

another 1880 partial matches


Guidelines for Pattern MatchesGuidelines for Pattern Matches

try to formulate your rule such that the number of matches is low full and partial matches

try to limit the number of old partial matches removing those also is time-consuming

in general, the state of the system should be reasonably stable matches assertion, retraction, modification of facts


Guidelines for Pattern OrderingGuidelines for Pattern Ordering

most specific patterns first smallest number of matching facts largest number of variable bindings to constrain other facts

patterns matching volatile facts go last facts that are changing frequently should be used by

patterns late in the LHS smallest number of changes in partial matches may cause a dilemma with the above guideline

patterns matching the fewest facts first reduces the number of partial matches


Multifield VariablesMultifield Variables

multifield wildcards and multifield variables are very powerful, but possible very inefficient should only be used when needed limit their number in a single slot of a pattern


Test Conditional ElementTest Conditional Element

the test conditional element should be placed as close to the top of the rule as possible reduces the number of partial matches evaluation of expressions during pattern matching is

usually more efficient


Built-In Pattern Matching ConstraintsBuilt-In Pattern Matching Constraints the built-in constraints are always more efficient than the

equivalent expression not so good:(defrule primary-color

color ?x&: (or (eq ?x red)(eq ?x green)(eq ?x blue)

==>(assert (primary-color ?x)))

better:(defrule primary-color

color ?x&red|green|blue)

==>(assert (primary-color ?x)))


General vs. Specific RulesGeneral vs. Specific Rules

some knowledge can be expressed through many specific, or a few general rules specific rules generate a top-heavy Rete network with

many pattern nodes and fewer join nodes general rules offer better opportunities for sharing pattern

and join nodes it usually is easier to write an inefficient general rule than

an inefficient specific rule


Simple vs. Complex RulesSimple vs. Complex Rules

simple rules are sometimes elegant, but not necessarily efficient storing temporary facts can be very helpful

especially in recursive or repetitive programs


Loading and Saving FactsLoading and Saving Facts

facts can be kept in a file, and loaded into memory when needed (load-facts) and (save-facts) functions may lead to visibility or scoping problems if the respective

deftemplates are not contained in the current module


Figure ExampleFigure Example


Post-TestPost-Test


Use of ReferencesUse of References

[Giarratano & Riley 1998] [Russell & Norvig 1995] [Jackson 1999] [Durkin 1994]



Important Concepts and TermsImportant Concepts and Terms agenda assert backward chaining constant fact expert system (ES) expert system shell forward chaining join node knowledge base knowledge-based system left (alpha) memory matches matching

pattern pattern matching pattern node RETE algorithm retract right (beta) memory rule substitution term test conditional element unification variable view working memory


Summary ES ImplementationSummary ES Implementation for rule-based systems, an efficient method for pattern

matching between the rule antecedents and suitable facts is very critical matching every rule against all possible facts repeatedly is very

inefficient the Rete algorithm is used in many expert system shells

it constructs a network from the facts and rules in the knowledge base since certain aspects of the knowledge base are quite static, repeated

matching operations can be avoided a few strategies can be used by programmers to achieve

better performance most specific patterns first, patterns with volatile facts last careful use of multifield variables, general rules use of the test conditional element, built-in pattern constraints loading and saving of facts

© 2002-9 Franz J. Kurfess ES Implementation 1 CPE/CSC 481: Knowledge-Based Systems Dr. Franz J....

Documents

Transcript of © 2002-9 Franz J. Kurfess ES Implementation 1 CPE/CSC 481: Knowledge-Based Systems Dr. Franz J....