Knowledge-Based Systems Expert Systemscobweb.cs.uga.edu/~potter/kbs/VaTechECE4524.pdf · MYCIN uses...

60
9expert.doc EE 4524 Page 1 ECpE 4524 Knowledge-Based Systems often called Expert Systems

Transcript of Knowledge-Based Systems Expert Systemscobweb.cs.uga.edu/~potter/kbs/VaTechECE4524.pdf · MYCIN uses...

Page 1: Knowledge-Based Systems Expert Systemscobweb.cs.uga.edu/~potter/kbs/VaTechECE4524.pdf · MYCIN uses IF-THEN rules Each rule has a certainty factor associated with it (a measure of

9expert.doc EE 4524 Page 1

ECpE 4524

Knowledge-BasedSystems

often called

Expert Systems

Page 2: Knowledge-Based Systems Expert Systemscobweb.cs.uga.edu/~potter/kbs/VaTechECE4524.pdf · MYCIN uses IF-THEN rules Each rule has a certainty factor associated with it (a measure of

9expert.doc EE 4524 Page 2

Knowledge-based systems(textbook, chapter 20)

Goal:Try to solve the kinds of problemsthat normally require human experts

Typical examples:medical diagnosis, financial analysis,factory production scheduling

Why study knowledge-based systems?To understand human reasoning

methodsHuman experts tend to take

vacations, get hired by othercompanies, ask for raises, retire,become ill, die, . . .

Lots of commercial successes!

Page 3: Knowledge-Based Systems Expert Systemscobweb.cs.uga.edu/~potter/kbs/VaTechECE4524.pdf · MYCIN uses IF-THEN rules Each rule has a certainty factor associated with it (a measure of

9expert.doc EE 4524 Page 3

Expert system overview:

KNOWLEDGEBASE

REASONINGMECHANISM

Problemdescription

Analysis andjustification

The knowledge base . . .● contains "domain knowledge,"

normally provided by human experts● is typically very specialized for a

particular problem domain● is often encoded as IF-THEN rules● may incorporate heuristics or

probabilities● is a valuable commodity

Page 4: Knowledge-Based Systems Expert Systemscobweb.cs.uga.edu/~potter/kbs/VaTechECE4524.pdf · MYCIN uses IF-THEN rules Each rule has a certainty factor associated with it (a measure of

9expert.doc EE 4524 Page 4

Building, validating, and maintaininga knowledge base is a skill (art)called knowledge engineering

The reasoning mechanism . . .● takes descriptions from the user

about the problem to be solved● requests additional information

from the user as needed● interprets the knowledge base to

make inferences, drawconclusions, and ultimately giveadvice

● explains its reasoning to the user(how were the conclusionsreached?)

● is sometimes called aninference engine

Page 5: Knowledge-Based Systems Expert Systemscobweb.cs.uga.edu/~potter/kbs/VaTechECE4524.pdf · MYCIN uses IF-THEN rules Each rule has a certainty factor associated with it (a measure of

9expert.doc EE 4524 Page 5

An example:

PUFF (1979)Pulmonary function analysis

Physician refers patient topulmonary testing lab

Patient inhales/exhales throughtube connected to computerizedinstrument which measures flowrates and air volumes

PUFF accepts this data along withauxiliary data (age, sex,smoking history), and printsdiagnosis in English

Now used on a routine basis (?)

Page 6: Knowledge-Based Systems Expert Systemscobweb.cs.uga.edu/~potter/kbs/VaTechECE4524.pdf · MYCIN uses IF-THEN rules Each rule has a certainty factor associated with it (a measure of

9expert.doc EE 4524 Page 6

Building PUFF's knowledge base:A knowledge engineer sat downwith an expert pulmonaryphysiologist at the PacificMedical Center in SanFrancisco and developed rules(64 in all)

IF 1. ----------2. ----------3. ----------. . .

THEN 1. ----------2. ----------. . .

(A more recent version had about400 rules.)

Page 7: Knowledge-Based Systems Expert Systemscobweb.cs.uga.edu/~potter/kbs/VaTechECE4524.pdf · MYCIN uses IF-THEN rules Each rule has a certainty factor associated with it (a measure of

9expert.doc EE 4524 Page 7

Example PUFF rule:

RULE31IF:

1. The severity of obstructive airwaysdisease of the patient is greater than or equalto mild, and2. the degree of diffusion defect of thepatient is greater than or equal to mild, and3. the TLC observed/predicted of the patientis greater than or equal to 110, and4. the observed/predicted difference inRV/TLC of the patient is greater than orequal to 10

THEN:1. There is strongly suggestive evidence(0.9) that the subtype of obstructive airwaysdisease is emphysema, and2. It is definite (1.0) that "OAD, DiffusionDefect, elevated TLC, and elevated RVtogether indicate emphysema" is one of thefindings

Page 8: Knowledge-Based Systems Expert Systemscobweb.cs.uga.edu/~potter/kbs/VaTechECE4524.pdf · MYCIN uses IF-THEN rules Each rule has a certainty factor associated with it (a measure of

9expert.doc EE 4524 Page 8

Sample PUFF session:

PATIENT DATA:The degree of dyspnea: MODERATELY-SEVEREThe severity of coughing: MILDSputum production: MODERATELY-SEVEREThe number of pack-years of smoking: 48Referal diagnosis: BRONCHITISIVC/IVC-predicted: 80RV/RV-predicted: 191FVC/FVC-predicted: 87TLC observed/predicted: 127Predicted FEV1/FVC: 83TLC(DLCO) observed/predicted: 83FEV1/FVC ratio: 50RV/TLC observed/predicted: 21MMF/MMF-predicted: 19DLCO/DLCO-predicted: 48The slope of (F50obs-F25obs)/FVCobs: 19

DEGREE OF OBSTRUCTIVE AIRWAYS DISEASE:OAD degree by slope: MODERATELY-SEVERE 700OAD degree by MMF: SEVERE 900OAD degree by FEV1: MODERATELY-SEVERE 700Final OAD degree: MODERATELY-SEVERE 910

SEVERE 900

Page 9: Knowledge-Based Systems Expert Systemscobweb.cs.uga.edu/~potter/kbs/VaTechECE4524.pdf · MYCIN uses IF-THEN rules Each rule has a certainty factor associated with it (a measure of

9expert.doc EE 4524 Page 9

INTERPRETATION:Obstruction is indicated by curvature of the flow-volume

loop.Forced Vital Capacity is normal and peak flow rates are

reduced, indicating severe airway obstruction.Change in expired flow rates following bronchodilation

shows that there is reversibility of airwayobstruction.

Elevated lung volumes indicate overinflation.Air trapping is indicated by the elevated difference

between observed and predicted RV/TLC ratios.Airway obstruction is consistent with the patient's

smoking history.The airway obstruction accounts for the patient's dyspnea.Although bronchodilators were not useful in this one

case, prolonged use may prove to be beneficial.Obstructive Airways Disease of mixed types.

Page 10: Knowledge-Based Systems Expert Systemscobweb.cs.uga.edu/~potter/kbs/VaTechECE4524.pdf · MYCIN uses IF-THEN rules Each rule has a certainty factor associated with it (a measure of

9expert.doc EE 4524 Page 10

How were the rules produced?100 cases (previously diagnosed

patients) were selected

The cases were chosen to span thevariety of known disease states

The pulmonary function expert posedhypothetical rules for diagnosingthe illness

The knowledge engineer encodedthe rules (in LISP) and testedthem with the test cases.

The expert reviewed the test resultsand modified or added rules tohandle the cases that wereincorrectly diagnosed

Looping continued until the expertwas satisfied

Page 11: Knowledge-Based Systems Expert Systemscobweb.cs.uga.edu/~potter/kbs/VaTechECE4524.pdf · MYCIN uses IF-THEN rules Each rule has a certainty factor associated with it (a measure of

9expert.doc EE 4524 Page 11

How to test PUFF's performance?150 additional different cases were

analyzed1) by human experts and2) by PUFF

The diagnoses were compared:90% matched to same degree ofseverity100% matched to within onedegree of severity

Effort:50 hours by the expert400 hours by the knowledgeengineer

The 64 rules were "popped into" anexisting expert system

Page 12: Knowledge-Based Systems Expert Systemscobweb.cs.uga.edu/~potter/kbs/VaTechECE4524.pdf · MYCIN uses IF-THEN rules Each rule has a certainty factor associated with it (a measure of

9expert.doc EE 4524 Page 12

OBSERVATIONSHuman experts are often unaware of

how they reach conclusionsThe expert usually knows more than

he/she is aware of knowingThe knowledge brought to bear by the

expert is often experiential, heuristic,and uncertain

General problem-solvers (domain-independent) are too weak forbuilding real-world, high-performance systems

The behavior of the best problem-solvers (humans) is weak andshallow except in areas ofspecialization

Expertise in one specialization areausually does not transfer well to otherareas

Page 13: Knowledge-Based Systems Expert Systemscobweb.cs.uga.edu/~potter/kbs/VaTechECE4524.pdf · MYCIN uses IF-THEN rules Each rule has a certainty factor associated with it (a measure of

9expert.doc EE 4524 Page 13

Recall weak vs. strong methods:

Weak methodsdomain-independent, general-

purpose(example: GPS)

Strong methodsdomain-specific, knowledge-rich(examples: knowledge-based

systems)

Page 14: Knowledge-Based Systems Expert Systemscobweb.cs.uga.edu/~potter/kbs/VaTechECE4524.pdf · MYCIN uses IF-THEN rules Each rule has a certainty factor associated with it (a measure of

9expert.doc EE 4524 Page 14

Example expert systemsMedicineMYCIN (1976)Identification of bacteria in blood and

urine samples; prescription ofantibiotics

INTERNIST / CADUCEUS (1970s /1984)

Diagnosis of majority of diseases in fieldof internal medicine

PUFF (1979)Interpretation of respiratory tests for

diagnosis of pulmonary disordersBABY (19??)Patient monitoring in a newborn

intensive care unitQMR (1988) (Quick Medical Record)Assists physicians in diagnosis of over

4000 disease manifestations (usesthe INTERNIST knowledge base)

Page 15: Knowledge-Based Systems Expert Systemscobweb.cs.uga.edu/~potter/kbs/VaTechECE4524.pdf · MYCIN uses IF-THEN rules Each rule has a certainty factor associated with it (a measure of

9expert.doc EE 4524 Page 15

CHEMISTRYDENDRAL (1960s and 1970s)Identification of molecular structure

of organic compoundsCRYSALIS (19??)Interpretation of electron density

maps in protein crystallographyMOLGEN (19??)Planning DNA-manipulation

experiments in moleculargenetics

AGRICULTUREPLANT/dsDiagnosing diseases in soybeansPLANT/cdDiagnosing cutworm damage in

corn

Page 16: Knowledge-Based Systems Expert Systemscobweb.cs.uga.edu/~potter/kbs/VaTechECE4524.pdf · MYCIN uses IF-THEN rules Each rule has a certainty factor associated with it (a measure of

9expert.doc EE 4524 Page 16

OTHERSPROSPECTOR (1978)Provides advice on mineral prospectingMACSYMA (1968 - present)Symbolic solutions to mathematical

problemsR1 / XCON (1982)Configures VAX computer systemsGATES (1988)Used by TWA at JFK airport to assist

ground controllers in assigning gatesto arriving and departing flights

DESIGN ADVISOR (1989)Critiques IC designsTOP SECRET (1989)Decide the correct security classification

to give a nuclear weapons document

Page 17: Knowledge-Based Systems Expert Systemscobweb.cs.uga.edu/~potter/kbs/VaTechECE4524.pdf · MYCIN uses IF-THEN rules Each rule has a certainty factor associated with it (a measure of

9expert.doc EE 4524 Page 17

DENDRALFeigenbaum (1960s and 70s)

One of the first expert systems

Identifies of molecular structure oforganic compounds

Uses mass spectrogram andnuclear magnetic resonance(NMR) data

Page 18: Knowledge-Based Systems Expert Systemscobweb.cs.uga.edu/~potter/kbs/VaTechECE4524.pdf · MYCIN uses IF-THEN rules Each rule has a certainty factor associated with it (a measure of

9expert.doc EE 4524 Page 18

MYCIN (a precursor to PUFF)(textbook, Section 8.2)

Shortliffe, 1976 (Stanford, in Interlisp)

MYCIN is possibly the best knownexpert system that has beendeveloped

MYCIN can diagnose bacterialinfections and recommendtreatment

MYCIN was designed solely as aresearch effort (never used inpractice?);it served as a model for muchsubsequent expert-system work

Page 19: Knowledge-Based Systems Expert Systemscobweb.cs.uga.edu/~potter/kbs/VaTechECE4524.pdf · MYCIN uses IF-THEN rules Each rule has a certainty factor associated with it (a measure of

9expert.doc EE 4524 Page 19

MYCIN uses IF-THEN rules

Each rule has a certainty factorassociated with it(a measure of the confidenceassociated with the rule's conclusion)

exampleRULE50IF 1) the infection is primary-bacteremia,

and2) the site of the culture is one of thesterile sites, and3) the suspected portal of entry of theorganism is the gastrointestinal tract

THENthere is suggestive evidence (0.7) thatthe identity of the organism isbacteroides

Page 20: Knowledge-Based Systems Expert Systemscobweb.cs.uga.edu/~potter/kbs/VaTechECE4524.pdf · MYCIN uses IF-THEN rules Each rule has a certainty factor associated with it (a measure of

9expert.doc EE 4524 Page 20

in LISP:PREMISE:(AND

(SAME CNTXT INFECTPRIMARY-BACTEREMIA)(MEMBF CNTXT SITESTERILISITES)(SAME CNTXT PORTAL GI) )

ACTION:(CONCLUDE CNTXT IDENT

BACTEROIDES TALLY 0.7)

Each rule:premise ⇒ action

Each premise:logical conjunction of clauses

Each clause:<predicate> <object> <attribute> <value>

Page 21: Knowledge-Based Systems Expert Systemscobweb.cs.uga.edu/~potter/kbs/VaTechECE4524.pdf · MYCIN uses IF-THEN rules Each rule has a certainty factor associated with it (a measure of

9expert.doc EE 4524 Page 21

MYCIN used a front-end known as

TEIRESIAS(textbook, section 20.3)

Davis, 1977, 1982

TEIRESIAS is a knowledge-acquisition manager; it providesexplanations of how conclusionsare reached

Page 22: Knowledge-Based Systems Expert Systemscobweb.cs.uga.edu/~potter/kbs/VaTechECE4524.pdf · MYCIN uses IF-THEN rules Each rule has a certainty factor associated with it (a measure of

9expert.doc EE 4524 Page 22

Example TEIRESIAS rule(actually a meta-rule)

IF1) the patient is a compromisedhost, and2) there are rules that mentionin their premise pseudomonas,and3) there are rules that mentionin their premise klebsiellas

THENthere is suggestive evidence(0.4) that the former should bedone before the latter

Page 23: Knowledge-Based Systems Expert Systemscobweb.cs.uga.edu/~potter/kbs/VaTechECE4524.pdf · MYCIN uses IF-THEN rules Each rule has a certainty factor associated with it (a measure of

9expert.doc EE 4524 Page 23

Example MYCIN / TEIRESIASsession:

1) Patient's name:** Fred Smith2) Sex** male3) Age:** 554) Have you been able to obtain positive cultures

from a site at which Fred Smith has aninfection?

** yes5) What is the infection?** Primary-bacteremia6) Please give the date and approximate time

when signs or symptoms of the primary-bacteremia (infection-1) appeared.

** May 5, 19807) From what site was the specimen for culture-1

taken?** Blood8) Please give the date and time when this blood

culture (culture-1) was obtained.** May 9, 1980

Page 24: Knowledge-Based Systems Expert Systemscobweb.cs.uga.edu/~potter/kbs/VaTechECE4524.pdf · MYCIN uses IF-THEN rules Each rule has a certainty factor associated with it (a measure of

9expert.doc EE 4524 Page 24

9) Enter the identity of organism-1.** unknown10) Is organism-1 a rod, or coccus (etc.)?** rod11) The gram stain of organism-1:** grammeg12) Have there been positive cultures from sites

which are NOT associated with infections aboutwhich you are seeking advice?

** no13) Did organism-1 grow aerobically?** yes (0.8)14) Did organism-1 grow anerobically?** yes15) What is the suspected portal of entry of

organism-1 into the sterile site?** GI (0.6). . . etc.

Page 25: Knowledge-Based Systems Expert Systemscobweb.cs.uga.edu/~potter/kbs/VaTechECE4524.pdf · MYCIN uses IF-THEN rules Each rule has a certainty factor associated with it (a measure of

9expert.doc EE 4524 Page 25

My therapy recommendation will be based on thefollowing probable infection(s) and potentialcausative organism(s):

INFECTION-1 is PRIMARY-BACTEREMIA<Item 1> Organism-1 may be PSEUDOMONAS-

AERUGINOS<Item 2> Organism-1 may be KLEBSIELLA-

PNEUMONIAE<Item 3> Organism-1 may be E. COLI<Item 4> Organism-1 may be BACTEROIDES-

FRAGILIS<Item 5> Organism-1 may be ENTEROBACTER<Item 6> Organism-1 may be PROTEUS-NON-

MIRABILIS

46) On a scale of 0 to 4, where higher numbersindicate increasing severity, how would you rateFred Smith's degree of sickness?

** 347) Does Fred Smith have a clinically significant

allergic reaction to any antimicrobial agent?** no48) Fred Smith's weight in kilograms:** 70

Page 26: Knowledge-Based Systems Expert Systemscobweb.cs.uga.edu/~potter/kbs/VaTechECE4524.pdf · MYCIN uses IF-THEN rules Each rule has a certainty factor associated with it (a measure of

9expert.doc EE 4524 Page 26

[Rec 1] My preferred therapy recommendation is asfollows:

In order to cover for Items <1 2 3 5 6>Give: GENTAMICINDose: 119 mg q8h IV [or IM] for 10 daysComments: Modify dose in renal failure

In order to cover for Item <4>Give: CLINDAMYCINDose: 595 mg q6h IV [or IM] for 14 daysComments: If diarrhea or other GI symptomsdevelop, patient should be evaluated forpossible pseudomembranous colitis

Page 27: Knowledge-Based Systems Expert Systemscobweb.cs.uga.edu/~potter/kbs/VaTechECE4524.pdf · MYCIN uses IF-THEN rules Each rule has a certainty factor associated with it (a measure of

9expert.doc EE 4524 Page 27

Do you wish to see the next-choice therapy?** yesWhich, if any, of the above drugs do you wish

eliminated from consideration?** Clindamycin

[Rec 2]In order to cover for Items <2 3 4 5 6>

Give: CHLORAMPHENICOLDose: 1 mg q6h IV for 14 daysComments: Monitor patient's white count

In order to cover for Item <1>Give: GENTAMICINDose: 119 mg q8h IV [or IM] for 10 daysComments: Modify dose in renal failure.

Page 28: Knowledge-Based Systems Expert Systemscobweb.cs.uga.edu/~potter/kbs/VaTechECE4524.pdf · MYCIN uses IF-THEN rules Each rule has a certainty factor associated with it (a measure of

9expert.doc EE 4524 Page 28

The ability to explain the reasoningpath is important

(MYCIN:) Is the patient's illness withorganism-1 a hospital-acquired infection?

(USER:) ** why(TEIRESIAS:) (WHY is it important to determine

whether or not the infection with organism-1 wasacquired while the patient was hospitalized?)

[1.0] This will aid in determining the category oforganism-1. It has already been establishedthat

[1.1] the gram stain of organism-1 is gramneg,and

[1.2] the morphology of organism-1 is rod, and[1.3] the aerobicity of organism-1 is faculativeTherefore, if[1.4] the infection with organism-1 was not acquired

while the patient was hospitalizedthen there is strongly suggestive evidence (0.8) that

the category of organism-1 isenterobacteriaceae.

Page 29: Knowledge-Based Systems Expert Systemscobweb.cs.uga.edu/~potter/kbs/VaTechECE4524.pdf · MYCIN uses IF-THEN rules Each rule has a certainty factor associated with it (a measure of

9expert.doc EE 4524 Page 29

Also: there is weakly suggestive evidence (0.1) thatthe identify of organism-1 is pseudomonos-aeruginosa [RULE37].

(USER:) how 1.3(TEIRESIAS:) (HOW was it established that the

aerobicity of organism-1 is faculative?)The following were used:[3.1] RULE27 indicated there is strongly

suggestive evidence (0.8) that the aerobicity oforganism-1 is faculative.

[3.2] RULE27 indicated there is weaklysuggestive evidence (0.2) that the aerobicity oforganism-1 is anerobic

Since this gave a cumulative CF of (0.8) forfaculative, and (0.2) for anerobic, it has beenestablished that the aerobicity of organism-1 isfaculative.

(USER:) why didn't you considerstreptococcus as a possibility for organism-1?

(TEIRESIAS:) The following rule could have beenused to determine that the identity of organism-1was streptococcus: RULE33.

Page 30: Knowledge-Based Systems Expert Systemscobweb.cs.uga.edu/~potter/kbs/VaTechECE4524.pdf · MYCIN uses IF-THEN rules Each rule has a certainty factor associated with it (a measure of

9expert.doc EE 4524 Page 30

But clause 2 ("the morphology of the organism iscoccus") was already known to be false fororganism-1, so the rule was never tried.

Page 31: Knowledge-Based Systems Expert Systemscobweb.cs.uga.edu/~potter/kbs/VaTechECE4524.pdf · MYCIN uses IF-THEN rules Each rule has a certainty factor associated with it (a measure of

9expert.doc EE 4524 Page 31

How does MYCIN know whatquestions to ask?

At each point, the question is determined byMYCIN's current hypothesis (andanswers to previous questions)

MYCIN is a backward-chaining system:Eg., to determine the cause of thepatient's illness, MYCIN looks for ruleswhich have a THEN clause suggestingdiseases;MYCIN then uses the IF clauses to setup subgoals, and looks for THENclauses of other rules to satisfy thesesubgoals, etc.

This approach makes it easier for thephysician to follow the "thought"process, and it simplifies the English-language interface

Page 32: Knowledge-Based Systems Expert Systemscobweb.cs.uga.edu/~potter/kbs/VaTechECE4524.pdf · MYCIN uses IF-THEN rules Each rule has a certainty factor associated with it (a measure of

9expert.doc EE 4524 Page 32

MYCIN summary

... recommends therapies forpatients with bacterial infections

... uses IF-THEN rules (withcertainty factors) to representknowledge

... interacts with a physician toacquire clinical data

... asks questions based on currenthypothesis and known data

... reasons backward from its goalof recommending a therapy fora particular patient

... stores approx. 500 IF-THENrules, and can recognize about100 causes of bacterial infection

Page 33: Knowledge-Based Systems Expert Systemscobweb.cs.uga.edu/~potter/kbs/VaTechECE4524.pdf · MYCIN uses IF-THEN rules Each rule has a certainty factor associated with it (a measure of

9expert.doc EE 4524 Page 33

TEIRESIAS summary

... serves as a front-end to MYCIN

... was the first program to provideexplanations of how conclusionswere reached

... intercepts questions such as "why"and "how" from the physician(i.e., why does MYCIN wantcertain information, and how didMYCIN reach a certainconclusion)

... TEIRESIAS can answer "why"questions by examining its internaltree of subgoals

... TEIRESIAS can answer "how"questions by identifying the piecesof evidence that supportedMYCIN's IF clauses

Page 34: Knowledge-Based Systems Expert Systemscobweb.cs.uga.edu/~potter/kbs/VaTechECE4524.pdf · MYCIN uses IF-THEN rules Each rule has a certainty factor associated with it (a measure of

9expert.doc EE 4524 Page 34

Expert system shells

After MYCIN was built, someoneobserved that the knowledge basecould be replaced by completely newrules

MYCIN without its knowledge base wascalled EMYCIN (Empty MYCIN)(and was used to implement PUFF)

Today you can buy similar "shells" thatcontain a user interface, a reasoningsubsystem, and an explanationsubsystem

With such a shell, the user canconcentrate on the knowledge base

Page 35: Knowledge-Based Systems Expert Systemscobweb.cs.uga.edu/~potter/kbs/VaTechECE4524.pdf · MYCIN uses IF-THEN rules Each rule has a certainty factor associated with it (a measure of

9expert.doc EE 4524 Page 35

In many expert systems, the rulesare written as follows:

symptom ⇒ disease

(the diagnosis must work from symptoms to find thecause)

But in reality, we know that

disease ⇒ symptom

Abductive reasoning is not truth-preserving:

P ⇒ QQ_________∴P

Page 36: Knowledge-Based Systems Expert Systemscobweb.cs.uga.edu/~potter/kbs/VaTechECE4524.pdf · MYCIN uses IF-THEN rules Each rule has a certainty factor associated with it (a measure of

9expert.doc EE 4524 Page 36

Reasoning under uncertainty(Inexact reasoning)

We can attach "confidence" or"belief" values to

● the inference itself:A ⇒ B (with confidence 0.8)

● the evidence:A (which has confidence 0.6)

⇒ B

● both

Page 37: Knowledge-Based Systems Expert Systemscobweb.cs.uga.edu/~potter/kbs/VaTechECE4524.pdf · MYCIN uses IF-THEN rules Each rule has a certainty factor associated with it (a measure of

9expert.doc EE 4524 Page 37

Our first impulse for inexact reasoning:use probability theory!

What is Pr(measles | spots) ?

Recall Bayes' theorem:

Pr( )Pr( ) Pr( )

Pr( )measles spots

spots measles measles

spots|

|=

Looks fine. Now we'd like toconsider other possiblediseases:

Pr( )Pr( ) Pr( )

Pr( )H spots

spots H H

spotsii i|

|=

If the diseases are exhaustive andmutually exclusive:

=∑

Pr( ) Pr( )

Pr( ) Pr( )

spots H H

spots H Hi i

i ii

|

|

Page 38: Knowledge-Based Systems Expert Systemscobweb.cs.uga.edu/~potter/kbs/VaTechECE4524.pdf · MYCIN uses IF-THEN rules Each rule has a certainty factor associated with it (a measure of

9expert.doc EE 4524 Page 38

Now consider two differentsymptoms for one disease:

Pr( )Pr( ) Pr( )

Pr( )H spots fever

spots fever H H

spots feveri

i i||

∧ =∧

Problem: how do we compute these ?Pr( )spots fever∧

Pr( )spots fever Hi∧ |

It is common (and absurd!) toassume that spots and fever areindependent:

Pr( ) Pr( ) Pr( )spots fever spots fever∧ =

To really use Bayes' theorem, we wouldneed probabilities for all possible

Page 39: Knowledge-Based Systems Expert Systemscobweb.cs.uga.edu/~potter/kbs/VaTechECE4524.pdf · MYCIN uses IF-THEN rules Each rule has a certainty factor associated with it (a measure of

9expert.doc EE 4524 Page 39

combinations of symptoms in allconditional expressions: not feasible!

Page 40: Knowledge-Based Systems Expert Systemscobweb.cs.uga.edu/~potter/kbs/VaTechECE4524.pdf · MYCIN uses IF-THEN rules Each rule has a certainty factor associated with it (a measure of

9expert.doc EE 4524 Page 40

Standard reasons why Bayesianreasoning cannot work:

● in "pure form" it requires animpossible number of probabilities

● the usual remedy is to imposeabsurd assumptions ofindependence

● knowing any probability may beunrealistic(usually just use statistical frequency)

● it only works for the single-diseasesituation

Still, it's a good starting point . . .

Page 41: Knowledge-Based Systems Expert Systemscobweb.cs.uga.edu/~potter/kbs/VaTechECE4524.pdf · MYCIN uses IF-THEN rules Each rule has a certainty factor associated with it (a measure of

9expert.doc EE 4524 Page 41

MYCIN's Confidence Factors

a MYCIN rule: E ⇒ H (CF=x)

Confidence Factor:1.0 true with completeconfidence

-1.0 false with complete confidence

If x = 1.0 and E is a predicate, then wehave normal logic

CF(H|E) = MB(H|E) - MD(H|E)

MB: "measure of belief"MD: "measure of disbelief"

Each is in range [0, 1]When one is nonzero, the other is

normally zero

Page 42: Knowledge-Based Systems Expert Systemscobweb.cs.uga.edu/~potter/kbs/VaTechECE4524.pdf · MYCIN uses IF-THEN rules Each rule has a certainty factor associated with it (a measure of

9expert.doc EE 4524 Page 42

Consider E E H1 2∧ ⇒ (CF = x)

If the Ei are all certain, then H has CF =x

If the Ei are not all certain, then weneed to "fold together" theconfidence factors

For conjunctive evidence:MB E E( )1 2∧ =

min( ( ), ( ))MB E MB E1 2MD E E( )1 2∧ =

max( ( ), ( ))MD E MD E1 2

Now consider E E H1 2∨ ⇒ (CF = x):For disjunctive evidence:MB E E( )1 2∨ =

max( ( ), ( ))MB E MB E1 2MD E E( )1 2∨ =

min( ( ), ( ))MD H MD H1 2

Page 43: Knowledge-Based Systems Expert Systemscobweb.cs.uga.edu/~potter/kbs/VaTechECE4524.pdf · MYCIN uses IF-THEN rules Each rule has a certainty factor associated with it (a measure of

9expert.doc EE 4524 Page 43

What CF do we assign to H, foruncertain evidence P?

P H⇒ (CF = x)

MB H MB H CF P( ) '( ) ( , ( ))= max 0

MD H MD H CF P( ) '( ) ( , ( ))= max 0

Page 44: Knowledge-Based Systems Expert Systemscobweb.cs.uga.edu/~potter/kbs/VaTechECE4524.pdf · MYCIN uses IF-THEN rules Each rule has a certainty factor associated with it (a measure of

9expert.doc EE 4524 Page 44

Now consider this:Rule 1: E H1 ⇒ (CF=x)Rule 2: E H2 ⇒ (CF=y)

If both Ei are true,then both should contribute tothe confidence that H is true:

MB H E E( )| 1 2∧ =0 11 2

1 2

1 2

MD H E E

MB H E MB H E otherwise

MB H E MB H E

( )

( ) ( )

( ) ( )

|

| |

| |

∧ =+

MD H E E( )| 1 2∧ =0 11 2

1 2

1 2

MB H E E

MD H E MD H E otherwise

MD H E MD H E

( )

( ) ( )

( ) ( )

|

| |

| |

∧ =+

(see text, p. 234)

Page 45: Knowledge-Based Systems Expert Systemscobweb.cs.uga.edu/~potter/kbs/VaTechECE4524.pdf · MYCIN uses IF-THEN rules Each rule has a certainty factor associated with it (a measure of

9expert.doc EE 4524 Page 45

In MYCIN, rules are invoked bybackwards-chaining usingexhaustive depth-first search

Eg., find all rules that conclude theidentity of an organism

Eg., see if all conditions are met; ifnot, set up subgoals (based onIF clauses)

If -0.2 < CF < 0.2, the CF value isregarded as unknown. In thiscase, MYCIN asks the user.

Page 46: Knowledge-Based Systems Expert Systemscobweb.cs.uga.edu/~potter/kbs/VaTechECE4524.pdf · MYCIN uses IF-THEN rules Each rule has a certainty factor associated with it (a measure of

9expert.doc EE 4524 Page 46

a different approach . . .

PLANT/ds

an expert system for diagnosingsoybean diseases

Rule form: extended propositionallogic

Page 47: Knowledge-Based Systems Expert Systemscobweb.cs.uga.edu/~potter/kbs/VaTechECE4524.pdf · MYCIN uses IF-THEN rules Each rule has a certainty factor associated with it (a measure of

9expert.doc EE 4524 Page 47

PLANT/ds rules

Let x1, x2, ..., xn representdifferent “features” that can beobserved or measured

[x2 != 3] [x3 = 1, 3] v [x4 < 4]=> [decision = A]

(each [...] is called a “selector”; the first oneis TRUE if x2 is not equal to 3)

0.9 ([x1 = 3] [x3 >= 2])+ 0.1 ( [x3 = 2..4])

=> [decision = B](90% of the support comes from 2

selectors, and 10% from another)

Page 48: Knowledge-Based Systems Expert Systemscobweb.cs.uga.edu/~potter/kbs/VaTechECE4524.pdf · MYCIN uses IF-THEN rules Each rule has a certainty factor associated with it (a measure of

9expert.doc EE 4524 Page 48

A sample PLANT/ds rule:

0.8 * ([time = Aug..Sept][precip = 0.8][fruiting bodies = present][stem cankers = above 2nd node][fruit pods = absent] )

+0.2 * ([temp >= normal]

[canker lesion color = brown][# years crop repeated = 2 yrs.])

=> [diagnosis = diaporthe stem canker]

Page 49: Knowledge-Based Systems Expert Systemscobweb.cs.uga.edu/~potter/kbs/VaTechECE4524.pdf · MYCIN uses IF-THEN rules Each rule has a certainty factor associated with it (a measure of

9expert.doc EE 4524 Page 49

Rule evaluation:● each selector [var = value]

contributes a strength of evidencefrom 0 to 1

● if selector is not matched exactly,and is for a variable with linearlyordered domain, then a normaldistribution is assumed

● for conjunctions, evidence iscombined by product, min, oraverage

● for disjunctions, evidence iscombined by max or (w1+w2-w1w2)

● evidence weights are scaled bymodule coefficients and added to getstrength of evidence of conclusion

● The rule that scores best wins;Rules coming within 25% of best areconsidered alternativesRules below 0.8 are not reported

Page 50: Knowledge-Based Systems Expert Systemscobweb.cs.uga.edu/~potter/kbs/VaTechECE4524.pdf · MYCIN uses IF-THEN rules Each rule has a certainty factor associated with it (a measure of

9expert.doc EE 4524 Page 50

PLANT/ds sessionEach question which follows will have a list of answers.

Choose the best answer, type the correspondingletter, then press return.

TIME OF OCCURRENCE?A. AprilB. MayC. JuneD. JulyE. AugustF. SeptemberG. OctoberH. Do not know** B

CONDITION OF LEAVES?A. NormalB. AbnormalC. Do not know** B

CONDITION OF STEM?A. NormalB. AbnormalC. Do not know** why

Page 51: Knowledge-Based Systems Expert Systemscobweb.cs.uga.edu/~potter/kbs/VaTechECE4524.pdf · MYCIN uses IF-THEN rules Each rule has a certainty factor associated with it (a measure of

9expert.doc EE 4524 Page 51

IF[CONDITION OF STEM = NORMAL] then thefollowing are indicated:Downy MildewBacterial PustulePhyllosticta Leaf SpotAlternaria Leaf Spot

IF[CONDITION OF STEM = ABNORMAL] then thefollowing are indicated:Charcoal RotRhizoctonia Root RotPhytophthora RotBrown Stem RotAnthracnoseFrog Eye Leaf Spot

CONDITION OF STEM?A. NormalB. AbnormalC. Do not know** A

Page 52: Knowledge-Based Systems Expert Systemscobweb.cs.uga.edu/~potter/kbs/VaTechECE4524.pdf · MYCIN uses IF-THEN rules Each rule has a certainty factor associated with it (a measure of

9expert.doc EE 4524 Page 52

CONDITION OF FRUIT-PODS?A. NormalB. DiseasedC. Few or none presentD. Does not applyE. Do not know** A

CONDITION OF SEED?A. NormalB. AbnormalC. Do not know** A

CONDITION OF ROOTS?A. NormalB. AbnormalC. Galls or cysts presentD. Do not know** A...

Page 53: Knowledge-Based Systems Expert Systemscobweb.cs.uga.edu/~potter/kbs/VaTechECE4524.pdf · MYCIN uses IF-THEN rules Each rule has a certainty factor associated with it (a measure of

9expert.doc EE 4524 Page 53

CROPPING HISTORY?A. Crop different than last yearB. Crop same as last yearC. Crop same for last three yearsD. Crop same for four or more yearsE. Do not know** A

THE EVIDENCE PRESENTED SUGGESTS

BROWN SPOT WITH A DEGREE OF CONFIDENCE1.00

ALTERNATIVELY

PHYLLOSTICTA LEAF SPOT WITH A DEGREE OFCONFIDENCE 0.82

Page 54: Knowledge-Based Systems Expert Systemscobweb.cs.uga.edu/~potter/kbs/VaTechECE4524.pdf · MYCIN uses IF-THEN rules Each rule has a certainty factor associated with it (a measure of

9expert.doc EE 4524 Page 54

The burden of theKnowledge Engineer

skill system rulesdiagnosing

soybeandiseases

PLANT/ds 25

identifyingbacteria

MYCIN 400

finding structureof organiccompounds

DENDRAL 445

playinggrandmasterchess

human 30,000

processing avisual scene

human ???

Page 55: Knowledge-Based Systems Expert Systemscobweb.cs.uga.edu/~potter/kbs/VaTechECE4524.pdf · MYCIN uses IF-THEN rules Each rule has a certainty factor associated with it (a measure of

9expert.doc EE 4524 Page 55

So many rules!

Human experts are not very goodat writing rules

What if the computer could learnits own rules?!

Page 56: Knowledge-Based Systems Expert Systemscobweb.cs.uga.edu/~potter/kbs/VaTechECE4524.pdf · MYCIN uses IF-THEN rules Each rule has a certainty factor associated with it (a measure of

9expert.doc EE 4524 Page 56

This was tried with PLANT/ds(Michalski & Chilausky, 1981,

Illinois)

Rules for diagnosing soybeandiseases were generated fromexamples that were correctlyclassified by disease type by ahuman expert

Surprise! (not really)

Machine-derived rules performedbetter than the rules given bythe human expert

original human rules 83% correctimproved human rules 93%machine-derived rules 99%

Page 57: Knowledge-Based Systems Expert Systemscobweb.cs.uga.edu/~potter/kbs/VaTechECE4524.pdf · MYCIN uses IF-THEN rules Each rule has a certainty factor associated with it (a measure of

9expert.doc EE 4524 Page 57

How did the machine "learn" thecorrect rules?

data was collected for 350 sick plantsthought to suffer from one of 15diseases

the plant expert characterized eachdiseased plant using 35 differentfeatures(each plant was represented as a pointin a 35-dimensional space)

the plant expert divided the 350 data pointsinto 15 different classes (one class perdisease)

an inductive learning program generalizedfrom the given points to find simple rulesto describe each class(this is called learning from examples)

Page 58: Knowledge-Based Systems Expert Systemscobweb.cs.uga.edu/~potter/kbs/VaTechECE4524.pdf · MYCIN uses IF-THEN rules Each rule has a certainty factor associated with it (a measure of

9expert.doc EE 4524 Page 58

the 15 rules (one rule for eachclass) were put into theknowledge base

these rules were tested using newcases of diseased plants

We could say that the machine"acquired knowledge" byexamining the given examples

The system "learns" the necessaryrules by performing inductiveinference (generalization) oversets of examples

Machine learning is important forbuilding large-scale expertsystems

Page 59: Knowledge-Based Systems Expert Systemscobweb.cs.uga.edu/~potter/kbs/VaTechECE4524.pdf · MYCIN uses IF-THEN rules Each rule has a certainty factor associated with it (a measure of

9expert.doc EE 4524 Page 59

Knowledge-based systems:summary

Knowledge-based systems areways to capture and use theknowledge of human experts

Knowledge-based systems need aknowledge base and areasoning mechanism

IF-THEN rules are common, butother knowledge-representations are possible(eg., semantic nets)

Machine learning methods canhelp with large knowledgebases

More commercial successes herethan any other part of AI

Page 60: Knowledge-Based Systems Expert Systemscobweb.cs.uga.edu/~potter/kbs/VaTechECE4524.pdf · MYCIN uses IF-THEN rules Each rule has a certainty factor associated with it (a measure of

9expert.doc EE 4524 Page 60

Knowledge-based systems:limitations

Knowledge-base generation andmaintenance are difficult chores

Knowledge-based systems "know"only the things in the knowledgebase

They do not know how their ruleswere developed

They do not know when to breaktheir own rules

They do not look at problems fromdifferent perspectives

Most cannot reason at multiplelevels

They typically cannot learn fromtheir own experiences