Lectures 5,6

MACHINE LEARNING

EXPERT SYSTEMS

Contents

Machine learning Knowledge representation Expert systems

INDUCTION OF DECISION TREES FROM DATA

Sunny Rain

Overcast

Outlook

Humidity Wind

High Normal Strong Weak

Outlook Temp. Humid. Wind Sport?

1 Sunny Hot High Weak No

2 Sunny Hot High Strong No

3 Overcast Hot High Weak Yes

4 Rain Mild High Weak Yes

5 Rain Cold Normal Weak Yes

6 Rain Cold Normal Strong No

7 Overcast Cold Normal Strong Yes

8 Sunny Mild High Weak No

9 Sunny Cold Normal Weak Yes

10 Rain Mild Normal Weak Yes

11 Sunny Mild Normal Strong Yes

12 Overcast Mild High Strong Yes

13 Overcast Hot Normal Weak Yes

14 Rain Mild High Strong No

Decision trees

Sport=No Sport=Yes

Data from credit history of loan applications

A simplified tree… But how to do it?

The induction algorithm ID3

Partially constructed decision trees

STEP 1

STEP 2

A heuristic problem

HOW TO SELECT THE BEST PROPERTY?

Approximate trees

High Normal

Humidity

85%Outlook

75%100%

Sunny Not Sunny

Outlook Temp. Humid. Wind Sport?

1 Sunny Hot High Weak No

2 Sunny Hot High Strong No

3 Overcast Hot High Weak Yes

4 Rain Mild High Weak Yes

5 Rain Cold Normal Weak Yes

6 Rain Cold Normal Strong No

7 Overcast Cold Normal Strong Yes

8 Sunny Mild High Weak No

9 Sunny Cold Normal Weak Yes

10 Rain Mild Normal Weak Yes

11 Sunny Mild Normal Strong Yes

12 Overcast Mild High Strong Yes

13 Overcast Hot Normal Weak Yes

14 Rain Mild High Strong No

CLASSIFICATION SYSTEMS

A full classification system

Pattern recognition

Patterns:– images, personal records, driving habits, etc.

Representation:– vector of features (inputs to a neural network)

Pattern classification: – Classify a pattern to one of the given classes

> classifier < Marks

> classifier < not Marks

> classifier < not Marks

> classifier < not Marks

> classifier < Marks

> classifier < not Marks

Classifier training

Classifier application

> Classifier > Marks

Note: The test image does not appear in the training data

LEARNING IN GENERAL

The data and the goals

We begin with a collection of positive (and usually negative) examples of a target class (a concept to be learnt)

The goal is to infer a general definition that will allow the learner to recognize future instances of the class

Knowledge representation

Positive and negative examples can be represented, e.g., in predicate calculus

Two positive instances of the concept of “ball” can be expressed as follows:size(obj1,small) color(obj1,red) shape(obj1,round) size(obj2,large) color(obj2,red) shape(obj2,round)

The general concept of “ball” could be defined by:

size(X,Y) color(X,Z) shape(X,round)

where any sentence that unifies with this general definition represents a ball

A general model of the learning

process

A set of operations

Given a set of training instances, the learner must construct a generalization, heuristic rule or plan that satisfies its goals

The concept space

Representation language and the operations define a space of potential concept definitions

The learner must search this space to find the desired concept

Heuristic search

Learning programs must commit to a direction and order of search, as well as…

…to the use of available data and heuristics to search efficiently

PATRICK WINSTON’S PROGRAM ON LEARNING CONCEPTS

Examples and near misses for the concept “arch”

Generalization of descriptions to include multiple examples (I)

Generalization of descriptions to include multiple examples (II)

Specialization of a description to exclude a near miss

so that this can’t match

Starting with the original

we add special constraints

A BRIEF HISTORY OF AI REPRESENTATIONAL SCHEMES

Semantic network developed by Collins & Quillian

in their research on human information

storage and response times

Network representation of properties of…

…snow and ice

…three definitions of the word “plant”

Three planes representing…

Intersection path between “cry” and “comfort” (Quillian 1967)

Case frame representation of the sentence “Sarah fixed thechair with glue.”

Conceptual dependency theory of four primitive conceptualizations

For example, all actions are assumed to reduce to one or more of the primitive ACTs listed below:

“John ate the egg”

“John prevented Mary from giving a book to

Bill”

Restaurant script

(Schank and Abelson 1977)

Restaurant script (continued)

FRAMES

A frame includes:

Frame identification information Its relationship to other frames Descriptors of requirements Procedural information on use

of the structure described Frame default information New instance information

Relationship to other frames

For instance, the “hotel phone” might be a special instance of “phone”, which might be an instance of a “communication device”

Descriptors of requirements

For instance, a chair has its seat between 20 and 40 cm from the floor, its back higher than 60 cm, etc.

These requirements may be used to determine when new objects fir the stereotype defined by the frame

Procedural information

An important feature of frames is the ability to attach procedural code to a slot

Frame default information

These are slot values that are taken to be true when no evidence to the contrary has been found

For instance, chairs have four legs, telephones are pushbutton, hotel beds are made by the staff

New instance information

Many frame slots may be left unspecified until given a value for a particular instance or when they are needed for some aspect of problem solving

For instance, the color of the bedspread may be left unspecified

Part of a frame description of a hotel room

“Specialization” indicates a pointer to a superclass

Spatial frame for viewing a cube (Minsky 1975)

CONCEPTUAL GRAPHS: A NETWORK LANGUAGE

Conceptual relations of

different arities

“Mary gave John the book”

“A dog named emma is brown”

Examples of restriction…

…join, and simplify operations

Inheritance in conceptual graphs

“Tom believes that Jane likes pizza”

This example shows the use of a propositional concept

RULE BASED EXPERT SYSTEMS

Architecture of a typical expert system for a particular problem

domain

Guidelines to determine whether a problem is appropriate for expert system solution (1)

The need for the solution justifies the cost and effort of building an expert system

Human expertise is not available in all situations where it is needed

The problem may be solved using symbolic reasoning

Guidelines to determine whether a problem is appropriate for expert system solution (2)

The problem domain is well structured and does not require commonsense reasoning

The problem may not be solved using traditional computing methods

Cooperative and articulate experts exist The problem is of proper size and scope

Reasoning with a typical expert system

The role of mental or conceptual models in problem solving

A small system for analysis of automotive problems

The and/or graph searched in the car diagnosis example

The production system at the start of a consultation in the car diagnostic example

Imagine that we want to get

information about spark plugs

The production system after Rule 1 has fired

The system after Rule 4 has fired

Note the stack-based approach to goal reduction

The following dialogue begins with the computer asking the user about the goals present in working memory

CASE BASED REASONING

Case based reasoners

Share a common structure For each new problem they

– Retrieve appropriate cases from memory– Modify a retrieved case so that it will apply

to the current situation– Apply the transformed case– Save the solution, with a record of success

or failure, for future use

Possible preference heuristics to help organize the storage and retrieval of cases (1) Goal-directed: Organize cases, at least in

part, by goal descriptions; Retrieve cases that have the same goal as the current situation

Salient-feature: Prefer cases that match the most important features or those matching the largest number of important features

Recency: Prefer cases used most recently

Possible preference heuristics to help organize the storage and retrieval of cases (2) Specify: Look for as exact as possible

matches of features before considering more general matches

Frequency: Check first the most frequently matched cases

Ease of adaptation: Use first cases most easily adapted to the current situation

Transformational analogy

Advantages of rule based approach

Disadvantages

Advantages of case based reasoning

Disadvantages

Combination of rule based and case based systems

3 Robotic Planning

ROBOTIC PLANNING

The blocks world

Predicate based representation

A number of truth relations or rules for robot’s performance

Portion of the search space

Goal state for the blocks world

Descriptor triples

Preconditions (P): Conditions the world must meet for an operator to be applied

Add List (A): Additions to the state description that are a result of applying the operator

Delete List (D): Items that are removed from a state description to create the new state when the operator is applied

Operators as triples of descriptions

A simple tree showing condition action rules

THANK YOU