Useful Techniques in Artificial Intelligence

-

Introduction

Cybernetics,University of Reading

WhiteknightsReading

UK

Cranfield University, 16th November 2005

PRESENTED BY: Dr WILL BROWNE

Picture of Lt Commander Data

This 1100 spin Bosch machine is incredibly quiet and positively high-end. It haseverything you would expect to find on a Bosch including exclusive features likethe 3D AquaSpa wash system with Fuzzy Control.

Stanley

$2 million Prize awarded to Stanford Racing TeamFive teams completed the Grand Challenge; four of them under the 10 hour limit. The Stanford Racing Team took the prize with a winning time of 6 hours, 53 minutes.

The SRT software system employs a number of advanced techniques from the field of artificial intelligence, such as probabilistic graphical models and machine learning.

http://www.darpa.mil/grandchallenge/index.asphttp://www.darpa.mil/grandchallenge/gallery.asp

http://en.wikipedia.org/wiki/Darpa_grand_challenge

Aim

To introduce the field of artificial intelligence,

so that it is possible to

Determine if an artificial intelligence technique is useful for a problem

and be able to

Select an appropriate technique for further investigation.

Objective

• Introduction to Artificial Intelligence

• Generic function of Artificial Intelligence tools

• Review of major techniques

• Benefit and pitfalls of applying these tools.

Contents

• Applications of Techniques

• Description of Artificial Intelligence Field

• Function of Important Techniques

• Benefit and Pitfalls of Applying Techniques

• Summary

Finance & Business

• Predict stock market trends

• Insurance/credit risk assessment

• Fraud detection

Industry

• Communication: mobile phone ground station & satellite networks

• Scheduling of work, transport, crane operations and so on

• Routing of computer networks.

INTELSAT operates a fleet of 19 satellites

Engineering

• Optimisation of route planning

• Design of complex structures

• Process optimisation

Control

• Domestic appliances, such as Microwave ovens

• Traffic flows

• Aircraft flight manoeuvres

Academia

• Game playing, e.g., chess

• Robotic football

• Test problems, e.g., iterated prisoner’s dilemma.

“Definition” of AI

Artificial :-

easily understood

Artificial Intelligence :-

whole concept can be discussed

Intelligence :-

easy to recognise

hard to define

Artificial

• Not Human, plant or animal

• Computer-based

(workstation, PC, parallel-computer or Mac)

• Computer programs

Artificial Intelligence

• Enable computers to perceive, reason and act.

• Do jobs that currently humans do better.

• Artificial Intelligence is what Artificial Intelligence researchers study.

Intelligence

• Intelligence is the ability to store, retrieve and act on data - efficiently and effectively.

• Intelligence has insight and can go beyond problem definition - but not experience?

• True intelligence does not exist!

“How do you speak ‘Alien’?”

Programme Languages

• Assembler

• C, C++, Java and FORTRAN

• Lisp, Small Talk and PROLOG

• Shells, e.g., G2 Expert System

• Toolboxes, e.g., Neural Networks in Matlab.

Function

NOT RELIANT UPON MATHEMATICAL DESCRIPTION

OF DOMAIN.

(stochastic)

• May include mathematics within technique

• May be similar to mathematical techniques

Functionality

Search Optimisation

Modelling

Knowledge-handling

Routing Scheduling

Visualisation Design

Querying Learning

Game-playing Adaptive-Control

Rule-Induction

Data-Access Data-Manipulation

Prediction Diagnosis

Function Summary

EXPLORE v EXPLOIT

EFFICIENTLY AND EFFECTIVELY

Functional Division of AI

Modelling -- Explore

Knowledge-Based -- Exploit

Optimisation -- Explore then

Exploit

Advanced -- Explore &

Exploit

Theoretical Division of AI

ARTIFICIAL INTELLIGENCE TECHNIQUES

LEARNING

GENETIC EVOLUTIONARY COMPUTATION NEURAL NETWORKS

LEARNING CLASSIFIER SYSTEMS

INTELLIGENT AGENTS(inc. Artificial Life)

IMMUNESYSTEMS

CELLULARAUTOMATA

KNOWLEDGE BASED

ExpertSystems

DecisionSupport

ENUMERATIVES

GUIDEDNON-GUIDED

Backtracking Branch &Bound

DynamicProgramming

Case BasedReasoning

FUZZY LOGIC

GUIDED

NON-GUIDED

Las Vegas

TabuSearch Simulated

Annealing

GENETIC ALGORITHMS GENETICPROGRAMMING

EVOLUTION STRATEGIES& PROGRAMMING

Hopfiled KohonenMaps

MultilayerPerceptrons

ANTCOLONY

HILL CLIMBING

REINFORCEMENT LEARNING

STATE-BASED

Knowledge-Based:

Expert Systems

What: Capture and reason about knowledge (especially human) in a transparent form.

How: Store of rules and information (the knowledge base)

Reason about information (inference engine).

Where: Rolling Mill Expert System project.

Satellite control/maintenance.

IF Temp < 400 oC THEN Rolling is Poor

Knowledge-Based: Case Based Reasoning (CBR)

What: Past examples (cases) used to reason about novel examples.

How: Store of cases and information Reason and interpolate information Update, maintain and repair cases.

Where: Decision support type systems.Initial bridge design selection.

Temp

400 oC

Rolling

Poor

Temp

450 oC

Rolling

Good

Temp

430 oC

Rolling

?

Enumerative:

Branch & Bound

What: Knowledge stored in decision trees. E.g., ID3 and C4.5

How: Domain is classified into sections

Tree of decisions is formed.

Where: Insurance fraud detection

Credit assessment.

Age > 25

T F

Sex = F

T F T F

250 300 300 425

Fuzzy Logic

What: Grey or fuzzy (i.e. human) thinking in computers.

How: Member sets formed to classify inputsOverlap of sets allows imprecise logic.

Where: Domestic appliance ‘intelligence’, e.g., washing machines & microwaves.

5.2 5.6 5.10 6.2Height

Distribution in department F M

Fuzzy Logic

What: Grey or fuzzy (i.e. human) thinking in computers.

How: Member sets formed to classify inputsOverlap of sets allows imprecise logic.

Where: Domestic appliance ‘intelligence’, e.g., washing machines & microwaves.

2 4 6 8Weight

Detergent : Water ratio

Silk Wool

Learning:

Guided Search

What: Optimisation techniques that avoid being trapped in local optima.

How: Simulated AnnealingProbability of accepting new search pointProbability reduced near to optimum.

How: Tabu SearchCan not search previously visited pointTherefor will not become stuck.

Where: Optimisation problems, where domain is described by a function.

http://www.exatech.com/Optimization/optimization.htm

Learning: Genetic Evolutionary Computation

What: Uses evolution to optimise fitness (function) of solution.

How: 1. Population of solutions created2. Fitness of each solution evaluated3. Best solutions mated for new

population4. Repeated until optimum solution.

Where: Design optimisationStock market investmentAutonomous programme development

Learning: Genetic Evolutionary Computation

Genetic Algorithms:Optimise numeric solution of fitness function.

Learning Classifier Systems:Optimise the co-operation of rules for solving and input/output thickness function.

Genetic Programming:Optimise the interaction of code to solve a programming function.

Evolutionary Systems:Optimise the solution based on a behavioural (phenotypic) instead of genetic (genotypic) level.

-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

1 1.5 2 2.5 3

F(x) = cos(x) + sin(x2) : 1 < x< 3

GA: j1 = 00010001

j2 = 01110001

j3 = 10010101

GP: j1 = sin(x) + 2sin(x2)

j2 = sin(x) + 2sin(x)cos(x)

j3 = sin(x) - 2sin(x)cos(x)

Intelligent-Agents:

Cellular Automata

What: Autonomous individuals (cells) reacting to state of neighbouring individuals - governed by rules.

How: Grid of individuals initiatedBehaviour rules introduced(e.g., if > 3 neighbours on, then on)Iteration until stable pattern emerges.

Where: Cast and mould designScreensavers!

Neural Networks: Back-Propagation

What: Mimic the function of the human brain within a computer.

How: Nodes (representing neurons) are linked to other nodes via connections (representing synapses)Nodes send messages to their output (firing) when a threshold from their inputs has been reached.

Where: Modelling of industrial systemsSpeech recognition programs.

INPUTS OUTPUTS

INPUTLAYER

HIDDENLAYER

OUTPUTLAYER

NODE

CONNECTION

Neural Networks: Self-Organising-Maps

What: Mimic the function of the human brain within a computer. To determine input relations (instead of input-output relationships).

How: Nodes are linked to other nodes via connectionsNetwork of nodes autonomously adjusts to represent input patterns.

Where: Fault diagnosis of industrial systemsGrowing patterns in crops

Technique Selection

Overall Strategy - Explore (search) or

Exploit (optimise)

Representation - Required

transparency

Learning - Domain / fitness

function known?

Supervision - Feedback from

domain available?

No Free Lunch Theorem

“...all algorithms that search for an extreme of a cost function perform exactly the same, according to any performance measures, when averaged over all possible cost functions.”

[Wolpert and Macready 96]

No Free Lunch Theorem

Reasons why theorem does not hold in practical situations:

• Inclusion of domain knowledge • Co-adaptation algorithms • Domain specific algorithms

• Non-infinite populations• Resampling is important• Representation style is important in

specific domains

[Wilson 97]

Interpolate & Extrapolate

• Aliasing

• Incomplete picture

-0.2

0

0.2

0.4

0.6

0.8

1

1.2

0 1 2 3

Learnt

Actual

x

x

x

x

-2

-1.8

-1.6

-1.4

-1.2

-1

-0.8

-0.6

-0.4

-0.2

0

0.7 1.2 1.7 2.2 2.7

xxxxx x x

Garbage In = Garbage Out

• Often blind acceptance of inputs• Often blind generation of outputs

• Practical need to:

Verify

Validate

Test

Lack of Transparency

• “Black Box” techniques, such as Neural Networks

• Semi-transparent techniques, such as Branch & Bound, become difficult for human interpretation with large problems

• Transparent techniques, such as Expert Systems, become difficult for human interpretation with very large problems - above 1000 rules, the logic chain becomes huge.

Benefits• Not reliant upon the mathematical

description of the domain

• Speed, efficient solution production

• New/novel answers, effective solutions produced

• Direct areas of further research (human or conventional techniques)

• Hybridisation of techniques is possible

• Cost, wide range of options available

Conclusion

• Useful tools to complement existing techniques

• Multiple uses from exploring to exploiting the domains of problems

• Beneficial in efficiently and effectively obtaining solutions to problems