www.abdn.ac.uk/sras

Artificial IntelligenceIn the Real World

Computing Science

University of Aberdeen

www.abdn.ac.uk/sras

Artificial IntelligenceIn the Real World

Artificial IntelligenceIn the Movies

www.abdn.ac.uk/sras

Artificial IntelligenceIn the Real World

Artificial IntelligenceIn the Movies

www.abdn.ac.uk/sras

Artificial IntelligenceIn the Real World

Artificial IntelligenceIn the Movies

?

www.abdn.ac.uk/sras

Artificial Intelligence Began in 1956…

• Great expectations…

““Machines will be capable, Machines will be capable,

within twenty years, within twenty years,

of doing any work that a man of doing any work that a man

can do.”can do.”

Herbert Simon, 1965.

www.abdn.ac.uk/sras

““Machines will be capable, Machines will be capable,

within twenty years, within twenty years,

of doing any work that a man can do.”of doing any work that a man can do.”

Herbert Simon, 1965.

What Happened?

www.abdn.ac.uk/sras

• Machines can’t do everything a man can do…• People thought machines could replace humans…

instead they are usually supporting humans

““Machines will be capable, Machines will be capable,

within twenty years, within twenty years,

of doing any work that a man can do.”of doing any work that a man can do.”

Herbert Simon, 1965.

What Happened?

www.abdn.ac.uk/sras

• Machines can’t do everything a man can do…• People thought machines could replace humans…

instead they are usually supporting humans– Healthcare, Science, Government, Business, Military…

““Machines will be capable, Machines will be capable,

within twenty years, within twenty years,

of doing any work that a man can do.”of doing any work that a man can do.”

Herbert Simon, 1965.

What Happened?

www.abdn.ac.uk/sras

• Machines can’t do everything a man can do…• People thought machines could replace humans…

instead they are usually supporting humans– Healthcare, Science, Government, Business, Military…

• Most difficult problems are solved by human+machine– astronomy, nuclear physics, genetics, maths, drug discovery…

““Machines will be capable, Machines will be capable,

within twenty years, within twenty years,

of doing any work that a man can do.”of doing any work that a man can do.”

Herbert Simon, 1965.

What Happened?

www.abdn.ac.uk/sras

Neural Networks

• Neural Networks are a popular Artificial Intelligence technique

• Used in many applications which help humans

• The idea comes from trying to copy the human brain…

www.abdn.ac.uk/sras

Fascinating Brain Facts…• 100,000,000,000 = 1011 neurons -100 000 are irretrievably lost each day!

• Each neuron connects to 10,000 -150,000 others

• Every person on planet make 200 000 phone calls

– same number of connections as in a single human brain in a day

• Grey part folded to fit - would cover surface of office desk

• The gray cells occupy only 5% of our brains

– 95% is taken up by the communication network between them

• About 2x106km of wiring (to the moon and back twice)

• Pulses travel at more than 400 km/h (250 mph)

• 2% of body weight… but consumes 20% of oxygen

• All the time! Even when sleeping

• What about copying neurons in Computers?

www.abdn.ac.uk/sras

Artificial Neural Network (ANN)

• loosely based on biological neuron

• Each unit is simple, but many connected in a complex network

• If enough inputs are received– Neuron gets “excited”

– Passes on a signal, or “fires”

• ANN different to biological:– ANN outputs a single value

– Biological neuron sends out a complex series of spikes

– Biological neurons not fully understoodImage from Purves et al., Life: The Science of Biology, 4th Edition, by Sinauer

Associates and WH Freeman

www.abdn.ac.uk/sras

Now play with the flash animation to see how synapses work

http://www.mind.ilstu.edu/curriculum/neurons_intro/flash_summary.php?modGUI=232&compGUI=1828&itemGUI=3160

www.abdn.ac.uk/sras

The Perceptron

add

weight1

output

input1

input2

input3

input4

weight4

(threshold)

weight2

wei

ght 3

www.abdn.ac.uk/sras

The Perceptron

add

weight1

output

input1

input2

input3

input4

weight4

(threshold)

weight2

wei

ght 3

Save Graph and Data

www.abdn.ac.uk/sras

The Perceptron

Note: example from Alison Cawsey

student first last year

male works hard

Lives in halls

First this year

1 Richard 1 1 0 1 0

2 Alan 1 1 1 0 1

3 Alison 0 0 1 0 0

4 Jeff 0 1 0 1 0

5 Gail 1 0 1 1 1

6 Simon 0 1 1 1 0

Save Graph and Data

www.abdn.ac.uk/sras

The Perceptron

add

0.2

_output

First last year _

Male_

_hardworking _

Lives in halls

0.2Threshold

= 0.5

0.2

0.2

Note: example from Alison Cawsey

student First last year male works hard Lives in halls First this year

1 Richard 1 1 0 1 0

www.abdn.ac.uk/sras

The Perceptron

add

0.15

_output

First last year _

Male_

_hardworking _

Lives in halls

0.15Threshold

= 0.5

0.15

0.2

Note: example from Alison Cawsey

student First last year male works hard Lives in halls First this year

1 Richard 1 1 0 1 0

www.abdn.ac.uk/sras

The Perceptron

add

0.15

_output

First last year _

Male_

_hardworking _

Lives in halls

0.15Threshold

= 0.5

0.15

0.2

Note: example from Alison Cawsey

student First last year male works hard Lives in halls First this year

2 Alan 1 1 1 0 1

www.abdn.ac.uk/sras

The Perceptron

add

0.2

_output

First last year _

Male_

_hardworking _

Lives in halls

0.15Threshold

= 0.5

0.2

0.25

Note: example from Alison Cawsey

student First last year male works hard Lives in halls First this year

2 Alan 1 1 1 0 1

www.abdn.ac.uk/sras

The Perceptron

add

0.2

_output

First last year _

Male_

_hardworking _

Lives in halls

0.15Threshold

= 0.5

0.2

0.25

Note: example from Alison Cawsey

student First last year male works hard Lives in halls First this year

3 Alison 0 0 1 0 0

www.abdn.ac.uk/sras

The Perceptron

add

0.2

_output

First last year _

Male_

_hardworking _

Lives in halls

0.15Threshold

= 0.5

0.2

0.25

Note: example from Alison Cawsey

student First last year male works hard Lives in halls First this year

4 Jeff 0 1 0 1 0

www.abdn.ac.uk/sras

The Perceptron

add

0.2

_output

First last year _

Male_

_hardworking _

Lives in halls

0.15Threshold

= 0.5

0.2

0.25

Note: example from Alison Cawsey

student First last year male works hard Lives in halls First this year

5 Gail 1 0 1 1 1

www.abdn.ac.uk/sras

The Perceptron

add

0.2

_output

First last year _

Male_

_hardworking _

Lives in halls

0.15Threshold

= 0.5

0.2

0.25

Note: example from Alison Cawsey

student First last year male works hard Lives in halls First this year

6 Simon 0 1 1 1 0

www.abdn.ac.uk/sras

The Perceptron

add

0.2

_output

First last year _

Male_

_hardworking _

Lives in halls

0.10Threshold

= 0.5

0.15

0.20

Note: example from Alison Cawsey

student First last year male works hard Lives in halls First this year

6 Simon 0 1 1 1 0

www.abdn.ac.uk/sras

The Perceptron

add

0.2

_output

First last year _

Male_

_hardworking _

Lives in halls

0.10Threshold

= 0.5

0.15

0.20

Note: example from Alison Cawsey

student First last year male works hard Lives in halls First this year

1 Richard 1 1 0 1 0

www.abdn.ac.uk/sras

The Perceptron

add

0.2

_output

First last year _

Male_

_hardworking _

Lives in halls

0.10Threshold

= 0.5

0.15

0.20

Note: example from Alison Cawsey

student First last year male works hard Lives in halls First this year

2 Alan 1 1 1 0 1

www.abdn.ac.uk/sras

The Perceptron

add

0.2

_output

First last year _

Male_

_hardworking _

Lives in halls

0.10Threshold

= 0.5

0.15

0.20

Note: example from Alison Cawsey

student First last year male works hard Lives in halls First this year

3 Alison 0 0 1 0 0

www.abdn.ac.uk/sras

The Perceptron

add

0.2

_output

First last year _

Male_

_hardworking _

Lives in halls

0.10Threshold

= 0.5

0.15

0.20

Note: example from Alison Cawsey

student First last year male works hard Lives in halls First this year

4 Jeff 0 1 0 1 0

www.abdn.ac.uk/sras

The Perceptron

add

0.2

_output

First last year _

Male_

_hardworking _

Lives in halls

0.10Threshold

= 0.5

0.15

0.20

Note: example from Alison Cawsey

student First last year male works hard Lives in halls First this year

5 Gail 1 0 1 1 1

www.abdn.ac.uk/sras

The Perceptron

add

0.25

_output

First last year _

Male_

_hardworking _

Lives in halls

0.15Threshold

= 0.5

0.15

0.25

Note: example from Alison Cawsey

student First last year male works hard Lives in halls First this year

5 Gail 1 0 1 1 1

www.abdn.ac.uk/sras

The Perceptron

add

0.25

_output

First last year _

Male_

_hardworking _

Lives in halls

0.15Threshold

= 0.5

0.15

0.25

Note: example from Alison Cawsey

student First last year male works hard Lives in halls First this year

6 Simon 0 1 1 1 0

www.abdn.ac.uk/sras

The Perceptron

add

0.25

_output

First last year _

Male_

_hardworking _

Lives in halls

0.10Threshold

= 0.5

0.10

0.20

Note: example from Alison Cawsey

student First last year male works hard Lives in halls First this year

6 Simon 0 1 1 1 0

www.abdn.ac.uk/sras

The Perceptron

add

0.25

_output

First last year _

Male_

_hardworking _

Lives in halls

0.10Threshold

= 0.5

0.10

0.20

Note: example from Alison Cawsey

Finished

www.abdn.ac.uk/sras

The Perceptron

add

0.25

_output

First last year _

Male_

_hardworking _

Lives in halls

0.10Threshold

= 0.5

0.10

0.20

Note: example from Alison Cawsey

FinishedReady to try unseen examples

www.abdn.ac.uk/sras

The Perceptron

add

0.25

_output

First last year _

Male_

_hardworking _

Lives in halls

0.10Threshold

= 0.5

0.10

0.20

Note: example from Alison Cawsey

student First last year male works hard Lives in halls First this year

James 0 1 0 1 ?

www.abdn.ac.uk/sras

The Perceptronadd

0.25

_output

0.10Threshold

= 0.5

0.10

0.20

• Simple perceptron works ok for this example but sometimes will never find weights that fit everything

• In our example:– Important: Getting a first last year, Being hardworking

– Not so important: Male, Living in halls

• Suppose there was an “exclusive or” - – Important: (male) OR (live in halls), but not both

– Can’t capture this relationship

www.abdn.ac.uk/sras

Stock Exchange ExampleCompany Name Company less

than 2 years old

Paid dividend >10% last year

Share price increases in following year

1 Robot Components Ltd. 1 1 0

2 Silicon Devices 1 0 1

3 Bleeding Edge Software

0 0 0

4 Human Interfaces Inc. 1 1 0

5 Data Management Inc. 0 1 1

6 Intelligent Systems 1 1 0

www.abdn.ac.uk/sras

Multilayer Networks

• We saw: perceptron can’t capture relationships among inputs

• Multilayer networks can capture complicated relationships

www.abdn.ac.uk/sras

Stock Exchange Example

Hidden Layer

www.abdn.ac.uk/sras

Neural Net example: ALVINN• Autonomous vehicle controlled by Artificial Neural Network

• Drives up to 70mph on public highways

Note: most images are from the online slides for Tom Mitchell’s book “Machine Learning”

www.abdn.ac.uk/sras

Neural Net example: ALVINN• Autonomous vehicle controlled by Artificial Neural Network

• Drives up to 70mph on public highways

• Note: most images are from the online slides for Tom Mitchell’s book “Machine Learning”

www.abdn.ac.uk/sras

ALVINN

Input is 30x32 pixels= 960 values

1 input pixel

4 hidden units

30 output units

Sharp right

Straight ahead

Sharp left

www.abdn.ac.uk/sras

ALVINN

Input is 30x32 pixels= 960 values

1 input pixel

4 hidden units

30 output units

Sharp right

Straight ahead

Sharp left

Learning means adjusting weight

values

www.abdn.ac.uk/sras

ALVINN

Input is 30x32 pixels= 960 values

1 input pixel

4 hidden units

30 output units

Sharp right

Straight ahead

Sharp left

www.abdn.ac.uk/sras

ALVINN

www.abdn.ac.uk/sras

ALVINN

This shows one hidden node

Input is 30x32 array of pixel values = 960 values Note: no special visual processing

Size/colour corresponds to weight on link

www.abdn.ac.uk/sras

ALVINN

This shows one hidden node

Input is 30x32 array of pixel values = 960 values Note: no special visual processing

Size/colour corresponds to weight on link

Output is array of 30 values This corresponds to steering

instructions E.g. hard left, hard right

www.abdn.ac.uk/sras

• Let’s try a more complicated example with the program…

• In this example we’ll get the program to help us to build the neural network

www.abdn.ac.uk/sras

Neural Network Applications• Particularly good for pattern recognition

www.abdn.ac.uk/sras

Neural Network Applications• Particularly good for pattern recognition

– Sound recognition – voice, or medical

www.abdn.ac.uk/sras

Neural Network Applications• Particularly good for pattern recognition

– Sound recognition – voice, or medical– Character recognition (typed or handwritten)

www.abdn.ac.uk/sras

Neural Network Applications• Particularly good for pattern recognition

– Sound recognition – voice, or medical– Character recognition (typed or handwritten)– Image recognition (e.g. human faces)

www.abdn.ac.uk/sras

Neural Network Applications• Particularly good for pattern recognition

– Sound recognition – voice, or medical– Character recognition (typed or handwritten)– Image recognition (e.g. human faces)– Robot control - hand-arm-block.mpg

www.abdn.ac.uk/sras

Neural Network Applications• Particularly good for pattern recognition

– Sound recognition – voice, or medical– Character recognition (typed or handwritten)– Image recognition (e.g. human faces)– Robot control– ECG pattern – had a heart attack?

www.abdn.ac.uk/sras

Neural Network Applications• Particularly good for pattern recognition

– Sound recognition – voice, or medical– Character recognition (typed or handwritten)– Image recognition (e.g. human faces)– Robot control– ECG pattern – had a heart attack?– Application for credit card or mortgage

www.abdn.ac.uk/sras

Neural Network Applications• Particularly good for pattern recognition

– Sound recognition – voice, or medical– Character recognition (typed or handwritten)– Image recognition (e.g. human faces)– Robot control– ECG pattern – had a heart attack?– Application for credit card or mortgage– Data Mining on Customers

www.abdn.ac.uk/sras

Neural Network Applications• Particularly good for pattern recognition

– Sound recognition – voice, or medical

– Character recognition (typed or handwritten)

– Image recognition (e.g. human faces)

– Robot control

– ECG pattern – had a heart attack?

– Application for credit card or mortgage

– Other types of Data Mining - Science

www.abdn.ac.uk/sras

Neural Network Applications• Particularly good for pattern recognition

– Sound recognition – voice, or medical– Character recognition (typed or handwritten)– Image recognition (e.g. human faces)– Robot control– ECG pattern – had a heart attack?– Application for credit card or mortgage– Spam filtering

www.abdn.ac.uk/sras

Neural Network Applications• Particularly good for pattern recognition

– Sound recognition – voice, or medical– Character recognition (typed or handwritten)– Image recognition (e.g. human faces)– Robot control– ECG pattern – had a heart attack?– Application for credit card or mortgage– Shape in go

www.abdn.ac.uk/sras

Neural Network Applications• Particularly good for pattern recognition

– Sound recognition – voice, or medical

– Character recognition (typed or handwritten)

– Image recognition (e.g. human faces)

– Robot control

– ECG pattern – had a heart attack?

– Application for credit card or mortgage

– Data Mining on Customers

– Other types of Data Mining

– Spam filtering

– Shape in Go… and many more!

www.abdn.ac.uk/sras

What are Neural Networks Good For?

• When training data is noisy, or inaccurate– E.g. camera or microphone inputs

• Very fast performance once network is trained• Can accept input numbers from sensors directly

– Human doesn’t need to interpret them first

www.abdn.ac.uk/sras

• Need a lot of data – training examples

• Training time could be very long– This is the big problem for large networks

• Network is like a “black box”– A human can’t look inside and understand what has been learnt

– Logical rules would be easier to understand

Disadvantages?