Ch3 Ann Fs Presentation
-
Upload
vasukoneti5124 -
Category
Documents
-
view
213 -
download
0
Transcript of Ch3 Ann Fs Presentation
-
8/9/2019 Ch3 Ann Fs Presentation
1/32
28/01/15 1
Perceptron Perceptron is one of the earliest models of articial neuron.
It was proposed by osenblatt in 1!58.
It is a sin"le layer neural networ# whose wei"hts can be
trained to produce a correct tar"et $ector when presentedwith the correspondin" input $ector
%he trainin" techni&ue used is called the Perceptronlearning rule.
%he Perceptron "enerated "reat interest due to its ability to
generalizefrom its trainin" $ectors and wor# withrandomly distributed connections.
Perceptron are especially suited for problems in patternclassication.
-
8/9/2019 Ch3 Ann Fs Presentation
2/32
28/01/15 2
Perceptrons
'inear separability
( set of )2*+ patterns )x1,x2+ of two classes is linearly
separable if there e-ists a line on the )x1,x2+ plane
w0 w1x1 w2x2 0
eparates all patterns of one class from the otherclass
( perceptron can be built with
inputx0 1,x1,x2with wei"hts w0, w1, w2
ndimensional patterns )x1,,xn+ 3yperplane w0 w1x1 w2x2 wnxn 0
di$idin" the space into two re"ions 4an we "et the wei"hts from a set of sample patterns
If the problem is linearly separable, then 67 )byperceptron learnin"+
-
8/9/2019 Ch3 Ann Fs Presentation
3/32
28/01/15
LINEAR SEPARABILITY Definition:Two sets of points A and B in an n-dimensional space are
called linearly separable if n+1 real numbers w1, w2, w3, . . . ., wn+1exist,
such that eery point !x1, x2, . . . , xn"A satisfies and eery point !x1, x2,. . . , xn" B satisfies .
Absolute #inear $eparability Two sets of points A and B in an n-dimensional space are called linearly
separable if n+1 real numbers w1, w2, w3, . . . ., wn+1exist, such thateery point !x1, x2, . . . , xn" A satisfies and eery point !x1, x2, . . . , xn"B satisfies .
%wo nite sets of points ( and , in n9dimensional space whichare linear separable are also absolute linearly separable.
In "eneral, absolute linearly separable9: linearly separable
but if sets are nite, linearly separable absolutely linearlyseparable
-
8/9/2019 Ch3 Ann Fs Presentation
4/32
7-amples of linearly separableclasses
9 'o"ical AND function
patterns )bipolar+ decisionboundary
-1 -2 output w1 191 91 91 w2 1
91 1 91 w0 911 91 911 1 1 91 -1 -2 0
9 'o"ical OR function
patterns )bipolar+ decisionboundary
-1 -2 output w1 191 91 91 w2 1
91 1 1 w0 11 91 11 1 1 1 -1 -2 0
x
oo
o
x: class I (output = 1)o: class II (output = -1)
x
xo
x
x: class I (output = 1)o: class II (output = -1)
28/01/15 ;
-
8/9/2019 Ch3 Ann Fs Presentation
5/32
28/01/15 5
Perceptron
-
8/9/2019 Ch3 Ann Fs Presentation
6/32
28/01/15 ?
in"le 'ayer *iscrete Perceptron @etwor#s
)'*P+
&lass&1
&lass &2
x1
x2
i!. ".2 Illustration of t$e $'per plane (in t$is example a strai!$t line)
as decision oundar' for a two dimensional two-class patron classification prolem.
%o de$elop insi"ht into the beha$ior of a pattern classier, it isnecessary to plot a map of the decision re"ions in n9dimensionalspace, spanned by the n input $ariables. %he two decision re"ionsseparated by a hyper plane dened by
=
=n
i
iw
0
i0x
-
8/9/2019 Ch3 Ann Fs Presentation
7/32
28/01/15 A
'*P
&la
ss
&1
&la
ss
&2
()
&la
ss
&1
&la
ss
&2
(a)
*ecision oundar'
i! (a) + pair of linearl' separale patterns
() + pair of nonlinearl' separale patterns.
Bor the Perceptron to function properly, the two classes 41 and 42 must belinearly separable.
In Bi"..)a+, the two classes 41 and 42 are suCciently separated fromeach other to draw a hyper plane )in this it is a strai"ht line+ as thedecision boundary.
-
8/9/2019 Ch3 Ann Fs Presentation
8/32
28/01/15 8
'*P
-
8/9/2019 Ch3 Ann Fs Presentation
9/32
28/01/15 !
'*P
-
8/9/2019 Ch3 Ann Fs Presentation
10/32
28/01/15 10
*iscrete Perceptron trainin"
al"orithm
-
8/9/2019 Ch3 Ann Fs Presentation
11/32
28/01/15 11
(l"orithm continued..
-
8/9/2019 Ch3 Ann Fs Presentation
12/32
28/01/15 12
(l"orithm continued..
-
8/9/2019 Ch3 Ann Fs Presentation
13/32
28/01/15 1
7-ample=
uild the Perceptron networ# to realiDe fundamental lo"ic"ates, such as (@*, E and FE.
-
8/9/2019 Ch3 Ann Fs Presentation
14/32
28/01/15 1;
-
8/9/2019 Ch3 Ann Fs Presentation
15/32
28/01/15 15
-
8/9/2019 Ch3 Ann Fs Presentation
16/32
28/01/15 1?
-
8/9/2019 Ch3 Ann Fs Presentation
17/32
28/01/15 1A
-
8/9/2019 Ch3 Ann Fs Presentation
18/32
1 2 3 4 5 6 7 8 9 100
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Number of epochs
Error
1 2 3 4 5 6 7 8 9 100
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0.5
Number of epochs
Error
Bi". .5 %he 7rror prole durin" thetrainin" of Perceptron to learn input9
output relation of (@* "ate
Bi". .; %he 7rror prole durin" the trainin"
of Perceptron to learn input9output relation ofE "ate
esults
28/01/15 18
-
8/9/2019 Ch3 Ann Fs Presentation
19/32
0 5 10 15 20 25 30 35 40 45 500.5
1
1.5
2
2.5
Number of epochs
Error
Bi". .? %he 7rror prole durin" the trainin" ofPerceptron to learn input9output relation of FE"ate
28/01/15 1!
-
8/9/2019 Ch3 Ann Fs Presentation
20/32
28/01/15 20
Single-Layer Continuous Perceptronnetwors)'4P+
%he acti$ation function that is used in modelin" the4ontinuous Perceptron is si"moidal, which isdi>erentiable.
%he two ad$anta"es of usin" continuous acti$ationfunction are )i+ ner control o$er the trainin" procedureand )ii+ di>erential characteristics of the acti$ationfunction, which is used for computation of the error"radient.
%his "i$es the scope to use the "radients in modifyin"
the wei"hts. %he "radient or steepest descent method isused in updatin" wei"hts startin" from any arbitrarywei"ht $ector G, the "radient 7)G+ of the current errorfunction is computed.
-
8/9/2019 Ch3 Ann Fs Presentation
21/32
28/01/15 21
Single-Layer Continuous Perceptronnetwors
-
8/9/2019 Ch3 Ann Fs Presentation
22/32
28/01/15 22
'4P
-
8/9/2019 Ch3 Ann Fs Presentation
23/32
28/01/15 2
'4P
-
8/9/2019 Ch3 Ann Fs Presentation
24/32
28/01/15 2;
'4P
-
8/9/2019 Ch3 Ann Fs Presentation
25/32
28/01/15 25
'4P
-
8/9/2019 Ch3 Ann Fs Presentation
26/32
28/01/15 2?
Perceptron Con!ergenceT"eore#
-
8/9/2019 Ch3 Ann Fs Presentation
27/32
28/01/15 2A
Perceptron Con!ergenceT"eore#
-
8/9/2019 Ch3 Ann Fs Presentation
28/32
28/01/15 28
Perceptron Con!ergenceT"eore#
-
8/9/2019 Ch3 Ann Fs Presentation
29/32
-
8/9/2019 Ch3 Ann Fs Presentation
30/32
28/01/15 0
Perceptron Con!ergenceT"eore#
-
8/9/2019 Ch3 Ann Fs Presentation
31/32
28/01/15 1
-
8/9/2019 Ch3 Ann Fs Presentation
32/32
28/01/15 2
'imitations of Perceptron
%here are limitations to the capabilities of Perceptronhowe$er.
%hey will learn the solution, if there is a solution to be found. Birst, the output $alues of a Perceptron can ta#e on only
one of two $alues )%rue or Balse+. econd, Perceptron can only classify linearly separablesets
of $ectors. If a strai"ht line or plane can be drawn toseparate the input $ectors into their correct cate"ories, theinput $ectors are linearly separable and the Perceptron willnd the solution.
If the $ectors are not linearly separable learnin" will ne$erreach a point where all $ectors are classied properly.
%he most famous e-ample of the PerceptionHs inability tosol$e problems with linearly non9separable $ectors is theboolean FE realiDation.