Scholastic Book NefghuralNetworks Part03 2013-10-22

7/25/2019 Scholastic Book NefghuralNetworks Part03 2013-10-22

1/16

Scholastic Video Book Series

Artificial Neural Networks

Part 3

(Back Propagation)

(with English Narrations)

http://scholastictutors.webs.com

(http://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part03-2013-10-22.pdf)

1

Scholastic Tutors (Oct, 2013)ISVT 911-0-20-131022-1

ARTIFICIAL NEURAL

NETWORKS
http://scholastictutors.webs.com/http://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part03-2013-10-22.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part03-2013-10-22.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part03-2013-10-22.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part03-2013-10-22.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part03-2013-10-22.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part03-2013-10-22.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part03-2013-10-22.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part03-2013-10-22.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part03-2013-10-22.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part03-2013-10-22.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part03-2013-10-22.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part03-2013-10-22.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part03-2013-10-22.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part03-2013-10-22.pdfhttp://scholastictutors.webs.com/


2/16

International Baccalaureate (IB)

2

Artificial Neural Networks - #3Back-Propagation


(ANN-003)

http://youtu.be/I2I5ztVfUSE

Click here to see the video
http://scholastictutors.webs.com/http://youtu.be/I2I5ztVfUSEhttp://youtu.be/I2I5ztVfUSEhttp://scholastictutors.webs.com/


3/16

Step 1:Calculate the local gradients (do1, do2, dh1 and dh2) for the nodes

in the network.

n1

n2

n3

n4

w31

y4

w41

b1

x1

x2

y3

do1b3

do2

dh1

dh2

1

( )1 exp

v

v

w32

w42

b2

w11

w21

w12

w22

1 = 1 3 1 31 2 32 (3 3)

2 = 1 4 1 41 2 42 (4 4)

1 = (1 1 1 11 2 12) 1 31 2 41

y1

y2

2 = (1 2 1 21 2 22) 1 32 2 42

= () 1 () +1

+1

+1


Back-Propagation Algorithm

b4+1

Desired = d3

Desired = d4
http://scholastictutors.webs.com/http://scholastictutors.webs.com/


4/16
http://scholastictutors.webs.com/


5/16

5

Lets perform a complete forward and backward sweep of the feedforward

network (2-2-1 architecture) shown below using the back propagation

algorithm discussed.Assume,

target output = 0.9,

= 0.25, and =0.0001

b1=0.1

Output = y3

w21= -0.1

Neuron 1

Neuron 2

Neuron 3

x1=0.1

x2=0.9

+1

b2= 0.1

+1

+1

w11= -0.2

w22= 0.3

w12= 0.1

w32= 0.3

w31= 0.2

b3= 0.2

y1

y2

v1

v2

v3

Also assume that the

activation function in all 3

neurons are sigmoid functions

given by:

1

( )1 exp

v

v

Back-Propagation - Example

dh1

dh2

do1



6/16

1 = 1 1 1 11 2 12 = 1 0.1 0.1 0.2 0.9 0.1 = 0.17

1 = 1 = 0.17 =

+exp(.)= 0.542

2 = 1 2 1 21 2 22 = 1 0.1 0.1 0.1 0.9 0.3 = 0.36

2 = 2 = 0.36 =

+exp(.)= 0.589

3 = 1 3 1 31 2 32 = 1 0.2 0.542 0.2 0.589 0.3 = 0.485

3 = 3 = 0.485 =

+exp(.)= 0.619

Therefore: = 3 3 = 0.9 0.619 = 0.281

Forward Pass



7/16

1 = 3 3 3 = 0.4851 0.281 = 0.4851 1 0.4851 0.281 = 0.619 1 0.619 0.281 = 0.0663

1 = 1 1 31 = 0.17 0.0663 0.2 = 0.17 1 0.17 0.01362 = 0.542 1 0.542 0.01362=0.0033

2 = 2 1 32 = 0.36 0.0663 0.3 = 0.36 1 0.36 0.01989 = 0.589 1 0.589 0.01989=0.0049

Backward Pass

1 = 0.17, 1 = 0.542

2 = 0.36, 2 = 0.589

3 = 0.485, 3 = 0.619

= 0.281



8/16

Backward Pass

1 = 0.542, 2 = 0.589

Step 2:Adjust the weights of the network using

the learning rule:

=

31 1 = 31 31 1 1 1

31 1 = 0.2 0.0001 0.2 0.25 0.0663 0.542 = 0.2090

32 1 = 32 32 1 1 2

32 1 = 0.3 0.0001 0.3 0.25 0.0663 0.589 = 0.3098

1 = 0.0663,

1 = 0.0033

2 = 0.0049

http://youtu.be/I2I5ztVfUSEClick here to see the video
http://youtu.be/I2I5ztVfUSEhttp://youtu.be/I2I5ztVfUSE


9/16

Backward Pass

1 = 0.542, 2 = 0.589


the learning rule:

=

11 1 = 11 11 1 1 1

11 1 = 0.2 0.0001 0.2 0.25 0.0033 0.1 = 0.1999

21 1 = 21 21 1 2 1

21 1 = (0.1) 0.0001 0.1 0.25 0.0049 0.1 = 0.0999

1 = 0.0663,

1 = 0.0033

2 = 0.0049



10/16

Backward Pass

1 = 0.542, 2 = 0.589


the learning rule:

=

12 1 = 12 12 1 1 2

12 1 = 0.1 0.0001 0.1 0.25 0.0033 0.9 = 0.1008

22 1 = 22 22 1 2 2

22 1 = 0.3 0.0001 0.3 0.25 0.0049 0.9 = 0.3011

1 = 0.0663,

1 = 0.0033

2 = 0.0049



11/16

Backward Pass

1 = 0.542, 2 = 0.589

Step 2:Adjust the weights of the network using the

learning rule:

=

3 1 = 3 3 1 1 1

3 1 = 0.2 0.0001 0.2 0.25 0.0663 1 = 0.2166

1 1 = 1 1 1 1 1

1 1 = 0.1 0.0001 0.2 0.25 0.0033 1 = 0.1008

3 1 = 3 3 1 2 1

3 1 = 0.1 0.0001 0.1 0.25 0.0049 1 = 0.1012

1 = 0.0663,

1 = 0.0033

2 = 0.0049



12/16

12

b1=0.1008

Output = y3

w21= -0.0999

Neuron 1

Neuron 2

Neuron 3

x1=0.1

x2=0.9

+1

b2= 0.1012

+1

+1

w11= -0.1999

w22= 0.3011

w12= 0.1008

w32= 0.3098

w31= 0.2090

b3= 0.2166

y1

y2

v1

v2

v3

After one Complete Forward &

Backward Pass

dh1

dh2

do1



13/16

1 = 0.17 1 = 0.1715

1 = 0.542 1 = 0.5428

2 = 0.36 2 = 0.3622

2 = 0.589 2 = 0.5896

3 = 0.4851 3 = 0.5127

3 = 0.619 3 = 0.6254

= 3 3 = 0.9 0.619 = 0.281 = 3 3 = 0.9 0.6254 =0.2746

After one Complete Forward &

Backward Pass



14/16

After second pass = 0.2683 After third pass = 0.2623

After forth pass = 0.2565 After 100 passes = 0.0693 After 200 passes = 0.0319 After 500 passes = 0.0038

Error is getting reduced after each pass.

After a few more Complete

Forward & Backward Passes



15/16

International Baccalaureate (IB)

15

Artificial Neural Networks - #3Back Propagation


(ANN-003)

END of the Book

If you like to see similar solutions to any Mathematics problems please

contact us at: [email protected] your request.

http://scholastictutors.webs.com/http://[email protected]/http://youtu.be/I2I5ztVfUSEhttp://youtu.be/I2I5ztVfUSEhttp://[email protected]/http://scholastictutors.webs.com/


16/16

Videos at: http://www.youtube.com/user/homevideotutor

16

(http://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part03-2013-10-22.pdf)

Scholastic Video Book Series

Artificial Neural Networks

Part 3

(Back Propagation)

(with English Narrations)

(END)

Scholastic Tutors (Sep, 2013)ISVT 911-0-20-131022-1
http://www.youtube.com/user/homevideotutorhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part03-2013-09-23.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part03-2013-09-23.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part03-2013-09-23.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part03-2013-09-23.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part03-2013-09-23.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part03-2013-09-23.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part03-2013-09-23.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part03-2013-09-23.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part03-2013-09-23.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part03-2013-09-23.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part03-2013-09-23.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part03-2013-09-23.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part03-2013-09-23.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part03-2013-09-23.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part03-2013-09-23.pdfhttp://www.youtube.com/user/homevideotutorhttp://www.youtube.com/user/homevideotutor

Scholastic Book NefghuralNetworks Part03 2013-10-22

Documents

Transcript of Scholastic Book NefghuralNetworks Part03 2013-10-22