Artificial Neural Network (Back-Propagation Neural Network)
description
Transcript of Artificial Neural Network (Back-Propagation Neural Network)
Artificial Neural Network(Back-Propagation Neural Network)
Yusuf Hendrawan, STP., M.App.Life Sc., Ph.D
Neurons
http://faculty.washington.edu/chudler/color/pic1an.gif
http://research.yale.edu/ysm/images/78.2/articles-neural-neuron.jpg
Biological
Artificial
A typical AI agent
Neural Network Layers•Each layer receives its inputs from the previous layer and forwards its outputs to the next layer
http://smig.usgs.gov/SMIG/features_0902/tualatin_ann.fig3.gif
Multilayer feed forward networkIt contains one or more hidden layers (hidden neurons).“Hidden” refers to the part of the neural network is not seen directly from either input or output of the network . The function of hidden neuron is to intervene between input and output.By adding one or more hidden layers, the network is able to extract higher-order statistics from input
Neural Network LearningBack-Propagation Algorithm:
function BACK-PROP-LEARNING(examples, network) returns a neural network inputs: examples, a set of examples, each with input vector x and output vector y network, a multilayer network with L layers, weights Wj,i , activation function g
repeat for each e in examples do for each node j in the input layer do aj ‰ xj[e] for l = 2 to M do ini ‰ åj Wj,i aj
ai ‰ g(ini) for each node i in the output layer do Dj ‰ g’(inj) åi Wji Di
for l = M – 1 to 1 do for each node j in layer l do Dj ‰ g’(inj) åi Wj,i Di
for each node i in layer l + 1 do Wj,i ‰ Wj,i + a x aj x Di
until some stopping criterion is satisfied return NEURAL-NET-HYPOTHESIS(network)
[Russell, Norvig] Fig. 20.25 Pg. 746
Back-Propagation Illustration
ARTIFICIAL NEURAL NETWORKS Colin Fahey's Guide (Book CD)
X1
X2
Y
Vo Wo
Z1
Z2
Z3
Z4
Input (X) Hidden Output (Y)
X1 X20.3 0.40.5 0.60.2 0.30.4 0.7
Input (X) Output / Target (T)
T0.10.80.40.5
Jumlah Neuron pada Input Layer 2Jumlah Neuron pada Hidden Layer 4Jumlah Neuron pada Output Layer 1Learning rate (α) 0.1Momentum (m) 0.9Target Error 0.01Maximum Iteration 1000
Bobot Awal Input ke Hidden
V11 = 0.75
V21 = 0.35
V12 = 0.54
V22 = 0.64
V13 = 0.44
V23 = 0.05
V14 = 0.32
V24 = 0.81
Bias ke Hidden
Vo11 = 0.07 Vo21 = 0.12Vo12 = 0.91 Vo22 = 0.23Vo13 = 0.45 Vo23 = 0.85Vo14 = 0.25 Vo24 = 0.09
Bobot Awal Hidden ke Output
W1 = 0.04W2 = 0.95W3 = 0.33W4 = 0.17
Bias ke Output
Wo1 = 0.66Wo2 = 0.56Wo3 = 0.73Wo4 = 0.01
Menghitung Zin & Z dari input ke hiddenZin(1) = (X1 * V11) + (X2 * V21) = (0.3 * 0.75) + (0.4 * 0.35) = 0.302Zin(2) = (X1 * V12) + (X2 * V22) = (0.3 * 0.54) + (0.4 * 0.64) = 0.418Zin(3) = (X1 * V13) + (X2 * V23) = (0.3 * 0.44) + (0.4 * 0.05) = 0.152Zin(4) = (X1 * V14) + (X2 * V24) = (0.3 * 0.32) + (0.4 * 0.81) = 0.42
575.01
11
1302.0)1( )1(
eeZ
inZ
603.01
11
1418.0)2( )2(
eeZ
inZ
538.01
11
1152.0)3( )3(
eeZ
inZ
603.011
11
42.0)4( )4(
ee
ZinZ
Menghitung Yin & Y dari hidden ke outputYin = (Z(1) * W1) + (Z(2) * W2) + (Z(3) * W3) + (Z(4) * W4)
= (0.57 * 0.04) + (0.603 * 0.95) + (0.538 * 0.33) + (0.603 * 0.17)
= 0.876706.0
11
11
876.0
eeY
inY
Menghitung dev antara Y dengan output nyata
dev = (T - Y) * Y * (1 - Y) = (0.1 – 0.706) * 0.706 * (1 – 0.706) = -0.126Menghitung selisihselisih = T - Y= -0.606
Back-Propagation
Menghitung din dari output ke hiddendin(1) = (dev * W1) = (-0.126 * 0.04) = -0.00504din(2) = (dev * W2) = (-0.126 * 0.95) = -0.1197din(3) = (dev * W3) = (-0.126 * 0.33) = -0.04158din(4) = (dev * W4) = (-0.126 * 0.17) = -0.02142
Menghitung dd (1) = (din(1) * Z(1) * (1 - Z(1) ) = (-0.00504 * 0.575 * (1 – 0.575) = -0.00123d (2) = (din(2) * Z(2) * (1 - Z(2) ) = (-0.1197 * 0.603 * (1 – 0.603) = -0.02865d (3) = (din(3) * Z(3) * (1 - Z(3) ) = (-0.04158 * 0.538 * (1 – 0.538) = -0.01033d (4) = (din(4) * Z(4) * (1 - Z(4) ) = (-0.02142 * 0.603 * (1 – 0.603) = -0.00512
Mengkoreksi bobot (W) dan bias (Wo)
W1 = W1 + (α * dev * Z(1) ) + (m * Wo(1)) = 0.04 + (0.1 * -0.126 * 0.575) + (0.9 * 0.66) = 0.627 W2 = W2 + (α * dev * Z(2) ) + (m * Wo(2)) = 0.95 + (0.1 * -0.126 * 0.603) + (0.9 * 0.56) = 1.45 W3 = W3 + (α * dev * Z(3) ) + (m * Wo(3)) = 0.33 + (0.1 * -0.126 * 0.538) + (0.9 * 0.73) = 0.98 W4 = W4 + (α * dev * Z(4) ) + (m * Wo(4)) = 0.17 + (0.1 * -0.126 * 0.603) + (0.9 * 0.01) = 0.171 Wo1 = (α * Z(1) ) + (m * Wo(1)) = (0.1 * 0.575) + (0.9 * 0.66) = 0.65 Wo2 = (α * Z(2) ) + (m * Wo(2)) = (0.1 * 0.603) + (0.9 * 0.56) = 0.564 Wo3 = (α * Z(3) ) + (m * Wo(3)) = (0.1 * 0.538) + (0.9 * 0.73) = 0.71 Wo4 = (α * Z(4) ) + (m * Wo(4)) = (0.1 * 0.603) + (0.9 * 0.01) = 0.0693
Mengkoreksi bobot (V) dan bias (Vo)
V11 = V11 + (α * d (1) * X1 ) + (m * Vo(11)) = 0.75 + (0.1 * -0.00123 * 0.3) + (0.9 * 0.07) = 0.8129 V12 = V12 + (α * d (2) * X1 ) + (m * Vo(12)) = 0.54 + (0.1 * -0.02865 * 0.3) + (0.9 * 0.91) = 1.3581V13 = V13 + (α * d (3) * X1 ) + (m * Vo(13)) = 0.44 + (0.1 * -0.01033 * 0.3) + (0.9 * 0.45) = 0.8446 V14 = V14 + (α * d (4) * X1 ) + (m * Vo(14)) = 0.32 + (0.1 * -0.00512 * 0.3) + (0.9 * 0.25) = 0.5448 V21 = V21 + (α * d (1) * X2 ) + (m * Vo(21)) = 0.35 + (0.1 * -0.00123 * 0.4) + (0.9 * 0.12) = 0.4579 V22 = V22 + (α * d (2) * X2 ) + (m * Vo(22)) = 0.64 + (0.1 * -0.02865 * 0.4) + (0.9 * 0.23) = 0.8458 V23 = V23 + (α * d (3) * X2 ) + (m * Vo(23)) = 0.05 + (0.1 * -0.01033 * 0.4) + (0.9 * 0.85) = 0.8145 V24 = V24 + (α * d (4) * X2 ) + (m * Vo(24)) = 0.81 + (0.1 * -0.00512 * 0.4) + (0.9 * 0.09) = 0.8907
Mengkoreksi bobot (V) dan bias (Vo)
Vo11 = (α * d (1) * X1 ) + (m * Vo11) = (0.1 * -0.00123*0.3)+(0.9*0.07) = 0.0629Vo12 = (α * d (2) * X1 ) + (m * Vo12) = (0.1 * -0.02865*0.3)+(0.9*0.91) = 0.8181Vo13 = (α * d (3) * X1 ) + (m * Vo13) = (0.1 * -0.01033*0.3)+(0.9*0.45) = 0.4046Vo14 = (α * d (4) * X1 ) + (m * Vo14) = (0.1 * -0.00512*0.3)+(0.9*0.25) = 0.2248Vo21 = (α * d (1) * X2 ) + (m * Vo21) = (0.1 * -0.00123*0.4)+(0.9*0.12) = 0.1079Vo22 = (α * d (2) * X2 ) + (m * Vo22) = (0.1 * -0.02865*0.4)+(0.9*0.23) = 0.2058Vo23 = (α * d (3) * X2 ) + (m * Vo23) = (0.1 * -0.01033*0.4)+(0.9*0.85) = 0.7645Vo24 = (α * d (4) * X2 ) + (m * Vo24) = (0.1 * -0.00512*0.4)+(0.9*0.09) = 0.0807