Scribd
Upload
67 views

Unit - I Artificial Neural Networks

The document discusses the multi-layer perceptron (MLP) neural network model for realizing the XOR function. It shows how an additional hidden layer is needed since a single layer cannot linearly separate the XOR function. The network architecture and weight calculations for the hidden and output layers are presented to solve the XOR problem using two hidden units with sigmoid activation functions. Backpropagation is introduced as a learning algorithm for training multi-layer networks by minimizing the total squared error.

Uploaded by

Mary Morse
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
67 views

Unit - I Artificial Neural Networks

The document discusses the multi-layer perceptron (MLP) neural network model for realizing the XOR function. It shows how an additional hidden layer is needed since a single layer cannot linearly separate the XOR function. The network architecture and weight calculations for the hidden and output layers are presented to solve the XOR problem using two hidden units with sigmoid activation functions. Backpropagation is introduced as a learning algorithm for training multi-layer networks by minimizing the total squared error.

Uploaded by

Mary Morse
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 23

UNIT - I

ARTIFICIAL NEURAL
NETWORKS
5. Ex OR function using MP neuron model :-
The truth table for XOR functions is

x1 x2 y x
1

1 1 0
y
1 0 1
0 1 1 x
2
0 0 0

The threshold of unit y is 1. MP Model for XOR


function
With one layer done, it is not possible to predict the value of
the threshold for the neuron to fire, hence another layer is
introduced
x1 XOR x2 = (x1 ANDNOT x2) OR (x2 ANDNOT x1)
x1 XOR x2 = H1 OR H2 ; where H1 = x1 ANDNOT x2
H2 = x2 ANDNOT x1
The activation of H1 and H2 are
=1 ;H in-1 1
H=
1 =
0 Hin-1
;H in-1 <1
=1 ;H in-2 1
H=
i2 =
0 Hin-2
;H in-2 <1
The net i/p and activation of H1 and H2 are
H1 = x1 ANDNOT xi2 ; Hin-1 = x1w1+x2w2
= x1-x2 (w1=1,w2=-1)
X1 X2 Hin-1 H1
1 1 0 0
1 0 1 1
0 1 -1 0
0 0 0 0

H2 = x2 ANDNOT x1 ; Hin-2 = x1w1+x2w2


= -x1+x2 (w1 = -1,w2 = 1)

X1 X2 Hin-2 H2
1 1 0 0
1 0 -1 0
0 1 1 1
0 0 0 0
The activation for the o/p unit y = 1
y=f(yin) = 1 ; if yin 1
0 ; if yin < 1
Presenting i/p patterns H1 & H2 and calculating net i/p and
activations gives o/p of XOR.
yin = H1w1+H2w2
= H1+H2 (w1 = w2 = 1)
H1 H2 yin y=H1
(or) H2
0 0 0 0
1 0 1 1
0 1 1 1
0 0 0 0

Thus the XOR function is realized.


Linearly separable means, A functions with n i/p bits and
single o/p bit, a hyper lane exists there by dividing the i/p
points into groups in n- dimensional space out of which one
group contains 0 output and other 1 output.

Discus various learning techniques ? (Rules (or) Laws)


Learning (or) training is the term used to describe process of
finding values weights.
The classification of learning algorithms are given below
Artificial
network

Supervised Unsupervised Rein forced


learning learning learning

Error correction gradient Competiti


Stochastic Hebbian
descent ve

Least mean Back


square propagation
1. Hebbian Learning Rule
.It is also call as correlational learning.
.It can be split into two part rule.
i. If two neurons on either side of a synapse are activated
simultaneously, than the strength of that synapse is
selectively increased.
ii. If two neurons on either side of a synapse are activated
asynchronously, than the strength of that synapse is
selectively weakened or eliminated.
.It is purely feedforward ,unsupervised learning.
.In this rule, the I/p o/p pattern pairs (x y i) are associated
I,

by the
n weight matrix w known as correlation matrix
computed as.
.W= x iy i T
i=0
y i T = Transpose of the associated o/p
vector yi.
2. Perceptron leaning rule
.It is also known as Discrete Perceptron leaning law.
.For the perceptron learning rule, the leaning signal is the
difference between the desired and actual neurons response.
.It is supervised leaning.
.It is applicable only for bipolar o/p functions f(.).
.The preceptor leaning rule states that for a finite n no. of I/p
training vector x(n), each with an associate target value t(n)
which is +1 (or) -1, and an activation function.
1 ; if y in >
y=0 ; if y in
-1 ; if y in <
Then the weight updated is given by
wnew = w old+t x ; if y t
wnew = w old ; if y = t
Perceptron Training Algorithm :
i. Start with random value of w.
ii. Test for w.x i >0, if test succeed for i=1,2,..n, then return w.
iii. Modify w, as wnew = w prev+xfail.
Limitations of Perceptron :
1. Non-linear reparability is not passive i.e it can only model linearly
single perceptron.
2. Single perceptron does not have enough computing power
SOL : 1. Use larger network.
2 . Tolerate error.
Perceptron Leaning Algorithm :
x(n) = i/p vector
w(n) = weight vector
b(n) = bias
y(x) = actual response
d(n) = desired response
= learning rate parameter
i. Initialization :- Set w(0) = 0
ii. Activation :- Activate perceptron by applying i/p
iii. Complete actual response of perceptron
y(x) = sgn [wT(n).x(n)]
iv. Adapt weight vector i.e. if y(n) & d(n) are different, then
w(n+1) = w(n)+ [d(n)-y(n)].x(n)
+1 ; x(n) c1 c1 = class 1
where
-1d(n) = c2
; x(n) c2 = class 2
v. Continuation :- Increment step n by 1 and go to activation
step.
3. Delta Learning law :-
.It is valid only for continuous activation functions and
differentiable o/p function.
.It is supervised learning .
.It is also known as continuous perceptron leaning
It states that
The adjustment made to a synaptic weight of neuron is
proportional to the product of the error signal and the i/p signal
of the synapse.
Delta rule for signal o/p unit is that
it changes the weight of the connection to minimize the
difference the net i/p to the o/p unit y inand the target value
t.
i.e. w i = (t-y in)x i
Where x = the vector of activation of i/p units.
y in = net i/p to o/p unit i.e. xw i
t = target vector.
= learning rate.
Delta rule for several o/p units is that,
wjk = (tj-yinj)xi
4. Competitive Learning Rule :-
5. Outstar Leaning Rule :-
.It is also known as gross berg leaning.
.It is supervised leaning .
.It is used to provide learning of repetitive and characteristic
properties of i/p o/p relationship.
.The weight matrix
(yk-wjk) ; if neuron j wins the competition.
0 ; if neuron j losses the competition.
wjk =
6. Boltzmann leaning :-
.It is also known as stochastic leaning.
.Here the weights are adjusted in a probabilistic fashion.
.Used in symmetric recurrent network.(i.e. symmetric : wij=
wji )
.Consist of binary units (+1 for on, -1 for off )
.Neurons are divided into two groups i.e. hidden & visible.
Compare supervised and unsupervised learning ?
Learning (or) Training is term used to describe process of
finding values of weights.

S.n Supervised Unsupervised


o
1 It depends on local and It depends on only local
external information. information.
2 External supervisor is External supervisor is not
required. required.
3 Leaning phase & operation Leaning and operates at
phase are distinct i.e. same time i.e. preferred on-
preferred off line leaning. line leaning.
4 Comparison is made between It does not require target
actual o/p & desired o/p and o/p is no comparison of o/p
desired o/p is compared with is made with desired o/p i.e.
target o/p i.e. it has o/p & i/p. it has only i/ps.
S.n Supervised Unsupervised
o

input Adaptive output


network input Adaptive output
5 network
Network Leanin
leaning g
algorithm signal
Compare single layer and multilayer ANNs ?
S. Single layer ANN Multi layer ANN
No
1 Not suitable for learning Suitable for learn large &
large & move complex more complex model.
model.
2 No hidden layers. Hidden layers present.
3 It is not an auto It is an auto associative
associative network. network.
4 No feedback. Feedback is present.
5 Cannot solve linear Solve linear separability
separability problems. problems.
6 Processing stage is not Processing stage is made of
made of neurons. neurons.
S. Single layer ANN Multi layer ANN
No
8 Connections with in a Connections within a
network are direct. network are not direct, but
with help of hidden layers.
9 Hard limiting function is Sigmoid function is used as
used as activation function activation function.
10 It makes use of Delta Rule It makes use of Back
as learning rule. Propagation in Algorithm as
learning Algorithm.
Basic Propagation Algorithm :-
It is also known as generalized delta rule.
The total squared error of the output computed by net is minimized
by a gradient descent method known as back propagation or
generalized delta rule.
It is used for training multi-layer forward network.
Network is trained by supervised learning method.
The architecture of back propagation network resembles a
multilayered feed forward network.
A multilayer feed forward back propagation network with one layer
and two hidden units and the output units and hidden units have
bias. The bias acts like a weights on connections from units whose
output is always 1. The input layer connected to hidden layer and
hidden layer connected to output layer by means of inter
connections weights. These can be used any no of hidden layers .
The training algorithm back propagation involves four stages
i.e .
1. Initialization of weights
2. Feed forward
3. back propagation of error
4. Updation of weights of and biases
.The various parameters used in training algorithm are
.x = input training vector = (x1,x2x1,..xn)
.T = output target vector = (t1,tk,tm)
. k = error at output unit yk
. j = error at output unit yk
. = learning rate
Voj = bias on hidden unit j.
Zj = hidden unit j.
wok= bias on output unit k.
yk = output unit k.

You might also like