Hebbian Learning Rule with Implementation of AND Gate
Last Updated :
26 Nov, 2020
Hebbian Learning Rule, also known as Hebb Learning Rule, was proposed by Donald O Hebb. It is one of the first and also easiest learning rules in the neural network. It is used for pattern classification. It is a single layer neural network, i.e. it has one input layer and one output layer. The input layer can have many units, say n. The output layer only has one unit. Hebbian rule works by updating the weights between neurons in the neural network for each training sample.
Hebbian Learning Rule Algorithm :
- Set all weights to zero, wi = 0 for i=1 to n, and bias to zero.
- For each input vector, S(input vector) : t(target output pair), repeat steps 3-5.
- Set activations for input units with the input vector Xi = Si for i = 1 to n.
- Set the corresponding output value to the output neuron, i.e. y = t.
- Update weight and bias by applying Hebb rule for all i = 1 to n:
Implementing AND Gate :
Truth Table of AND Gate using bipolar sigmoidal function
There are 4 training samples, so there will be 4 iterations. Also, the activation function used here is Bipolar Sigmoidal Function so the range is [-1,1].
Step 1 :
Set weight and bias to zero, w = [ 0 0 0 ]T and b = 0.
Step 2 :
Set input vector Xi = Si for i = 1 to 4.
X1 = [ -1 -1 1 ]T
X2 = [ -1 1 1 ]T
X3 = [ 1 -1 1 ]T
X4 = [ 1 1 1 ]T
Step 3 :
Output value is set to y = t.
Step 4 :
Modifying weights using Hebbian Rule:
First iteration -
w(new) = w(old) + x1y1 = [ 0 0 0 ]T + [ -1 -1 1 ]T . [ -1 ] = [ 1 1 -1 ]T
For the second iteration, the final weight of the first one will be used and so on.
Second iteration -
w(new) = [ 1 1 -1 ]T + [ -1 1 1 ]T . [ -1 ] = [ 2 0 -2 ]T
Third iteration -
w(new) = [ 2 0 -2]T + [ 1 -1 1 ]T . [ -1 ] = [ 1 1 -3 ]T
Fourth iteration -
w(new) = [ 1 1 -3]T + [ 1 1 1 ]T . [ 1 ] = [ 2 2 -2 ]T
So, the final weight matrix is [ 2 2 -2 ]T
Testing the network :
The network with the final weights
For x1 = -1, x2 = -1, b = 1, Y = (-1)(2) + (-1)(2) + (1)(-2) = -6
For x1 = -1, x2 = 1, b = 1, Y = (-1)(2) + (1)(2) + (1)(-2) = -2
For x1 = 1, x2 = -1, b = 1, Y = (1)(2) + (-1)(2) + (1)(-2) = -2
For x1 = 1, x2 = 1, b = 1, Y = (1)(2) + (1)(2) + (1)(-2) = 2
The results are all compatible with the original table.
Decision Boundary :
2x1 + 2x2 - 2b = y
Replacing y with 0, 2x1 + 2x2 - 2b = 0
Since bias, b = 1, so 2x1 + 2x2 - 2(1) = 0
2( x1 + x2 ) = 2
The final equation, x2 = -x1 + 1
Decision Boundary of AND Function
Similar Reads
Implementation of Teaching Learning Based Optimization
The previous article Teaching Learning Based Optimization (TLBO) talked about the inspiration of teaching learning-based optimization, it's mathematical modeling and algorithms. In this article we will implement Teaching learning-based optimization (TLBO) for two fitness functions 1) Rastrigin funct
7 min read
Genetic Algorithm for Reinforcement Learning : Python implementation
In reinforcement learning, the challenge is to find the best policy or the best set of parameters for a given environment. Genetic Algorithm (GA) is an optimization algorithm inspired by the process of natural evolution. It is used to find approximate solutions to complex problems by evolving a popu
7 min read
Implementing Deep Q-Learning using Tensorflow
Prerequisites: Deep Q-Learning This article will demonstrate how to do reinforcement learning on a larger environment than previously demonstrated. We will be implementing Deep Q-Learning technique using Tensorflow. Note: A graphics rendering library is required for the following demonstration. For
4 min read
Types Of Learning Rules in ANN
The learning rule enhances the Artificial Neural Network’s performance by applying this rule over the network. Learning in ANNs involves adjusting the weights of the connections between neurons to improve performance on a given task. This adjustment is governed by learning rules, which define how we
7 min read
Implementation of Artificial Neural Network for AND Logic Gate with 2-bit Binary Input
Artificial Neural Network (ANN) is a computational model based on the biological neural networks of animal brains. ANN is modeled with three types of layers: an input layer, hidden layers (one or more), and an output layer. Each layer comprises nodes (like biological neurons) are called Artificial N
4 min read
Implementation of Artificial Neural Network for OR Logic Gate with 2-bit Binary Input
Artificial Neural Network (ANN) is a computational model based on the biological neural networks of animal brains. ANN is modeled with three types of layers: an input layer, hidden layers (one or more), and an output layer. Each layer comprises nodes (like biological neurons) are called Artificial N
4 min read
ML | Reinforcement Learning Algorithm : Python Implementation using Q-learning
Prerequisites: Q-Learning technique. Reinforcement Learning is a type of Machine Learning paradigms in which a learning algorithm is trained not on preset data but rather based on a feedback system. These algorithms are touted as the future of Machine Learning as these eliminate the cost of collecti
6 min read
Reinforcement Learning in Python: Implementing SARSA Agent in Taxi Environment
SARSA (State-Action-Reward-State-Action) is an on-policy reinforcement learning algorithm that updates its policy based on the current state-action pair, the reward received, the next state, and the next action chosen by the current policy. In this article, we will implement SARSA in Gymnasium's Tax
8 min read
Implementation of Artificial Neural Network for NOR Logic Gate with 2-bit Binary Input
Artificial Neural Network (ANN) is a computational model based on the biological neural networks of animal brains. ANN is modeled with three types of layers: an input layer, hidden layers (one or more), and an output layer. Each layer comprises nodes (like biological neurons) are called Artificial N
4 min read