Faculty of Computing
CS-405 Deep Learning
BS-EE
Lab 2
CLO 3, CLO 4
Date: 5-02-2025
Name: Muhammad Zohaib Irfan
Lab Engineer: Ms. Areeba Rameen
Instructor: Ms. Huma Ameer
Objective
In this lab, you will:
● Implement the classic perceptron learning algorithm using PyTorch.
● Train the perceptron on a linearly separable dataset.
● Design perceptrons for an XOR dataset.
● Explore the applicability of perceptrons on a dataset of your choice.
● Answer key conceptual questions about perceptrons.
Task 1: Implement a Classic Perceptron in PyTorch
(from scratch)
Instructions
● Create a Perceptron class in PyTorch using [Link].
● Implement the step function as the activation function (no sigmoid, just
thresholding).
● Train the perceptron using weight updates based on the Perceptron Learning
Rule:
● Do not forget the bias term!
● Train the perceptron on the AND function.
● Print weight updates after each training sample.
● After training, visualize the decision boundary to confirm learning.
Code and output:
import torch
import [Link] as nn
import [Link] as optim
import numpy as np
import [Link] as plt
class Perceptron([Link]):
def __init__(self, input_size):
super(Perceptron, self).__init__()
[Link] = [Link]([Link](input_size + 1) * 0.01)
def forward(self, x):
x = [Link](([Link]([Link][0], 1), x), dim=1)
return [Link]([Link](x, [Link]) >= 0, 1, 0)
#and gate input data would look something like this
X = [Link]([[0, 0], [0, 1], [1, 0], [1, 1]], dtype=torch.float32)
y = [Link]([0, 0, 0, 1], dtype=torch.float32)
# Training the perceptron
perceptron = Perceptron(input_size=2)
learning_rate = 0.1
for epoch in range(8): # We can write epochs here
total_loss = 0
for i in range(len(X)):
x_sample = X[i].unsqueeze(0)
y_sample = y[i]
prediction = perceptron(x_sample).squeeze()
error = y_sample - prediction
with torch.no_grad():
[Link] += learning_rate * error *
[Link](([Link]([1.0]), x_sample.squeeze()))
print(f"Sample {X[i].tolist()}, Prediction: {[Link]()},
Error: {[Link]()}, Weights: {[Link]()}")
# now to plot decision boundary
def plot_decision_boundary(perceptron, X, y):
x_min, x_max = -0.5, 1.5
y_min, y_max = -0.5, 1.5
xx, yy = [Link]([Link](x_min, x_max, 100), [Link](y_min,
y_max, 100))
grid = [Link](np.c_[[Link](), [Link]()], dtype=torch.float32)
Z = perceptron(grid).reshape([Link])
[Link](xx, yy, [Link]().numpy(), levels=[-1, 0.5, 1], alpha=0.3,
colors=['red', 'blue'])
[Link](X[:, 0], X[:, 1], c=y, edgecolors='k', cmap=[Link])
[Link]('X1')
[Link]('X2')
[Link]('Perceptron Decision Boundary for AND Gate')
[Link]()
plot_decision_boundary(perceptron, X, y)
Task 2: Implement XOR through Perceptrons
🛠 Instructions
● Modify your perceptron code to train on the XOR dataset:
● Train the perceptron using the same Perceptron Learning Rule.
● Observe the final predictions and weight updates.
● Plot the decision boundary and analyze whether it correctly classifies XOR.
Code:
import torch
import numpy as np
import [Link] as plt
# Perceptron for XOR
class Perceptron:
def __init__(self, input_size, learning_rate=0.1):
[Link] = [Link](input_size + 1) * 0.01
self.learning_rate = learning_rate
def step_function(self, x):
return 1 if x >= 0 else 0
def predict(self, x):
x = [Link](([Link]([1.0]), x))
return self.step_function([Link]([Link], x))
def train(self, X, y, epochs=6):
for epoch in range(epochs):
for i in range(len(X)):
x_sample = X[i]
y_sample = y[i]
prediction = [Link](x_sample)
error = y_sample - prediction
[Link] += self.learning_rate * error *
[Link](([Link]([1.0]), x_sample))
print(f"Sample {X[i].tolist()}, Prediction: {prediction},
Error: {error}, Weights: {[Link]()}")
X = [Link]([[0, 0], [0, 1], [1, 0], [1, 1]], dtype=torch.float32)
y = [Link]([0, 1, 1, 0], dtype=torch.float32) # XOR function labels
perceptron = Perceptron(input_size=2)
[Link](X, y)
def plot_decision_boundary(perceptron, X, y):
x_min, x_max = -0.5, 1.5
y_min, y_max = -0.5, 1.5
xx, yy = [Link]([Link](x_min, x_max, 100), [Link](y_min,
y_max, 100))
grid = [Link](np.c_[[Link](), [Link]()], dtype=torch.float32)
Z = [Link]([[Link](point) for point in
grid]).reshape([Link])
[Link](xx, yy, [Link](), levels=[-1, 0.5, 1], alpha=0.3,
colors=['red', 'blue'])
[Link](X[:, 0], X[:, 1], c=y, edgecolors='k', cmap=[Link])
[Link]('X1')
[Link]('X2')
[Link]('Perceptron Decision Boundary for XOR Gate')
[Link]()
plot_decision_boundary(perceptron, X, y)
Output:
Results from different epochs:
Task 3: Apply Perceptron to a Dataset of Your Choice
Instructions
● Think of a use-case you want to solve.
● Generate a small synthetic dataset using a Large Language Model (LLM) or
choose a simple dataset that interests you.
● Check if a perceptron can solve the problem:
○ Train a single-layer perceptron on the dataset.
○ Evaluate its performance.
○ If it fails, analyze why.
● Show your working in code, including dataset generation, training, and evaluation.
● Discuss whether the problem is linearly separable and explain your findings.
Code:
Ok so I am taking a random dataset but its arranged in a circle, again it will not be a linear
data and we will see what happens, obviously a simple line wont be able to do it
import torch
import numpy as np
import [Link] as plt
# Perceptron Implementation for Custom Dataset
class Perceptron:
def __init__(self, input_size, learning_rate=0.1):
[Link] = [Link](input_size + 1) * 0.01 # Including bias
self.learning_rate = learning_rate
def step_function(self, x):
return 1 if x >= 0 else 0 # Simple threshold activation
def predict(self, x):
x = [Link](([Link]([1.0]), x)) # Add bias term
return self.step_function([Link]([Link], x))
def train(self, X, y, epochs=6):
for epoch in range(epochs):
for i in range(len(X)):
x_sample = X[i]
y_sample = y[i]
prediction = [Link](x_sample)
error = y_sample - prediction
[Link] += self.learning_rate * error *
[Link](([Link]([1.0]), x_sample))
print(f"Sample {X[i].tolist()}, Prediction: {prediction},
Error: {error}, Weights: {[Link]()}")
def generate_circle_data(n_samples=100):
X = [Link]((n_samples, 2)) * 2 - 1
y = [Link]([(1 if x[0]**2 + x[1]**2 < 0.5**2 else 0) for x in X],
dtype=torch.float32)
return X, y
X, y = generate_circle_data(100)
perceptron = Perceptron(input_size=2)
[Link](X, y)
def plot_decision_boundary(perceptron, X, y):
x_min, x_max = -1.1, 1.1
y_min, y_max = -1.1, 1.1
xx, yy = [Link]([Link](x_min, x_max, 100), [Link](y_min,
y_max, 100))
grid = [Link](np.c_[[Link](), [Link]()], dtype=torch.float32)
Z = [Link]([[Link](point) for point in
grid]).reshape([Link])
[Link](xx, yy, [Link](), levels=[-1, 0.5, 1], alpha=0.3,
colors=['red', 'blue'])
[Link](X[:, 0], X[:, 1], c=y, edgecolors='k', cmap=[Link])
[Link]('X1')
[Link]('X2')
[Link]('Perceptron Decision Boundary (Inside vs. Outside Circle)')
[Link]()
plot_decision_boundary(perceptron, X, y)
Changing epochs size:
We tried to train a single-layer perceptron to classify points as inside or outside a circle. The
perceptron learned something, but it failed to correctly separate the data. The perceptron failed
because it can only learn linear decision boundaries, but our dataset requires a nonlinear one
(a circle).
Task 4: Answer These Conceptual Questions
Write a short reflection answering these questions:
1. Does the perceptron successfully classify the AND function? Why or why not?
Yes. The AND function is linearly separable, so a perceptron can find a straight-line
decision boundary to classify it correctly.
2. Why does a single-layer perceptron fail to learn XOR?
Because XOR is not linearly separable. A single-layer perceptron can only learn straight-
line boundaries, but XOR requires a more complex separation.
3. What type of problems can a perceptron solve?
Linearly separable problems like AND, OR, and simple classification tasks where a
single straight line can split the classes
4. What modifications would allow a model to solve XOR?
Use a Multi-Layer Perceptron. Adding a hidden layer allows the model to learn complex,
nonlinear decision boundaries.
Following is output after adding multi layers and finally solving it for XOR too.
Submission Guidelines
● Your code should be written in PyTorch.
● Ensure that you print weight updates at each step.
● Include a decision boundary plot for both AND and XOR.
● Submit a brief explanation answering the conceptual questions.
Hints
● Remember to add a bias term to your input features!
● Use a step function (thresholding) for activation, not sigmoid.
● The perceptron should update weights after every training sample, not after the
entire dataset.
● Try visualizing the decision boundary to understand model behavior.
Important Note:
● Copied labs will be marked zero.
○ All submissions must reflect your own understanding and effort.
Deadline:
● Submission Deadline for this task is 11:00 am Thursday-(6-Feb-2025)
If you have any questions or need assistance, please reach out to the lab engineer.
